RAG Assistant

Overview

A knowledge assistant that answers questions from a document corpus rather than from model memory alone. The system retrieves candidate passages, assembles them with the running conversation, and generates responses under explicit control constraints. The work covers the retrieval pipeline, knowledge grounding, conversation context, response control, and system evaluation.

Problem

Question answering over a private document set fails in predictable ways. Without retrieval, answers drift from the source material. With naive retrieval, top results match the query’s vocabulary rather than its intent, so the wrong passages reach the generator. Follow-up questions add another failure mode: pronouns and ellipses resolve against earlier turns, so a query must be resolved against conversation context before retrieval is meaningful. An open-ended generator will also state more than the sources support.

Solution

A staged pipeline. The incoming question is resolved against conversation memory so follow-ups stand alone. Retrieval uses vector search over embedded passages to return a candidate set. Reranking reorders that set against the full query, promoting passages that answer the question rather than ones that merely share terms. Context assembly selects what enters the prompt and marks source boundaries so claims can be attributed. Response generation is constrained to the assembled context, and response control checks the output before it reaches the user.

Architecture

Each stage has one job and one failure mode to reason about.

User Question
  ↓
Conversation Context
  ↓
Retrieval
  ↓
Reranking
  ↓
Context Assembly
  ↓
Response Generation
  ↓
Response Control
  ↓
Evaluation

Engineering Challenges

Recall versus precision. Vector search is built to cast a wide net; reranking exists because that first cut is noisy. The stages stay separate so each can be tuned against its own failure mode.

Grounding without over-restriction. Context assembly must include enough evidence to answer the question while keeping out passages that pull generation off-topic. Source boundaries stay explicit so grounding can be checked rather than assumed.

Conversation context has a budget. Memory carries earlier turns forward, but retrieval should not run against the whole dialogue. Query resolution compresses what matters from the history into the current retrieval request.

Response control is a stage, not a prompt instruction. The pipeline checks that generated claims trace back to assembled context and that refusal triggers when the sources are silent.

Technology

RAG ties the stages together. Vector Search and Retrieval handle candidate generation. Reranking orders candidates against the resolved query. Conversation Memory carries dialogue state across turns. Evaluation examines the behaviour of every stage.

Current Status

Active development, and the work is ongoing. Current effort sits at the pipeline level: tightening query resolution so follow-up questions retrieve the right material, tuning the split between retrieval recall and reranking precision, and shaping context assembly so grounding stays checkable. Response control is being specified against concrete failure cases rather than left to generation. Evaluation is a first-class concern — retrieval relevance, knowledge grounding, and response control behaviour are examined qualitatively as the pipeline evolves. Nothing here is shipped or running in production.