Retrieval from source document to answer

The mechanism

Retrieval converts a source collection into an index and selects material for a particular query. Chunk boundaries, identifiers and metadata determine which evidence can be returned. A lexical index matches terms; a dense index uses learned representations. Their scores are different objects, and neither is automatically a calibrated probability that an answer is correct. Reranking, evidence selection and answer generation or extraction are further stages. The current lexical application exposes TF–IDF and a deterministic selector, including a case where the correct passage is retrieved but the wrong entity’s fact is chosen. A neural retrieval-and-generation pipeline requires actual learned artifacts and recorded outputs; the original RAG paper supplies method context, not evidence that the current lexical example implements that model.

1. Documents and chunks retain stable source identities, versions and task-relevant metadata.
2. An index maps each query to a ranked candidate set under a declared lexical or learned scoring method.
3. Relevance labels evaluate retrieval separately from the answer selector or generator.
4. Task correctness, source support and justified abstention are scored with their own denominators.

Science and assumptions

Engineering example

Suppose two chunks are relevant and the top three results contain one of them. Recall at three is 1/2, regardless of the third result’s high similarity score. On a separate five-question answer set, four questions receive answers and all four answers are supported by their cited passages, but only three answer the requested entity correctly. Source support among answers is 4/4 while task accuracy over all questions is 3/5. Better retrieval recall may help, yet it cannot by itself repair a selector that uses the wrong entity.

Distinctions and tradeoffs

A relevant source or a high similarity score does not establish a correct answer.

Larger contexts may improve evidence recall while adding irrelevant material and increasing downstream interpretation cost.

Grounding is a pipeline of inspectable decisions, not simply attaching a citation to generated text.

Specifications and interfaces

Mechanism boundary

Which error came from indexing, ranking or answer selection?

Reference relevance and task labels are fixed independently of the method. Empty relevance sets require a separate convention rather than division by zero.

Evidence type

Authored worked example

Worked explanation with a bounded numeric or state example. This lesson does not claim an executing implementation of an external model or platform.

Interfaces

Declared input and state Input

Documents and chunks retain stable source identities, versions and task-relevant metadata.

Checked outcome Output

Locate indexing, ranking and answer-selection errors and calculate each evaluation denominator correctly.

Check your understanding

Evidence and scope

This is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. Reference relevance and task labels are fixed independently of the method. Empty relevance sets require a separate convention rather than division by zero.

The worked example explains this mechanism. General numerical labs are a reference; they do not execute this lesson’s full mechanism.

Sources

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

Published lesson data · Atlas releases and licensing

Continue exploring

Foundations

• Embedding table
• Evaluation & the AGI frontier
• Tools, memory & agent loop