# Retrieval from source document to answer

Canonical: https://semiaifoundry.com/learn/atlas/retrieval-indexing-evaluation/
Revision: 2026-10-07
Rights: https://semiaifoundry.com/ai-atlas/licenses.html

Models become evaluated applications · Atlas v1.3.0

# Retrieval from source document to answer

Chunking, lexical versus dense retrieval, provenance, reranking and answer validation are distinct stages.

[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#retrieval-indexing-evaluation)Logical state, task evidence and declared units; no physical assembly scale

Learning objectiveLocate indexing, ranking and answer-selection errors and calculate each evaluation denominator correctly.

## The mechanism

Retrieval converts a source collection into an index and selects material for a particular query. Chunk boundaries, identifiers and metadata determine which evidence can be returned. A lexical index matches terms; a dense index uses learned representations. Their scores are different objects, and neither is automatically a calibrated probability that an answer is correct. Reranking, evidence selection and answer generation or extraction are further stages. The current lexical application exposes TF–IDF and a deterministic selector, including a case where the correct passage is retrieved but the wrong entity’s fact is chosen. A neural retrieval-and-generation pipeline requires actual learned artifacts and recorded outputs; the original RAG paper supplies method context, not evidence that the current lexical example implements that model.

1.  Documents and chunks retain stable source identities, versions and task-relevant metadata.
2.  An index maps each query to a ranked candidate set under a declared lexical or learned scoring method.
3.  Relevance labels evaluate retrieval separately from the answer selector or generator.
4.  Task correctness, source support and justified abstention are scored with their own denominators.

## Science and assumptions

### Retrieval from source document to answer · explicit relationship

$$
\operatorname{Recall@}k=\frac{|R_k\cap G|}{|G|},\qquad A_{\mathrm{task}}=\frac{N_{\mathrm{correct}}}{N_{\mathrm{questions}}}
$$

Retrieval recall counts retrieved relevant items over all relevant items; task accuracy counts correct outcomes over all evaluated questions.

Symbols and units

Rₖ

Top-k retrieved item identifiers

set

G

Reference relevant item identifiers

nonempty set

Ncorrect

Task-correct outcomes, including any correctly required abstentions

count

Nquestions

All questions in the declared evaluation set

positive count

Atask

Task accuracy on that set

0–1

Suppose two chunks are relevant and the top three results contain one of them. Recall at three is 1/2, regardless of the third result’s high similarity score. On a separate five-question answer set, four questions receive answers and all four answers are supported by their cited passages, but only three answer the requested entity correctly. Source support among answers is 4/4 while task accuracy over all questions is 3/5. Better retrieval recall may help, yet it cannot by itself repair a selector that uses the wrong entity.

Assumptions Reference relevance and task labels are fixed independently of the method. Empty relevance sets require a separate convention rather than division by zero.

## Engineering example

Suppose two chunks are relevant and the top three results contain one of them. Recall at three is 1/2, regardless of the third result’s high similarity score. On a separate five-question answer set, four questions receive answers and all four answers are supported by their cited passages, but only three answer the requested entity correctly. Source support among answers is 4/4 while task accuracy over all questions is 3/5. Better retrieval recall may help, yet it cannot by itself repair a selector that uses the wrong entity.

## Distinctions and tradeoffs

A relevant source or a high similarity score does not establish a correct answer.

Larger contexts may improve evidence recall while adding irrelevant material and increasing downstream interpretation cost.

Grounding is a pipeline of inspectable decisions, not simply attaching a citation to generated text.

## Specifications and interfaces

Mechanism boundary

Which error came from indexing, ranking or answer selection?

Reference relevance and task labels are fixed independently of the method. Empty relevance sets require a separate convention rather than division by zero.

Evidence type

Authored worked example

Worked explanation with a bounded numeric or state example. This lesson does not claim an executing implementation of an external model or platform.

### Interfaces

Declared input and state Input

Documents and chunks retain stable source identities, versions and task-relevant metadata.

Checked outcome Output

Locate indexing, ranking and answer-selection errors and calculate each evaluation denominator correctly.

## Check your understanding

### Four supported answers contain three task-correct answers among five questions. What is task accuracy?

1.  100%
2.  75%
3.  60%

Answer and reasoning

60%

Task accuracy uses all five questions, including unanswered or incorrect cases; it is 3/5.

## Evidence and scope

This is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. Reference relevance and task labels are fixed independently of the method. Empty relevance sets require a separate convention rather than division by zero.

The worked example explains this mechanism. General numerical labs are a reference; they do not execute this lesson’s full mechanism.

## Sources

-   [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401)
-   [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://arxiv.org/abs/2406.12045)

[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/retrieval-indexing-evaluation-07d7a69b371fc1e4.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)

## Continue exploring

### Foundations

-   [Embedding table](https://semiaifoundry.com/learn/atlas/embedding/)
-   [Evaluation & the AGI frontier](https://semiaifoundry.com/learn/atlas/evaluation/)
-   [Tools, memory & agent loop](https://semiaifoundry.com/learn/atlas/tool-loop/)