Chapter C37 · Interactive Course
Give a model useful outside information
Finding, chunking, ranking, grounding, freshness and permission.
Core Learning Concept
Trace the answer pipeline from document retrieval to generated claims. Determine whether failures stem from access permissions, search quality, or model generation, remembering that an existing document helps only if successfully retrieved.
What You’ll Learn in This Chapter
-
Retrieval finds material; generation writes with it
Separate the two operations in a RAG system.
Retrieval-Augmented Generation pairs information retrieval with language generation. An external retrieval system fetches relevant documents from a knowledge base, supplying them to the language model as explicit context for grounded answering.
-
Split documents without losing meaning
Identify a bad document boundary for retrieval.
Document chunks are pieces that a retrieval system can select. Their boundaries influence which facts and qualifications arrive together. Keep definitions, rules, and exceptions connected where possible, and preserve source references. Small chunks are not automatically better if they remove the context needed to interpret them.
-
An index helps find candidate material
Explain the role of a searchable document index.
An index organizes information to make candidate lookup efficient. It may support keywords, vectors, metadata filters, or combinations. Its job is to help locate material and connect it back to sources. The index does not by itself decide whether a passage answers the question or whether its claims are true.
-
Similarity search is not exact keyword search
Recognize why related wording can retrieve a passage.
Dense vector retrieval matches query semantics rather than exact keyword overlap. Comparing learned embeddings surfaces conceptually relevant passages effectively, even when users and authors express ideas using entirely different vocabulary words.
-
Select the passages that actually help
Distinguish initial retrieval from relevance review or reranking.
Candidate retrieval is often followed by reranking to select passages that genuinely answer the user’s question. A high similarity score indicates topical closeness, but reranking verifies whether necessary constraints are satisfied.
-
The answer still needs source support
Check whether a generated claim follows from supplied passages.
A grounded answer needs support for each important claim in the supplied sources. A citation indicates where to check; it does not guarantee entailment. Compare the exact statement with the passage and preserve conditions. If the source does not establish the detail, state the limit or retrieve additional evidence.
-
Freshness and permission belong in retrieval
Identify a passage that should not be available for this request.
Retrieval should respect who is asking and whether the information is current. Apply access controls before private material enters the model’s context. A relevant document is not automatically an authorized document. Track source versions or dates so stale material can be identified and refreshed.
-
Choose context, retrieval or training for the need
Select the appropriate kind of change in a supplied scenario.
Choose the intervention by the need. Context can supply information for one task; retrieval can locate current or large collections; training can adapt learned behavior. These approaches can be combined. Changing facts often favor an updateable source, but no single rule replaces evaluating quality, permissions, maintenance, and cost.
-
Find the broken link
Diagnose one missing passage, one stale source and one unsupported generated claim in a fictional help-desk answer.