Chapter C92 · Interactive Course
Use it: budget a long document
Fit a long source into a small window, honestly.
Core Learning Concept
Plan how to answer one question from a source larger than the available context. Measure with the relevant tokenizer, select passages and their exceptions, split at meaningful boundaries, and record consequential omissions. Then check the answer against what was actually supplied rather than against the document you wish the model had seen.
What You’ll Learn in This Chapter
-
Measure before you send
Estimate the number of pieces a source will need.
Measure the text in the units the model limits: tokens under its tokenizer. Page count and word count give only rough clues. Include the surrounding instructions and required output allowance when calculating remaining space. Measurement lets you choose an excerpt or retrieval strategy before a request fails or loses context.
-
What fits, what waits
Choose which parts of a source to include first.
Selection starts from the question. Include material that directly answers it and nearby definitions or exceptions that could change the answer. Defer unrelated sections rather than filling the window simply because space remains. A focused excerpt is easier to inspect and can make omissions explicit.
-
Split without breaking meaning
Divide a source at a place that preserves sense.
Chunking divides a source into smaller pieces. A useful boundary preserves enough context to understand each piece, especially rules and their exceptions. Modest overlap can help at boundaries but does not guarantee completeness. Inspect the resulting chunks rather than choosing size alone as the quality measure.
-
Say what you left out
Record the omission that could change an answer.
An omission note states what relevant material was not included and how that limits the answer. If membership tiers may change a return period, say that those rules were not supplied. This makes uncertainty actionable: the next step is to obtain the missing section, not to repeat the same confident answer.
-
Check the answer used it
Verify a response against the part you actually supplied.
Providing an excerpt does not prove the answer used it faithfully. Compare each important claim with a supporting passage in the actual input. Mark claims that require omitted material or outside knowledge. A carefully chosen small source set makes this check manageable and exposes unsupported additions.
-
Apply it: budget a long document
Carry out the whole decision once: plan how to fit a long source into a limited context, and say what was left out.
-
Explain why context needs a better route
Reconstruct text pieces, read a toy representation, and find the distant information a small summary missed.