Your next step
What happens while an answer is generated
Distributions, greedy, sampling, temperature, top-p, caching.
Start learningWhat you’ll learn
The next piece has a distribution
Read a supplied set of candidate-token probabilities.
About 4 minutes · Open activity
Greedy choice takes the top candidate
Recognize greedy decoding.
About 4 minutes · Open activity
Sampling allows other likely candidates
Explain why the same starting context can yield variation.
About 4 minutes · Open activity
Temperature reshapes the distribution
Interpret a temperature change in a fixed illustrative sampling setup.
About 4 minutes · Open activity
Top-p limits the candidate pool
Explain nucleus sampling using a supplied cumulative share.
About 4 minutes · Open activity
Examples in a prompt can guide a response
Distinguish in-context adaptation from fine-tuning.
About 4 minutes · Open activity
Reuse earlier computation
Explain the purpose of an inference cache at a high level.
About 4 minutes · Open activity
More attempts need a useful selection rule
Explain the limit of inference-time candidate search.
About 4 minutes · Open activity
Explain two different replies
Use a fixed-model scenario to identify sampling, prompt examples and extra checking as different causes of output change.
About 5 minutes · Open activity