Your next step
Why responses take time and cost money
Prefill, decode, caches, memory limits and end-to-end budgets.
Start learningWhat you’ll learn
Prefill reads the input
Identify processing that happens before the first generated token.
About 4 minutes · Open activity
Decode generates the continuation
Distinguish generation steps from input processing.
About 4 minutes · Open activity
A KV cache reuses attention work
Explain why stored keys and values can avoid repeated computation.
About 4 minutes · Open activity
Prompt caching and answer caching differ
Distinguish reused prefix computation from reusing a previous response.
About 4 minutes · Open activity
Memory capacity can limit a run
Identify weights and temporary state as separate memory demands.
About 4 minutes · Open activity
A local interface can call a remote model
Trace where inference and tools actually run.
About 4 minutes · Open activity
Budget the whole task
Account for input, output, tool work and retries.
About 4 minutes · Open activity
Read a request receipt
Locate prefill, decode, cache use and repeated work in two fictional receipts, and explain the tradeoff without naming a winning model.
About 5 minutes · Open activity