Chapter C36 · Interactive Course
Why models become more capable
Data, capacity, compute, synthetic examples and answer-time work.
Core Learning Concept
Explain each claimed improvement by identifying what changed: data, capacity, training, supplied information, tools, or work at answer time. Then ask for a comparison that tests that explanation. Include regressions and resource costs so “better” describes a useful outcome under stated conditions.
What You’ll Learn in This Chapter
-
Better data can matter more than more copies
Identify a quality improvement in a training collection.
Training data quality encompasses accuracy, diversity, and task coverage. Removing duplicate samples and correcting mislabeled rows provides far clearer learning signals than simply scaling raw row counts with redundant examples.
-
More capacity changes what can be learned
Interpret parameter count as capacity rather than an intelligence score.
Model capacity concerns what the architecture and parameters can represent, but parameter count is only one indicator. More parameters do not prove that useful structure was learned. Data, optimization, architecture, and inference conditions also matter. Evaluate the capability needed rather than converting size directly into a quality ranking.
-
Compute is work done by hardware
Recognize the physical resource behind training computation.
Compute measures total arithmetic operations executed by hardware computing devices during training or inference. Distinguishing floating-point operations from hardware device-hours clarifies the true physical resource investment behind a model run.
-
Balance capacity, data and training work
Explain why scale involves tradeoffs under a budget.
A fixed training budget requires choices among model size, data volume, and optimization work. A larger model trained insufficiently may lose to a smaller well-trained one. Published scaling relationships describe particular settings, not permanent ratios for every task. Include later serving costs when choosing a practical system.
-
Synthetic examples need a quality filter
Recognize useful and harmful roles for generated training material.
Synthetic data is generated rather than directly collected from the target source. It can expand useful examples when quality, diversity, and relevance are checked. Unfiltered generation can also repeat mistakes and narrow the distribution. Evaluate the resulting model on independent cases rather than judging the synthetic examples only by appearance.
-
Learning to solve can differ from learning to sound helpful
Distinguish reasoning-oriented training from surface response preference.
Training objectives influence what improves. Preference for a convincing explanation is different from passing a reliable task check, though both can contribute. Evaluate correctness and communication separately when needed. A model that sounds more helpful has not thereby demonstrated better problem solving on fresh cases.
-
Extra work at answer time is another lever
Distinguish inference-time search or verification from retraining.
Inference-time search generates multiple candidate completions in parallel and filters them using automated verifiers. This allocates additional compute during generation to improve reasoning accuracy without modifying underlying trained model weights.
-
Different gains have different evidence
Identify whether a claimed improvement concerns the model or the system around it.
A product improvement can come from the model or from surrounding retrieval, tools, prompts, and execution. Separate those changes when interpreting a release. Ask for evidence at the layer claimed and a fair before-and-after comparison. Several simultaneous changes may prevent attributing the gain to one cause.
-
Explain a fictional model release
Read four stated changes, identify their mechanisms, and choose the fresh evidence needed before calling the release better for a particular user.