Chapter C39 · Interactive Course
Useful models can be smaller and simpler
Linear models, trees, ensembles, routing, distillation, precision.
Core Learning Concept
Choose a model under the device’s actual quality, memory, latency, and operating constraints. Compare simple models, combined predictors, routed computation, distillation, and quantization as different options. Use the provided evidence to eliminate unsuitable candidates and state the tradeoff; there is no universal winner for every task.
What You’ll Learn in This Chapter
-
A simple model can fit a simple task
Recognize a linear model used for a numerical estimate.
A simple model, such as a linear predictor, may be sufficient for a task with a suitable relationship between features and outcome. It can be cheaper and easier to inspect. Establish a baseline and test it on relevant cases before paying for complexity that may not improve the practical result.
-
A decision tree follows learned splits
Trace a tree through a small classification example.
A decision tree applies learned feature-based splits sequentially. Following comparison branches from root to leaf yields an interpretable prediction, though trees remain vulnerable to overfitting on noisy or unrepresentative training data.
-
An ensemble combines model outputs
Explain how multiple predictions can be combined.
An ensemble combines outputs from several independent models using aggregation rules such as averaging or voting. Blending complementary model predictions reduces individual variance, though overall serving compute and latency costs increase.
-
A mixture of experts routes computation
Distinguish sparse model routing from several chatbots voting.
A mixture-of-experts model routes representations through selected internal subnetworks. A learned router can activate a subset of experts for each token or input. “Expert” names a component, not necessarily a human-recognizable subject specialist or a separate conversational agent. Routing and combining remain parts of one model.
-
Total parameters and active work differ
Interpret two size numbers for a sparse model.
Total parameter count determines hardware memory storage requirements, whereas active parameter count governs per-token compute work. Sparse mixture-of-experts models balance large representational capacity with fast, computationally efficient inference task execution.
-
Distillation teaches another model from a teacher
Recognize a student model learning from teacher behavior.
Distillation trains a student using information from a teacher, such as outputs or probabilities. A smaller student may reproduce useful behavior more cheaply on the target tasks. Teacher errors and data coverage can limit the result. Evaluate the student independently rather than assuming it perfectly inherits every teacher capability.
-
Quantization uses lower-precision numbers
Explain the memory tradeoff of reduced numeric precision.
Quantization stores model weights using lower numerical precision, such as 8-bit or 4-bit formats. This significantly decreases memory bandwidth and storage footprints, enabling larger models to run on resource-constrained hardware.
-
Choose the tradeoff for the task
Select a model using supplied quality, delay and resource evidence.
A tradeoff is a choice among competing requirements, such as quality, memory, speed, privacy, and cost. State hard limits first, then compare candidates that satisfy them. A smaller model can be best for a constrained task, while another task may justify a larger one. Use evidence for that specific choice.
-
Choose a model for a small device
Use supplied task results and memory constraints to compare a simple model, a compressed model and a larger remote option without declaring a universal winner.