Chapter C20 · Interactive Course
Different ways to learn
Find where a training setup's signal comes from.
Core Learning Concept
Find the learning signal: what tells the model that one result is better than another? It may come from supplied labels, hidden parts of existing data, relationships between examples, or outcomes after actions. In the reward example, compare the measured score with the outcome people actually want.
What You’ll Learn in This Chapter
-
Learning with supplied answers
Identify supervised learning from paired examples and targets.
Supervised learning pairs inputs with target answers, such as photos labeled fragile. Training compares predictions to these targets. The learned model's quality depends on how accurately and consistently those targets reflect the real task.
-
Learning without supplied answers
Identify unsupervised learning from unlabelled examples.
Unsupervised learning seeks structure without a supplied target answer for every example. Clustering book descriptions is one example. The absence of labels does not remove human choices: people still select the data, representation, method, and how the resulting structure will be interpreted or used.
-
Making a practice question from the data
Explain self-supervision using a hidden part of an example.
Self-supervised learning forms training tasks directly from unannotated data, such as hiding words and predicting the missing text. This creates abundant training examples without manual labeling, shaping the representations a model learns.
-
Predicting what comes next
Identify the target in a next-piece training example.
Next-token training asks a model to predict the next text piece from the preceding pieces. The actual next token in the training sequence supplies the target. Repeating this across many examples teaches language patterns. The target is the observed continuation, not a guarantee that every sentence is factually correct.
-
Matching things that belong together
Recognize a contrastive training task.
Contrastive learning uses relationships between examples, often encouraging matching items to have compatible representations. A picture and its caption can be a positive pair. The training setup defines what counts as matching and contrasting; useful similarity is learned from those choices rather than supplied by a universal meaning scale.
-
Learning through consequences
Identify reinforcement learning from action and outcome feedback.
Reinforcement learning trains policies through feedback following actions. Rewards may arrive after sequences of moves, requiring credit assignment to earlier choices. The reward function dictates which strategies the system ultimately develops.
-
The score is a design choice
Separate a reward from the real intention.
A training score is a measurable proxy for an intended goal. If metrics omit key qualities like safety or cleanliness, optimizing them can produce unwanted side effects. Always inspect what a metric omits.
-
A small labeled set can guide a larger collection
Recognize semi-supervised learning from mixed labeled and unlabeled examples.
Semi-supervised learning combines labeled and unlabeled examples. A small labeled set supplies target information while a larger unlabeled set may help reveal structure. This can be useful when labels are expensive, but the benefit depends on the method and data. Unlabeled examples do not guarantee correct targets.
-
Methods can be combined
Recognize successive learning methods in one model's development.
A model can pass through several learning stages with different signals. Broad prediction training may be followed by demonstrations and preference feedback. Each stage can change behavior and knowledge in different ways. To explain an observed improvement, inspect what actually changed and how it was evaluated.
-
Find the learning signal
Inspect three training setups, identify the target or feedback source in each, and repair one badly chosen reward.