Chapter C34 · Interactive Course
From a base model to a useful assistant
Pretraining, fine-tuning, demonstrations, and where a change lives.
Core Learning Concept
Locate each proposed change in the system: current context, saved product information, or learned parameters. Then match the method to the need and check for regressions. A supplied file, an adapter, and a full fine-tune can all influence answers, but they do so through different mechanisms and persistence.
What You’ll Learn in This Chapter
-
Pretraining builds a broad starting point
Identify the broad initial learning stage.
Pretraining develops a broad foundational model on large diverse text mixtures via self-supervised next-token prediction. It imparts reusable language structure and general representations before later instruction tuning and alignment stages.
-
A base model is not an assistant by default
Distinguish continuation training from following a user's instruction.
A base language model is the result before the particular assistant-oriented adaptation being discussed. Its training may primarily reward continuation rather than following a user request. It can still exhibit useful capabilities. Post-training aims to make desired interaction behavior more reliable under specific objectives and evaluation.
-
Transfer learning reuses learned structure
Explain why a new task need not begin from zero.
Transfer learning reuses structure from a pretrained model for a new task. Starting from existing representations sharply reduces data and compute needs, though specific organizational facts still require targeted inputs.
-
Fine-tuning is further parameter learning
Identify a training update for a narrower task or behavior.
Fine-tuning performs further training on an existing model or selected trainable components. The process changes parameters or adapter values that persist after the run. Supplying a document in one prompt changes the available context instead. Distinguish those mechanisms before deciding what needs to be stored, updated, or tested.
-
Instruction tuning uses demonstrations
Recognize supervised examples of desired assistant responses.
Instruction tuning trains models on carefully paired demonstration examples showing desired assistant responses. Supervised updates align generation patterns with user requests, making instruction-following behavior far more dependable across diverse tasks.
-
Small trainable additions can adapt a model
Explain the purpose of LoRA at a conceptual level.
Parameter-efficient adaptation trains a smaller set of parameters, such as low-rank additions, while much of the base model remains frozen. This can reduce training and storage needs. It is still learning, not merely adding a prompt. Check how adapters are applied or merged and measure their effect on the task.
-
Specializing can change earlier behavior
Recognize the need to retest earlier capabilities after adaptation.
Specialization can improve a target behavior while weakening another. This is a regression, and it can occur with full fine-tuning or adapters. Keep representative checks for earlier requirements as well as new ones. Evaluate the adapted system against both before deciding that the change is an improvement overall.
-
Different changes happen in different places
Distinguish prompt context, product memory and model training.
Context supplies information for a response; product memory can store and later supply selected information; training changes learned parameters. Each has different scope and persistence. Deleting a chat, editing a memory, and replacing a trained model are distinct operations, so identify the layer before promising what a change removes.
-
Locate the change
Classify a document attachment, a saved preference and a reviewed fine-tuning dataset, and choose the change matching a stated need.