Chapter C26 · Interactive Course
Representing relationships with numbers
Learned rows, similarity, shares and weighted mixing.
Core Learning Concept
Read the small numerical representations and compare what each number means. Distinguish an arbitrary token ID from a learned vector, a similarity score from a probability, and a two-dimensional teaching picture from a real embedding space. Use the supplied values to follow the comparison and mixing operations.
What You’ll Learn in This Chapter
-
A vector is a list of numbers
Read a small labeled numeric representation.
A vector is an ordered list of numbers, such as [2, 1, 0]. The positions must be interpreted consistently when comparing vectors. Some teaching examples name each position; learned representations often do not have simple human-readable dimensions. Numbers make operations such as comparison and weighted mixing possible.
-
An embedding is a learned representation
Explain why a token ID is mapped to a numeric vector.
An embedding is a learned numerical vector capturing semantic relationships. An embedding table maps discrete token IDs to dense vectors. The token ID selects the table row, supplying the model's initial numerical representation.
-
Nearness can make comparison useful
Interpret closeness in an illustrative embedding map.
Nearby embeddings indicate similarity under a learned representation, aiding retrieval of candidate passages. Nearness does not establish factual agreement or task accuracy, so verify retrieved content independently before relying on it.
-
The axes are not usually named ideas
Recognize limits of a two-dimensional embedding illustration.
A two-dimensional embedding plot is a simplified view of a space that may have hundreds or thousands of dimensions. Projection can distort distances and apparent clusters. Axis labels in teaching examples are explanatory aids, not a guarantee that a real model has one dimension for each named human concept.
-
Context can change the intended meaning
Identify which surrounding words disambiguate a token.
A token’s initial embedding can be the same across occurrences while later contextual representations differ. The word “bank” in a river sentence receives different surrounding information from “bank” in a money sentence. Context-processing layers update the representation of that occurrence; the token ID itself need not change.
-
A score is a degree of match
Compare supplied numerical match scores without treating them as truth probabilities.
A similarity score measures alignment under a specific metric rather than probability of truth. Use similarity values to rank candidates, but evaluate retrieved content directly to ensure it answers your specific question.
-
Turn scores into shares
Interpret normalized nonnegative weights that add to one.
Normalization can turn nonnegative values into shares that sum to one. Values 2 and 3 become 0.4 and 0.6 when divided by their total. Softmax is another transformation commonly used for scores. In an attention mixture, these shares control contributions; they are not probabilities that statements are true.
-
Representations can improve through training
Identify an embedding as part of a learned system.
Learned embeddings can change during training along with other parameters. Updates aim to make the representation useful for the training objective. Closer-looking clusters do not automatically prove better performance on your task. Compare relevant evaluation results and remember that a frozen embedding remains unchanged during ordinary inference.
-
Mix information in unequal amounts
Predict which value contributes most to a weighted combination.
A weighted mixture multiplies each value by its share, then adds the results. For values 10 and 20 with shares 0.25 and 0.75, the mixture is 17.5. The same operation works component by component for vectors. A larger share means a larger multiplier, not necessarily the largest numerical contribution.
-
Read a relationship map
Interpret a toy embedding map, distinguish a lookup ID from its vector, and predict the dominant contribution in a weighted mix.