Chapter C22 · Interactive Course
A neural network, one small part at a time
Trace a numerical model through connected layers.
Core Learning Concept
Follow numeric inputs through a small network. Multiply inputs by weights, sum them, add the bias, and apply any activation. Trace each parameter's distinct contribution to see how neural computations function.
What You’ll Learn in This Chapter
-
Turn observations into numbers
Interpret a simple numeric representation.
A neural network receives numerical representations of observations. A parcel might be represented by its weight and a flag indicating glass. The encoding determines what the numbers mean. Category codes do not automatically justify treating the distance between codes as a meaningful measurement, so choose representations suited to the task.
-
A weight changes influence
Predict how an input's weight changes a toy output.
A weight is a learned multiplier applied to an input or intermediate value. If an input is 2 and its weight is 0.5, its contribution is 1. Changing the weight changes that contribution while the input stays fixed. In a larger network, a weight’s effect also depends on later computations.
-
A bias shifts the starting point
Distinguish a numeric bias term from an input weight.
A bias term is an added number that shifts a node’s result independently of a particular input. In “2 × 0.5 + 0.3,” the weight is 0.5 and the bias is 0.3. This mathematical use of “bias” differs from unfair social bias in a system’s behavior.
-
A node combines inputs
Trace inputs through one simple computing node.
A simple artificial neuron, or node, combines weighted inputs, adds a bias, and applies an activation function. For inputs 2 and 1 with weights 0.5 and 2, the weighted sum is 3 before bias. The word “neuron” is an analogy; this calculation is not a complete biological cell.
-
A bend makes richer patterns possible
Explain why a nonlinear activation changes what a network can represent.
A nonlinear activation changes the shape a network can represent. ReLU, for example, returns zero for negative inputs and leaves positive inputs unchanged. Without nonlinear operations, stacking ordinary linear layers still gives a linear transformation. The example shows why changing a shape matters, not just increasing the number of steps.
-
Layers build on earlier representations
Identify how one layer's output becomes another's input.
A layer produces intermediate values that later layers use. Those values are representations: numerical descriptions shaped by the training task. In the simple chain shown, one layer’s output becomes the next layer’s input. Some architectures also pass earlier values through additional connections, so not every network is only a straight chain.
-
Deep means multiple learned layers
Interpret deep learning without an intelligence ranking.
“Deep” refers to multiple learned layers, not a direct ranking of intelligence. Extra depth can support useful representations, but performance also depends on data, training, architecture, and the task. Compare measured results and costs rather than assuming the deeper of two networks must be the better choice.
-
Parameters and temporary signals
Distinguish stored learned settings from values produced for one input.
Parameters are stored learned values, such as weights and biases. Activations are values produced while processing a particular input. A new message usually changes activations without changing parameters. Products may retain state or later use feedback for training, so ordinary inference and explicit learning must be distinguished.
-
Follow the small network
Trace an unseen parcel through a supplied diagram, change one influence, and identify parameters versus temporary values.