Your next step

Build the Transformer picture

The block, the stack, and what 2017 actually contributed.

What you’ll learn

Attention is one part of a block
Distinguish information mixing from the whole model.
About 4 minutes · Open activity
A feed-forward step transforms each position
Identify a position-wise network after attention.
About 4 minutes · Open activity
A residual route preserves a path
Recognize a skip connection that combines earlier and transformed information.
About 4 minutes · Open activity
Normalization manages internal scale
Explain normalization's supporting role without deriving it.
About 4 minutes · Open activity
Stack blocks to refine representations
Trace several blocks without treating each as a different agent.
About 4 minutes · Open activity
The encoder represents the source
Identify the source side of the original translation Transformer.
About 4 minutes · Open activity
The decoder produces the next piece
Trace masked output context and source access in the original design.
About 4 minutes · Open activity
Training many positions differs from generating a reply
Distinguish parallel position computation from autoregressive generation.
About 4 minutes · Open activity
What Attention Is All You Need contributed
Identify the paper's historical claim in its original context.
About 4 minutes · Open activity
Explain the Transformer to a friend
Trace source to output on a supplied diagram, explain masking, and distinguish the 2017 contribution from earlier attention.
About 6 minutes · Open activity