Your next step
The problem attention helps solve
Order, distance, and the fixed-summary bottleneck.
Start learningWhat you’ll learn
Order changes the message
Identify a relationship changed by rearranging words.
About 4 minutes · Open activity
A useful clue can be far away
Identify a relevant distant part of a sequence.
About 4 minutes · Open activity
Reading with a running state
Explain the basic idea of a recurrent network.
About 4 minutes · Open activity
Some recurrent designs manage what to retain
Recognize the purpose of gates in an LSTM-style model.
About 4 minutes · Open activity
Compressing a sentence can lose useful detail
Explain the motivation for revisiting source positions.
About 4 minutes · Open activity
Earlier attention connected source and output
Place the 2014 attention work before the 2017 Transformer paper.
About 4 minutes · Open activity
Attention is computation, not awareness
State the boundary of the attention metaphor.
About 4 minutes · Open activity
Solve the missing-context puzzle
Find a distant clue, explain the summary bottleneck, and place earlier attention before the Transformer milestone.
About 5 minutes · Open activity