Your next step
Seeing, hearing and combining media
Pixels, patches, sound over time, and reading a capability card.
Start learningWhat you’ll learn
Pictures become numerical input
Recognize pixels as one image representation.
About 4 minutes · Open activity
A convolution looks for local patterns
Explain the idea of a shared local filter in a CNN.
About 4 minutes · Open activity
Image patches can form a sequence
Recognize the patch representation used in a Vision Transformer.
About 4 minutes · Open activity
Sound can be represented over time
Recognize a time-based numerical view of audio.
About 4 minutes · Open activity
Transcribing is a prediction task
Identify how a speech model can make a transcription error.
About 4 minutes · Open activity
Learning to connect text and images
Explain cross-media matching in a shared representation task.
About 4 minutes · Open activity
Multimodal does not mean every input and output
Distinguish supported input media from supported output media.
About 4 minutes · Open activity
Video adds change across time
Identify a temporal relationship missing from a single frame.
About 4 minutes · Open activity
Choose the right media capability
Use fictional model cards to choose image input, speech transcription and time-sensitive video analysis, and name one evidence limit that remains.
About 6 minutes · Open activity