Where attention shows up

Attention was invented for one narrow task — machine translation — and then quietly ate the entire field. The same query-key-value idea you just built now runs under a startling range of AI, not just text. This closing tour shows how far it spread.

One mechanism, many modalities

Why does the same idea work for words, pixels, proteins and code? Because attention doesn't care what the tokens are — it only needs a set of items that should influence each other. Give it image patches and it becomes a vision transformer; give it amino acids and it becomes AlphaFold; give it a mix of text and image tokens and it becomes a multimodal model. The "everyone looks at everyone" operation is astonishingly general.

That generality — plus the parallelism from the last chapter — is why one 2017 paper reshaped nearly all of AI in under a decade.

That's the course

You built attention from the ground up:

• A word's meaning depends on context, so it must look at the others.
• Query, key, value is how one word gathers meaning from the rest.
• Self-attention runs that for every word at once — the attention matrix.
• Multi-head attention tracks many relationships in parallel.
• The Transformer stacks it into the architecture behind every LLM.

Now, when you read that a model is a "transformer," you'll know exactly what's happening inside: words looking at words, over and over, layer by layer.

If you enjoyed this, it pairs perfectly with How ChatGPT Works (what the transformer is predicting), What Inference Really Is (how it runs), and the other courses on how AI works.