Why "Attention Is All You Need" replaced recurrence with self-attention, the query/key/value mechanism, positional encoding, multi-head attention, encoder-decoder vs. decoder-only architectures, and the tokenization layer underneath it all.
Module 3 ended on the specific limitation that recurrent networks never fully escaped: information from early in a long sequence degrades as it is relayed step by step toward later positions. This module covers the architecture built specifically to close that gap — the 2017 transformer, whose self-attention mechanism lets every token in a sequence attend directly to every other token in a single computation, regardless of distance, while also unlocking far greater training parallelization than any recurrence-based architecture allowed.
You will work through queries, keys, and values as the mechanism behind attention weights; positional encoding as the fix for the ordering information recurrence used to provide for free; multi-head attention's parallel relationship views; the encoder-decoder vs. decoder-only distinction that explains why most modern LLMs are decoder-only; and subword tokenization as the layer everything else in this module actually operates on. Module 5 picks up exactly here, treating a large language model as this same architecture at far greater scale.