BIG IDEA An LLM is a next-token predictor
Strip away the mystique and a large language model does one thing: given a sequence of tokens, it predicts the probability of the next token. It picks one, appends it, and predicts again โ over and over. That loop is called autoregressive generation. Everything impressive an LLM does โ answering, coding, reasoning โ emerges from doing this next-token guess extremely well, at massive scale.
DIAGRAM The pipeline, end to end
text โ tokens"] TK --> EM["2 ยท Embed
each token โ a vector"] EM --> PE["+ position info"] PE --> BLK subgraph BLK["3 ยท Transformer blocks (stacked รN)"] direction TB AT["Self-attention
(tokens look at each other)"] --> FF["Feed-forward
(think about each token)"] end BLK --> LG["4 ยท Logits
a score for every possible next token"] LG --> SM["5 ยท Softmax + sample
pick the next token"] SM --> OUT["Next token"] OUT -.append & repeat.-> TK classDef hi fill:#f3f0ff,stroke:#764ba2,color:#4a3b7a; class AT,FF,SM hi;
- Tokenize โ split text into tokens (see tokens).
- Embed โ turn each token into a vector of numbers that captures meaning, plus its position in the sequence.
- Transformer blocks โ stacked layers (dozens to 100+) that repeatedly mix information between tokens (attention) and process each token (feed-forward).
- Logits โ the final layer outputs a score for every token in the vocabulary.
- Sample โ convert scores to probabilities (softmax) and pick one โ how "random" depends on temperature. Append it and go again.
ATTENTION The key trick: tokens look at each other
Attention is what lets the model understand context. When processing a word, the model attends to the other words that matter for its meaning โ no matter how far away they are. Take:
To resolve "it," the model attends back to "trophy" (not "suitcase"). Attention computes, for every token, how much each other token should influence it, and blends them accordingly. Doing this across many layers and many parallel "attention heads" (each learning different relationships โ grammar, references, topic) is what builds rich understanding.
SCALE Why "just predicting the next token" is so powerful
To predict the next token well across the entire internet, a model is forced to learn grammar, facts, translation, code, and reasoning patterns โ because all of those help it guess better. Push three dials up together โ more parameters (the learned weights, the "B" in "70B"), more data, more compute โ and new capabilities appear. That's the "scaling" story behind modern LLMs.
โ What the Transformer gave us
- Handles long-range context via attention
- Processes a sequence in parallel (fast to train)
- Scales smoothly with size & data
โ ๏ธ The costs
- Attention is quadratic in length โ long context is expensive
- Huge models need huge compute to train & run
- It predicts plausible text โ hence hallucination
TAKEAWAYS
GLOSSARY
| Term | In one sentence |
|---|---|
| Transformer | The neural-network architecture behind modern LLMs, built around attention. |
| Self-attention | Each token weighs how much every other token should influence it. |
| Attention head | One of many parallel attention mechanisms, each learning a different kind of relationship. |
| Autoregressive | Generating one token at a time, each conditioned on all previous ones. |
| Parameters / weights | The billions of learned numbers that store what the model "knows." |
| Logits โ softmax | Raw next-token scores turned into probabilities to sample from. |
| Embedding | A vector representing a token's meaning (also used in RAG). |