How Large Language Models Actually Work, for Non-ML People

You don’t need to build a model to understand what it’s doing. You just need a mental model that isn’t wrong. Here’s the whole pipeline in one picture before we go section by section:

TRAINED ONCE, LONG BEFORE YOU SEE IT
  pretraining    predict the next token, over enormous text
  → fine-tuning  curated examples of how to behave
  → RLHF         human preference shapes what it does

EVERY TIME YOU SEND A MESSAGE
  "Unbelievable" (your text)
  → tokens       [Un] [believ] [able]
  → embeddings   each token becomes a list of numbers
  → attention    each token weighs the tokens before it
  → prediction   guess the next token, one at a time
  → output       a response, generated token by token

It doesn’t read words, it reads tokens

Text gets broken into chunks called tokens — sometimes a whole word, sometimes a fragment. “Unbelievable” might become three tokens. Pricing, context limits, and odd behavior around rare words all trace back to this.

Every token becomes a list of numbers

Each token turns into an embedding — a list of numbers representing its meaning in a way the model can do math with. Tokens used in similar ways end up with similar numbers — the property behind the classic word-vector result “king − man + woman ≈ queen.” That result comes from older, standalone word embeddings, and it’s a good intuition for what this layer holds, with one difference that matters: in an LLM these starting numbers are only the input. The layers above reshape them using the surrounding text, so the same word carries different numbers in different sentences — which is how “bank” stops being ambiguous.

Attention is how it decides what matters

For each token, the model weighs the tokens that came before it to figure out what’s relevant right now. This is how it resolves “it” back to a noun three sentences earlier, and why more relevant context tends to produce better answers.

Training is just next-token prediction, at enormous scale

Given the text so far, predict the next token — that’s the whole objective. Do that well enough over enough text, and the model ends up implicitly learning grammar, facts, reasoning, and style, none of which were labeled as separate goals.

Fine-tuning and RLHF shape behavior, not raw knowledge

A raw next-token predictor is a strange conversational partner. Extra training — fine-tuning on curated examples, then reinforcement learning from human feedback — teaches it to follow instructions, refuse certain requests, and format answers usefully. This is also where a lot of personality and guardrails get set. All of it happens before the model is ever deployed — none of it runs when you send a message.

Why it hallucinates

The model is still just predicting a plausible next token, not looking anything up. If the most plausible continuation happens to be false, it comes out with the same fluency as a true one. That’s not a rare glitch — it’s the same mechanism that makes the model useful, misfiring on a case where plausible and true diverged.

Why this matters

Once “plausible next token, not a lookup” clicks, a lot makes more sense — why more context helps, why it can state something false with total confidence, and why “it made something up” is never actually surprising.