From Raw Text to Chat Assistant: The Five Stages of Building an LLM

People talk about “the model” as one thing. It is the output of a pipeline, and the pipeline explains its behaviour. Why does a base model ramble instead of answering? Why does a chat model refuse some requests? Why can it not learn from your chat? Different stages, different answers.

Stage 1: Data collection and tokenization

Training starts with massive text: books, web pages, code. It is cleaned, filtered and deduplicated, then split into tokens, the units the model actually reads. Quality here caps everything later. A model cannot learn facts or styles that the data never contained, and it inherits the data’s biases and mistakes.

Stage 2: Pretraining

The core objective is simple: given the tokens so far, predict the next one. The architecture is a transformer, whose self-attention lets every token weigh every earlier token, and which trains efficiently in parallel. Trained on trillions of tokens, billions of parameters get adjusted until next-token prediction is good.

To predict well, the model has to pick up grammar, facts, writing styles and patterns of reasoning. That is why one objective yields such broad ability. It is also expensive, and it produces a base model: it continues text brilliantly but does not follow instructions.

Stage 3: Supervised fine-tuning

Now the model learns the assistant format from examples: a prompt paired with a good response. Training on thousands to millions of such pairs shifts it from “continue the text” to “respond to the user”.

Fine-tuning every weight is costly, so many teams use parameter-efficient methods. LoRA freezes the original weights and trains small low-rank matrices added to certain layers, cutting memory and cost sharply while adapting behaviour. The same approach lets you specialise a model for a domain without retraining it from scratch.

Stage 4: Preference alignment

Fine-tuning teaches format. Alignment teaches judgement: which of two plausible answers do people prefer? In RLHF, humans rank candidate responses, a reward model learns to predict those rankings, and the language model is optimized to produce answers the reward model scores highly. Newer methods reach similar goals more directly from preference pairs.

This stage is where much of the “helpful, honest, harmless” behaviour comes from, including refusals. It shapes tone and caution, and it can introduce trade-offs such as over-refusal or flattery.

Stage 5: Inference

Serving the model is autoregressive: it reads the prompt, produces a probability distribution over the next token, picks one (sampling settings like temperature control randomness), appends it, and repeats. Weights are frozen. Nothing in your chat updates the model.

Production adds optimisation: quantization to shrink memory, batching to serve many users, caching of already-computed attention state, and safety filtering. Retrieval and tool calls extend what the model can answer by adding text to its context, not by changing it.

Which stage explains which behaviour

Behaviour	Where it comes from
Knows facts, writes fluent text	Pretraining
Does not know last week’s news	Pretraining cutoff (data stops)
Follows instructions and formats	Supervised fine-tuning
Declines some requests, polite tone	Alignment
Makes things up confidently	Next-token prediction rewards plausible text, not verified truth
Cannot learn from your chat	Inference uses frozen weights

Takeaway

An LLM is a stack of decisions: what data it saw, what objective it trained on, how it was taught to respond and what people rewarded. Know the five stages and you can tell which lever fixes which problem, and which problems no lever on the same stage can fix.