How models are trained (and why it matters to you)

Training a modern LLM happens in three acts. If you can keep the distinction between them straight, you'll make better decisions about which model to use, when to fine-tune, and why some models feel smarter while others feel more "helpful."

Act 1 — Pre-training

The model reads a huge corpus of internet text, books, and code. It learns next-token prediction. At the end of pre-training you have a model that can continue text fluently but doesn't necessarily follow instructions. Pre-training is where most of the capability comes from — and most of the cost.

Act 2 — Instruction tuning

Humans (and other models) show the base model examples of prompts and desired responses. The model learns to respond to instructions rather than just continue text. This is what turns "fancy autocomplete" into "helpful assistant."

Act 3 — Alignment (RLHF, Constitutional AI, etc.)

The model is further tuned — typically using reinforcement learning from human feedback — to prefer responses that are helpful, honest, and harmless. This is where a model's "personality" is shaped: how cautious it is, how it refuses, how it pushes back.

Why you should care