Skip to content
Learn Motion
ExploreHow it worksMembership
Log in
Learn Motion

How Large Language Models Work: From Tokens to Inference

1From Text to Tokens2Embeddings: Turning Tokens into Vectors3The Transformer Block: Attention and Feed-Forward Processing4Stacking Blocks into a Language Model5Training: How the Model Learns to Predict6From Pretraining to Instruction Following7Inference: Generating Text Token by Token
From Pretraining to Instruction Following

Teaching the Model to Answer by Showing It Answers

1 / 2
The key shift is where the loss is applied. In pretraining, every token in the text is a target. In supervised fine-tuning, the prompt is context and only the response tokens are graded. So the model is not being taught facts it already has; it is being taught which continuation shape to prefer when it sees an instruction. That is why a relatively small set of high-quality demonstrations can change behavior noticeably.
0:00 / 0:00

Why a pretrained model does not follow instructions

Pretraining optimizes the probability of the next token given all preceding text, over a corpus that contains questions, answers, code, dialogue, and lists in no particular arrangement. Nothing in that objective marks a question as something that should be answered. A prompt like "What is the capital of France?" is just a prefix, and the model completes it with whatever continuation is statistically most likely across the corpus, which may be another question, a quiz heading, or a list of unrelated capitals. The model has the knowledge but no reason to apply it in the form the user expects.

What changes during fine-tuning

Fine-tuning reuses the same architecture and the same loss. What changes is the data distribution and where the loss is applied. In a training example, the prompt tokens are fed in as context, and the loss is computed only over the response tokens. So the gradient rewards the model for raising the probability of the demonstrated response given that prompt, and it does not reward or penalize the model for the prompt itself. Repeating this over many instruction-response pairs shifts the conditional distribution toward answer-shaped continuations for instruction-shaped inputs.

A single training pair

Consider the pair: instruction "Summarize this paragraph in one sentence," followed by a paragraph, followed by a one-sentence summary written by a human. During fine-tuning, the model reads the instruction and paragraph as context, then is graded on how much probability it assigns to each token of the human summary. If the model instead assigns high probability to a continuation like "This paragraph discusses..." followed by a restatement, the loss is high and the gradient corrects it. After many such pairs, the model learns the shape of a summary response, not just the content of any one summary.

The ceiling of imitation

SFT can only reproduce what the demonstrations show. If every demonstrated response is polite and concise, the model tends to be polite and concise. But imitation gives no direct signal about which of two plausible responses is better when both are acceptable in format. That gap is what preference-based alignment addresses.

Previous1 / 2Next

Learn Motion

Generate a course. Learn it properly.

Operated by Wuhan Daoyin Technology Co., Ltd.

Contact: [email protected]
Privacy PolicyTerms of Service

© 2026 Learn Motion