Skip to content
Learn Motion
ExploreHow it worksMembership
Log in
Learn Motion

How Large Language Models Work: From Tokens to Inference

1From Text to Tokens2Embeddings: Turning Tokens into Vectors3The Transformer Block: Attention and Feed-Forward Processing4Stacking Blocks into a Language Model5Training: How the Model Learns to Predict6From Pretraining to Instruction Following7Inference: Generating Text Token by Token
Inference: Generating Text Token by Token

Choosing the Token: Temperature, Top-k, and Top-p

2 / 3
Start with temperature near zero and press sample a few times. The same token comes up every time, because the distribution has collapsed onto the top candidate. Now raise the temperature and watch the bars flatten; the low-probability tokens become reachable and the samples start to vary. Then set k to three and p to about zero point nine. The bars outside the eligible set grey out, and sampling only draws from what remains. The thing to notice is that temperature changes the shape of the distribution, while k and p change which candidates are allowed at all. That is the whole trade-off: tighten these controls when you need one reliable answer, loosen them when you want variety.
0:00 / 0:00

The model's output at each step is a distribution over the vocabulary, not a decision. Decoding is the rule that converts that distribution into one token, and it is the main lever controlling how deterministic or how varied the output is.

Greedy decoding takes the single highest-probability token every time. It is deterministic: the same prompt and the same model give the same output. Its weakness is repetition and blandness, because the most likely continuation is often the safest one.

Temperature rescales the logits before the softmax. With logits \(z_i\), the sampling distribution is \[ p_i = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)} \] where \(T\) is the temperature. As \(T \to 0\) the distribution concentrates on the top token and approaches greedy decoding; as \(T\) grows, the distribution flattens and low-probability tokens become reachable. Temperature changes the shape of the distribution but never removes candidates.

Top-k and top-p change which candidates are eligible. Top-k keeps the \(k\) highest-probability tokens and renormalizes. Top-p, also called nucleus sampling, keeps the smallest set of tokens whose cumulative probability exceeds \(p\), so the candidate pool adapts to how confident the model is: a peaked distribution yields a small pool, a flat one a larger pool. The two are often combined, for example top-p with a moderate temperature.

The practical rule is that consistency and variety pull in opposite directions. Tasks with a defensible single answer, such as extracting a field or writing code that must compile, favor low temperature or greedy decoding. Tasks that benefit from many acceptable phrasings, such as brainstorming or drafting alternatives, favor higher temperature with top-p sampling.

Previous2 / 3Next

Learn Motion

Generate a course. Learn it properly.

Operated by Wuhan Daoyin Technology Co., Ltd.

Contact: [email protected]
Privacy PolicyTerms of Service

© 2026 Learn Motion