The model's output is a ranked distribution across the entire vocabulary, not a single stored answer. A few candidates hold most of the probability mass, and a long tail of thousands of entries holds a small share each. Because the output is a distribution, many tokens remain possible at every step, and the ranking is the model's entire contribution at this position.
How ChatGPT Writes One Token at a Time
Predicting the Next Token
A Ranked Field, Not an Answer
4 / 4
So when the model finishes this step, what does it actually hand over? Not an answer. It hands over a ranked list of candidates, ordered by probability.
0:00 / 0:00