Before selection, the distribution can be reshaped. Temperature rescales the logits before softmax: a low temperature sharpens the distribution so the top candidate dominates, while a high temperature flattens it so many tokens become nearly equally likely. Top-k filtering keeps only the k highest-probability tokens and discards the rest. Top-p filtering keeps the smallest set of tokens whose cumulative probability reaches p, cutting off the long tail. Both filters restrict which tokens are eligible before the actual selection step.
How ChatGPT Writes One Token at a Time