After a token is selected and appended, the extended sequence is fed back into the model as input. The model computes a fresh probability distribution for the next position, conditioned on everything that came before, including the token just added. That new distribution is used to select another token, which is appended, and the cycle repeats. This is the autoregressive loop: each output token becomes part of the input for the next prediction.
How ChatGPT Writes One Token at a Time