A language model does not write a whole answer and then release it. It produces the reply in small steps, and at each step it does one thing: given everything written so far, it picks what would most likely come next. The choice is a probability over possible next pieces of text, and the model takes one of the likely ones, adds it, and then repeats the whole process with the longer text.
This is why the output arrives smoothly. Each new piece is chosen to fit the pieces already there, so the sentence stays grammatical and on topic. The model is not retrieving a finished paragraph from storage; it is assembling one, token by token, where a token is just a small chunk of text such as a word or part of a word.
A useful way to picture it: if you have ever typed a sentence on your phone and watched it suggest the next word, you have seen a tiny version of the same idea. The suggestion is based on what usually follows, not on whether the resulting sentence is true. The model is doing that, at a much larger scale, for every piece of the answer.