If the chunk containing the answer is not among the retrieved top-k, the generation step has nothing to ground on. The model still produces fluent text, but it is drawing on its own stored knowledge rather than the documents, so the answer drifts or becomes vague. This is why retrieval quality sets the ceiling on answer quality: a better prompt cannot supply evidence that was never retrieved.
How AI Answers Questions About Documents It Never Learned