Each token ID indexes a learned embedding vector, but that vector is only a starting point. The sequence then passes through a transformer, where self-attention lets every token look at every other token and update its own representation. The output is a sequence of contextual embeddings: the vector for "boat" already encodes that the boat is paper and that it is drifting. This is why the encoder's output is a sequence, not one vector — each position carries meaning shaped by its neighbors.
How Text-to-Video AI Works Under the Hood