Skip to content
Learn Motion
ExploreHow it worksMembership
Log in
Learn Motion

How Text-to-Video AI Works Under the Hood

1From Prompt to Conditioning Signal2Latent Diffusion: Generating Video in Compressed Space3Spatio-Temporal Attention: Keeping Frames Consistent4Where the Pipeline Breaks: Failure Modes and Their Causes
From Prompt to Conditioning Signal

Splitting the Prompt into Tokens

1 / 4
Take a prompt like "a paper boat drifting down a rainy street." The model cannot read that. Before anything else happens, the sentence gets chopped into tokens, and each token becomes an integer the network can handle.
0:00 / 0:00

A text-to-video model cannot read characters directly, so the prompt is first broken into tokens. Most systems use subword tokenization: common words become single tokens, while rare or novel words are split into smaller reusable pieces. This lets a fixed vocabulary cover words the model never saw during training, and it keeps related forms such as "drift", "drifting", and "drifted" sharing a common stem token. The result is a short sequence of integer token IDs, which is the actual input to the text encoder.

References

  1. [1]Hugging Face NLP Course: Tokenizershuggingface.co
  2. [2]Google SentencePiece: subword tokenizationgithub.com
1 / 4Next

Learn Motion

Generate a course. Learn it properly.

Operated by Wuhan Daoyin Technology Co., Ltd.

Contact: [email protected]
Privacy PolicyTerms of Service

© 2026 Learn Motion