Skip to content
Learn Motion
ExploreHow it worksMembership
Log in
Learn Motion

How Text-to-Video AI Works Under the Hood

1From Prompt to Conditioning Signal2Latent Diffusion: Generating Video in Compressed Space3Spatio-Temporal Attention: Keeping Frames Consistent4Where the Pipeline Breaks: Failure Modes and Their Causes
Latent Diffusion: Generating Video in Compressed Space

The Denoising Loop

3 / 5
The model's job is to reverse that destruction. It starts from pure noise and runs a loop, one timestep at a time, from the noisiest step back down to zero.
0:00 / 0:00

Reverse denoising runs the noising process backwards. At each step the denoiser — typically a U-Net or a diffusion transformer, a DiT — receives three inputs: the current noisy latent, the timestep, and the prompt conditioning embeddings from the text encoder. It outputs a prediction of the noise that was added. That prediction is subtracted from the noisy latent to produce a slightly cleaner one, and the loop repeats. The timestep matters because the amount of noise present changes how aggressively the model should correct. The prompt conditioning enters through cross-attention at every step, so the same prompt steers the latent from pure noise all the way to a coherent video.

References

  1. [1]High-Resolution Image Synthesis with Latent Diffusion Modelsarxiv.org
Previous3 / 5Next

Learn Motion

Generate a course. Learn it properly.

Operated by Wuhan Daoyin Technology Co., Ltd.

Contact: [email protected]
Privacy PolicyTerms of Service

© 2026 Learn Motion