跳到主要内容
Learn Motion
探索课程产品能力价格

下载 Learn Motion

用手机扫描二维码,下载 App。

App Store

扫码从 App Store 下载 Learn Motion打开商店页面

Google Play

扫码从 Google Play 下载 Learn Motion打开商店页面
登录
Learn Motion

How Text-to-Video AI Works Under the Hood

1From Prompt to Conditioning Signal2Latent Diffusion: Generating Video in Compressed Space3Spatio-Temporal Attention: Keeping Frames Consistent4Where the Pipeline Breaks: Failure Modes and Their Causes
From Prompt to Conditioning Signal

How Conditioning Enters the Generator

4 / 4
So how does that sequence actually steer the video? It enters through cross-attention. Picture the spatial grid of the video as a set of queries, and the prompt tokens as keys and values.
0:00 / 0:00

The conditioning embeddings reach the generator through cross-attention. In cross-attention, the spatial positions of the video latent act as queries, while the prompt token embeddings act as keys and values. Each position computes how strongly it matches each token and pulls in a weighted mixture. That is why different regions of the frame can attend to different words: the region containing the boat queries "boat" most strongly, while the region containing rain queries "rainy". The prompt is therefore a control signal applied at every generation step, not a key that retrieves a stored video.

参考资料

  1. [1]Attention Is All You Need (Vaswani et al., 2017)arxiv.org
  2. [2]High-Resolution Image Synthesis with Latent Diffusion Models (Rombach et al., 2022)arxiv.org
上一页4 / 4下一页

Learn Motion

生成课程,认真学会。

运营方:Wuhan Daoyin Technology Co., Ltd.

联系我们: [email protected]
隐私政策用户服务协议

© 2026 Learn Motion