Attention is not the only way to mix frames. Temporal layers and 3D convolutions use a small kernel that slides across the frame stack, combining each position with its neighbors in space and in time. Because the kernel is local, it is cheap, and because it spans several frames at once, it responds to how content shifts between them — the basis of motion cues. A moving edge produces a consistent pattern across the kernel's temporal extent, and the layer learns to encode that pattern. The limitation is range: a 3D convolution only sees a few frames at a time, so long-range consistency still needs attention.
How Text-to-Video AI Works Under the Hood