跳到主要内容
Learn Motion
探索课程产品能力价格

下载 Learn Motion

用手机扫描二维码,下载 App。

App Store

扫码从 App Store 下载 Learn Motion打开商店页面

Google Play

扫码从 Google Play 下载 Learn Motion打开商店页面
登录
Learn Motion

How AI Sees: A Conceptual Guide to Image and Video Understanding

1What a Machine Actually Sees2From Pixels to Patterns: Features3How Deep Networks Learn to See4Beyond Labels: Locating and Describing What Is Seen5Adding Time: Understanding Video6How These Systems Learn and How We Judge Them
Beyond Labels: Locating and Describing What Is Seen

Turning an Image into a Sentence

4 / 4
Think of the captioning model as two specialists passing a note. The vision encoder looks at the image and writes down a compact summary — not words, just a set of numbers that capture what is visible. The language model reads that note and writes a sentence, one word at a time. At each step it looks at both the note and the words it has already written. That is why the caption can describe a scene the model has never seen: it is composing, not retrieving. But it also explains the failure mode — if the note is vague, the language model will still write something fluent, and fluent does not mean correct.
0:00 / 0:00

Two models working together

A captioning system has two parts. The vision encoder turns the image into a compact set of features — a numerical summary of what is visible. The language model takes those features and generates a sentence word by word. At each step it conditions on both the visual features and the words already produced, so the sentence stays connected to the image.

Generating a caption step by step

Suppose the image shows a dog jumping over a log. The model might generate: start → "a" → "dog" → "jumping" → "over" → "a" → "log" → end. Each word is chosen based on the visual features and the words so far. If the visual features are weak on the log, the model might still produce "a dog jumping" and stop, or produce a plausible but incorrect ending. The fluency of the sentence does not guarantee its accuracy.

Why captions can be confidently wrong

The language model is optimized to produce fluent, likely sentences. If the visual signal is ambiguous or the model has a strong prior from training, it can generate a sentence that reads well but misdescribes the image. Caption quality depends on both the visual features and the language model's tendency to stay faithful to them.

上一页4 / 4下一页

Learn Motion

生成课程,认真学会。

运营方:Wuhan Daoyin Technology Co., Ltd.

联系我们: [email protected]
隐私政策用户服务协议

© 2026 Learn Motion