Learning from Rewards Over Time | Visual AI Learning Guide

Watch the agent move through the environment. At the start its choices are almost random, so it wanders and rewards are rare. Each time it reaches a reward, that outcome is fed back and the choices that led there become slightly more likely. Notice the delay: the reward appears at the end of a path, but the decisions that earned it happened several steps earlier, so the agent has to work out which earlier choices deserve credit. Over the rounds you can see the paths tighten toward the rewarding route, while a little exploration remains so the agent does not get stuck in a single habit. That shift from exploring to favoring what has paid off is the whole learning process here.

0:00 / 0:00

Auto-play

Reinforcement learning differs from the other two styles in when feedback arrives. The model — usually called an agent — takes an action in an environment, receives a reward or penalty, and then faces a new situation shaped by that action. There is no labeled example saying what the right action was. The only signal is the reward, and it often arrives many steps after the decision that caused it.

Because feedback is delayed and tied to sequences of actions, the agent must solve a credit-assignment problem: which earlier choices deserve credit for a reward received later? A useful way to think about it is that the agent is not learning to match answers but to choose actions that lead to good long-run outcomes. Early on it explores — trying actions to see what happens — and over time it exploits what it has learned, favoring actions that have paid off.

The animation shows this progression: an agent starts by acting almost randomly, receives sparse rewards, and gradually shifts its action choices toward the paths that produced reward, while still occasionally exploring.