跳到主要内容
Learn Motion
探索课程产品能力价格

下载 Learn Motion

用手机扫描二维码,下载 App。

App Store

扫码从 App Store 下载 Learn Motion打开商店页面

Google Play

扫码从 Google Play 下载 Learn Motion打开商店页面
登录
Learn Motion

How AI Learns from Rewards and Mistakes

1The Trial-and-Error Loop: How an Agent Learns2Formalizing the Problem: States, Actions, Rewards, and Policies3Measuring Success: Return, Discounting, and Value4Learning from Experience: Value Updates and Q-Learning5Exploring vs. Exploiting: Where Mistakes Come From6Learning Policies Directly and Scaling Up
Learning from Experience: Value Updates and Q-Learning

The Temporal-Difference Update, Step by Step

1 / 2
Watch what happens when you step through one update. The target is the reward you just received plus the discounted best value you currently believe is in the next state. The error is the target minus the value you had stored for this state and action. If the outcome beat your expectation, the error is positive and the stored value rises; if it fell short, the value drops. The learning rate controls how big each step is. Run the same update a few times and notice the value creeping toward the target instead of jumping there at once.
0:00 / 0:00

The temporal-difference error measures how wrong the current estimate was. After taking action a in state s, observing reward r and landing in state s', we form the target
\[ \text{target} = r + \gamma \max_{a'} Q(s', a') \]
and the error
\[ \delta = \text{target} - Q(s, a). \]
The Q-learning update then moves the estimate a fraction alpha of the way toward the target:
\[ Q(s, a) \leftarrow Q(s, a) + \alpha , \delta. \]
Here alpha is the learning rate, a small positive number such as 0.1 or 0.5. The term \(\max_{a'} Q(s', a')\) uses the best action value we currently believe is available in the next state; because it reuses an estimate rather than the true return, this is called bootstrapping. A positive error means the outcome was better than expected, so the estimate rises; a negative error means it was worse, so the estimate falls. Repeating this on many transitions pushes the table toward the true action values.

上一页1 / 2下一页

Learn Motion

生成课程,认真学会。

运营方:Wuhan Daoyin Technology Co., Ltd.

联系我们: [email protected]
隐私政策用户服务协议

© 2026 Learn Motion