跳到主要内容
Learn Motion
探索课程产品能力价格

下载 Learn Motion

用手机扫描二维码,下载 App。

App Store

扫码从 App Store 下载 Learn Motion打开商店页面

Google Play

扫码从 Google Play 下载 Learn Motion打开商店页面
登录
Learn Motion

How Large Language Models Work: From Tokens to Inference

1From Text to Tokens2Embeddings: Turning Tokens into Vectors3The Transformer Block: Attention and Feed-Forward Processing4Stacking Blocks into a Language Model5Training: How the Model Learns to Predict6From Pretraining to Instruction Following7Inference: Generating Text Token by Token
Training: How the Model Learns to Predict

Following the Gradient Downhill

2 / 2
Watch the loss curve as training proceeds. Each step computes gradients by backpropagation, which tells every parameter how the loss would change if that parameter moved slightly. The optimizer then nudges each parameter in the opposite direction, scaled by the learning rate. Early on the loss drops quickly because the parameters are far from good values; later the curve flattens as the model refines. Notice that each update uses a batch, a small group of examples, so the step is an estimate of the true gradient and the curve is a little noisy. An epoch is one full pass over the data. If the learning rate were much larger, the curve would jump around or even climb; if much smaller, it would crawl. The downward trend across batches and epochs is the model learning to predict the corpus.
0:00 / 0:00

Once the loss is computed, training adjusts the model's parameters to reduce it. Backpropagation applies the chain rule through the network to compute, for every parameter, the partial derivative of the loss with respect to that parameter. This gradient points in the direction of steepest increase of the loss, so moving each parameter a small step opposite the gradient reduces the loss. Gradient descent expresses this as an update: each parameter \(\theta\) becomes \(\theta - \eta , \nabla_\theta L\), where \(\eta\) is the learning rate and \(\nabla_\theta L\) is the gradient of the loss with respect to that parameter.

The learning rate controls the size of each step. If it is too large, the loss can oscillate or diverge; if it is too small, training takes impractically long. In practice the update is computed from a batch, a small group of examples, rather than the entire dataset, so each step is an estimate of the true gradient. An epoch is one full pass over the training data. Across many batches and epochs the loss trends downward as the parameters settle into values that predict the corpus well.

上一页2 / 2下一页

Learn Motion

生成课程,认真学会。

运营方:Wuhan Daoyin Technology Co., Ltd.

联系我们: [email protected]
隐私政策用户服务协议

© 2026 Learn Motion