跳到主要内容
Learn Motion
探索课程产品能力价格

下载 Learn Motion

用手机扫描二维码,下载 App。

App Store

扫码从 App Store 下载 Learn Motion打开商店页面

Google Play

扫码从 Google Play 下载 Learn Motion打开商店页面
登录
Learn Motion

How Large Language Models Work: From Tokens to Inference

1From Text to Tokens2Embeddings: Turning Tokens into Vectors3The Transformer Block: Attention and Feed-Forward Processing4Stacking Blocks into a Language Model5Training: How the Model Learns to Predict6From Pretraining to Instruction Following7Inference: Generating Text Token by Token
The Transformer Block: Attention and Feed-Forward Processing

The Rest of the Block: Feed-Forward, Residuals, and Normalization

3 / 3
Attention moves information between positions, but it never transforms a single position by itself. The feed-forward network does that, applying the same small network to every position separately, which is where new features get computed. The residual connection adds each sublayer's input back to its output, so the block only has to learn a change, and gradients have a short path home. Layer normalization keeps each vector's scale under control. Those two are the reason a transformer can be stacked many times without the signal blowing up or fading away.
0:00 / 0:00

What the feed-forward network adds

Self-attention only recombines the vectors already present, so on its own it cannot create a new feature from a single position. The feed-forward network is applied to each position separately: it projects the vector up to a wider intermediate dimension, applies a nonlinearity, and projects back down. Because it is nonlinear and position-wise, it lets the block compute new combinations of features rather than just averaging existing ones.

Why residuals and layer normalization matter

A residual connection adds the input of a sublayer to its output, so each sublayer learns a modification of the signal instead of a replacement. This gives gradients a direct path backward and lets a block begin close to the identity, which is what makes many stacked blocks trainable. Layer normalization rescales each position's vector to a controlled mean and variance, keeping activations in a stable range as depth increases. Without these two devices, deep transformer stacks tend to suffer exploding or vanishing signals.

Order inside the block

A common arrangement is: layer normalization, then attention, added back to the input; then layer normalization, then the feed-forward network, added back again. The exact placement varies between model families, but the pattern of a mixing step plus a per-position step, each wrapped in a residual, is what defines the block.

参考资料

  1. [1]Attention Is All You Needarxiv.org
上一页3 / 3下一页

Learn Motion

生成课程,认真学会。

运营方:Wuhan Daoyin Technology Co., Ltd.

联系我们: [email protected]
隐私政策用户服务协议

© 2026 Learn Motion