Skip to content
Learn Motion
ExploreHow it worksMembership
Log in
Learn Motion

How Machines Learn from Data

1Learning Without Being Told the Rules2Measuring Mistakes and Adjusting3Generalizing Beyond the Training Data4The Main Ways Machines Learn
Generalizing Beyond the Training Data

Memorizing vs. Generalizing

2 / 3
The student analogy captures the whole distinction. Memorizing last year's answers gets a perfect score on last year's paper and nothing on a new one. Understanding the principles handles questions you have never seen. A model can do the same thing: record the training answers, or learn the pattern behind them. Both look identical on the training loss. And memorizing is not a rare accident — when a model has many adjustable settings compared to the number of examples, fitting the training answers exactly, noise included, is often the easiest path to a low loss.
0:00 / 0:00

There are two very different ways a model can achieve a low training loss. It can learn the pattern that actually connects inputs to outputs — the features that make an email spam, the visual cues that make an image a cat. Or it can simply record which training inputs had which answers, without extracting any pattern. The first is generalizing; the second is memorizing. Both produce low training loss, but only the first survives contact with new data.

A student who memorizes the answers to last year's exam questions can score perfectly on that paper. Give them a new paper with different questions and the memorized answers are useless. A student who understood the underlying principles can handle questions they have never seen. The model that memorizes is the first student; the model that generalizes is the second. The training loss cannot tell them apart — only new questions can.

Memorizing is not a rare failure mode — it is often the easiest way to reduce training loss, especially when the model has many adjustable settings relative to the number of examples. With enough settings, a model can fit almost any set of training answers exactly, including the random noise in them, without learning anything that transfers.

Previous2 / 3Next

Learn Motion

Generate a course. Learn it properly.

Operated by Wuhan Daoyin Technology Co., Ltd.

Contact: [email protected]
Privacy PolicyTerms of Service

© 2026 Learn Motion