Overfitting and underfitting are the two failure modes of generalization. Underfitting means the model is too simple or too rigid to capture the real pattern, so it does poorly on both training and new data. Overfitting means the model is so flexible that it fits the training examples — including their noise — almost perfectly, but fails on new data because it learned the noise instead of the signal. The amount and variety of training data determine how hard each failure mode is to avoid: more examples, and more varied examples, make memorizing harder and push the model toward the real pattern.
Learn MotionHow Machines Learn from Data
Generalizing Beyond the Training Data
Too Simple, Too Complex, Just Right
3 / 3
Think of the three panels as three models of increasing flexibility. On the left, the model is too rigid — it misses the real pattern, so it is wrong on the training points and wrong on new ones. That is underfitting. On the right, the model is so flexible it threads through every training point, including the noisy ones, and its curve swings wildly between them. It looks perfect on training data but fails on new data. That is overfitting. The middle panel is the goal: flexible enough to follow the real trend, not so flexible that it chases noise. More examples, and more varied examples, make the right panel harder to reach, because there is less room to succeed by memorizing.
0:00 / 0:00