A model that scores perfectly on the data it trained on has not necessarily learned anything useful. Overfitting is what happens when the model captures the accidental details of the training examples instead of the general pattern that makes them what they are. The classic illustration is a decision boundary drawn through a scatter of labeled points: a smooth boundary that misses a few training points will usually classify new points correctly, while a boundary that threads through every single training point will fail badly on anything new. Generalization is the ability to do well on data the model has never seen, and it is the only performance that matters in the real world. The practical consequence is that you always evaluate on a held-out set — examples the model was not trained on — because training-set performance tells you how well the model memorized, not how well it will work.
Learn MotionHow AI Sees: A Conceptual Guide to Image and Video Understanding
How These Systems Learn and How We Judge Them
Memorizing versus generalizing
2 / 4
Look at the two boundaries side by side. The wiggly one passes through every training point — zero training errors. The smooth one misses a couple of training points. Now bring in the new points, the ones the model never saw during training. The wiggly boundary misclassifies them, because it was shaped around the accidental positions of the training points rather than the underlying pattern. The smooth boundary gets them right. That is the whole lesson: perfect training performance is not the goal. The goal is performance on data the model has never seen, which is why you always hold some examples back and evaluate on those.
0:00 / 0:00