One Recipe, Many Kinds of Task | AI Learning Guide

Look at the transfer learning example, because it is where the flexibility becomes concrete. Suppose a model has already been trained on a very large collection of images. Along the way it picked up general visual structure, the kind of thing that shows up in many pictures. Now you have a new, narrow task, telling two similar product categories apart, and you only have a small labeled set. You can reuse the trained model and get good results with far fewer examples than starting from nothing would need. The earlier learning carried over. That is the flexibility in action. Compare it with the traditional side. The features there were chosen for the original problem, so they do not move to a new problem; you would rebuild the feature work from scratch. But keep the cost in view. The very thing that lets learning carry over is the thing that needed the huge image collection and the heavy computation in the first place.

0:00 / 0:00

Auto-play

Traditional pipelines are rebuilt for each task

Because the features are chosen by a person for a specific problem, a new problem means a new set of features. A spam filter, a credit model, and an image classifier each require their own design effort, and the work does not carry over from one to the next.

Deep learning reuses one recipe across task types

A deep model is not tied to one kind of feature, because it learns its representations rather than receiving them. The same basic recipe — a stack of layers trained on examples — applies to images, audio, text, and sequences. What changes between tasks is mostly the shape of the input and the shape of the output, not the underlying approach.

Transfer learning as flexibility in practice

A model trained on a very large image collection learns general visual structure along the way. That trained model can be reused for a related task, such as telling two similar product categories apart, with far fewer labeled examples than training from scratch would need. The earlier learning carries over. A traditional pipeline has no equivalent, because its features were built for the original task and do not move to a new one.

Flexibility is the same property that costs

The flexibility that lets one model family cover many tasks is the same property that produces the data hunger and the compute bill discussed earlier. Learning representations instead of receiving them is what buys the range, and it is also what requires many examples and heavy computation.