Stack convolution layers and something systematic happens. The first layer sees pixels, so its filters can only respond to the simplest local structure: edges, color boundaries, small gradients. Its output map is handed to the second layer, which therefore sees edges rather than pixels. Filters in the second layer can combine edges into slightly larger motifs — a corner, a short curve, a repeated texture. The third layer sees those motifs and can combine them into parts: an eye, a wheel, a patch of fur. Deeper layers see parts and combine them into whole objects.
This is hierarchical feature learning, and it is the answer to the brittleness that ended the previous chapter. Nobody wrote the recipe for an eye detector. The network discovered that an eye detector is useful because it helps reduce error on the training data, and it built one out of the edge detectors below it. The same architecture that finds edges in layer one finds faces in layer twelve, purely because each layer is allowed to re-describe the layer beneath it.
The layers are not independent. Changing a filter in an early layer changes what every later layer receives, so the whole stack has to be adjusted together. That coupling is why training a deep network is a global problem rather than a sequence of local ones.