One Unit, Then a Layer of Units | Visual AI Learning Guide

Here you can drive the network yourself. Start with the single unit on the left: slide the input and weight values and watch each product appear, then get added into one weighted sum. Now switch the activation between a rectifier and a sigmoid and see how that sum gets bent — the rectifier clips everything negative to zero, the sigmoid squeezes it between zero and one. That bend is what keeps the unit from being a plain weighted average. Next, look at the layer: many units side by side, each with its own weights, all reading the same inputs. Finally, drag the first layer's weights and watch the second layer's outputs move. The second layer never sees your original inputs — only the first layer's responses — so it is combining simple preferences into something more specific.

0:00 / 0:00

Auto-play

A single artificial unit takes several input numbers, multiplies each by its own weight, adds the results, and passes that sum through a simple nonlinearity. Written out, the unit computes \(y = f(w_1 x_1 + w_2 x_2 + \dots + b)\), where the \(x_i\) are the inputs, the \(w_i\) are learned weights, \(b\) is a learned offset, and \(f\) is a fixed nonlinear function such as a rectifier that passes positive values through and clips negatives to zero. The weights decide what the unit is sensitive to; the nonlinearity is what stops the unit from being just a weighted average.

One unit is weak. Its response is essentially a single weighted sum, so it can only express one simple preference — brighter here, darker there. The change that matters is putting many units side by side in a layer, each with its own weights, and then feeding that entire layer's outputs into a second layer. Now the second layer is not looking at raw pixels; it is looking at the first layer's responses. That is the whole trick of depth: each layer re-describes the previous layer's output, so combinations of simple preferences become available to the next layer.