A convolution is one small grid of numbers sliding over a larger one. Padding, stride and dilation are variations on where that window goes. Channels are what turn it into a layer. And the weights inside it are not designed — they are learned. Five sections, each one a thing you can run.
Here is the whole operation in one line. As the window moves below, the symbols in this equation light up with the values they currently hold. Set the stepping to single multiply and you can walk through the nine products one at a time.
Two coordinate systems are at work here, and confusing them is the usual stumbling block. x and y say where the window sits on the image, and change once per output value. i and j scan inside the kernel, and reset every time the window moves. The expression I(x+i, y+j) is an address, not a product — add first, then look up that cell. And notice that F takes only i and j: the weights are identical wherever the window is standing.
Nothing changes except the size of the grid. A photograph is a grid of numbers too, so those nine multiplications happen once per pixel. Each kernel below holds a different set of nine weights and produces a different feature map. Try the camouflage image — the animal is there, and most of these filters cannot find it.
Everything above used a single plane of numbers. A real image has three, and deeper in a network a feature map has hundreds. A convolution does not run separately on each — it runs on all of them at once and adds the results into one output. Step through the accumulation and watch a single output map being built from three inputs.
The sum over cin sits inside the expression. That is the whole point: a convolution collapses every input channel in one step, which is why the kernel is not a 3×3 grid but a four-dimensional tensor of shape (Cout, Cin, k, k), plus one bias per output channel.
This is where most people's intuition is wrong. A 3×3 convolution does not have nine parameters.
Pooling has no weights and learns nothing. It divides the map into blocks and replaces each block with one number — the largest, or the average. On a grid of digits that difference looks trivial. On an image it is not.
Every filter you have used so far was designed by a person decades ago. A convolutional network designs its own. Below, nine weights start as random numbers and a target output is given. Press play and watch gradient descent turn noise into an edge detector — nobody tells it the answer, only how wrong it currently is.
The error δ is simply the difference between what the filter produced and what was wanted. The gradient with respect to each weight is that error correlated against the input — another cross-correlation, the same operation running backwards. Each weight then takes a small step downhill. This is the entire learning algorithm.