符 · 号 · 復 号
Encoder–Decoder Lab
Lab 02 built a U-Net and trained it. This one is about everything people change afterwards:
swapping in a pretrained backbone, redesigning the decoder, widening the receptive field
without destroying resolution, and bolting attention onto the parts that need it.
One
階 · 層
An encoder is a feature pyramid
A backbone is not a black box that emits "features". It is a staircase: at every step the
map gets smaller, the channel count grows, and each position sees more of the original
image. Segmentation lives on the tension between those last two — deep features know
what, shallow features know where, and no single stage knows both.
Input — click to place the probe
The box shows what one neuron at the selected stage can see. Arrow keys work too.
The pyramid
All channels at this stage
Two
転 · 移
What a pretrained encoder actually buys
The argument for pretrained backbones is usually made with a citation. Here it is made with
a measurement — and the measurement does not say what you have been told it says.
Same network, same camouflage task as lab 02. The only difference is what the first layer's
weights are on iteration zero.
"Scratch" starts from random noise. "Pretrained" starts from six classical filters —
edges, a Laplacian, a blur, a centre-surround — which is close to what the first layer
of a trained vision network converges to anyway. Run both. Then read the box underneath,
because the result is the interesting part.
Input
the shape is in the texture
Target
Prediction
IoU over time
vermillion scratch · sage pretrained
Encoder filters, right now
Why the demo refuses to show the effect you were promised. Measured over
five seeds before this page shipped: at full learning rate scratch reached IoU 0.65 and
pretrained 0.57; frozen, 0.52 and 0.51; at a tenth encoder rate, 0.66 and 0.69. Pretraining
never reliably wins. That is not a broken demo — it is a network with 77 parameters
training on a single image for unlimited epochs. None of the conditions that make
transfer learning valuable are present. Transfer pays when data is scarce relative to model
capacity, and here data is effectively infinite relative to a very small model.
So what does a real pretrained backbone give you? Not better first-layer
filters — those are cheap, as you have just watched a network rediscover them in a few
hundred iterations. What you cannot afford to train yourself is everything section one was
about: depth. A ResNet-50 encoder hands you five stages, a receptive field covering
most of the image, and 23 million parameters shaped by a million labelled photographs. With
a few hundred medical scans or camouflage images you could never fit that from scratch; it
would memorise the training set before it learned anything general. You are borrowing the
hierarchy, not the edge detectors.
The controls are still the real ones. Set the encoder rate to zero and you
have frozen the backbone — the standard recipe when data is very scarce. Note that
freezing costs you something even here: about 0.13 IoU against a fully trainable encoder,
because the head alone cannot compensate for features that were never adapted to the task.
The range in between is "differential learning rates", a small rate for the encoder and a
full one for the decoder, which is what most fine-tuning scripts are quietly doing.
Three
膨 · 張
Context without throwing away resolution
Section one showed the bind: to see more context you downsample, and downsampling is exactly
what destroys the boundaries you need. Dilated convolution is the way out — spread the
kernel's taps apart instead of shrinking the map. The grid below is the input, and the
shading shows which pixels actually reach one output neuron after three stacked layers.
What one output neuron can see
reached once
reached often
never reached
Dilation rate, layer by layer
Four
並 · 列
ASPP — look at every scale at once
If one dilation rate gives you one scale, the obvious move is to run several in parallel and
let the network combine them. That is Atrous Spatial Pyramid Pooling, the centrepiece of
DeepLab. Switch branches off and watch the fused map lose the scale each one was providing.
Input feature map
48 × 48
Fused output
Why toggling image pooling does not change the fused picture. That branch
averages the entire map to a single number and broadcasts it back, so it is spatially
constant — it carries global context, not local structure. Real ASPP
concatenates its branches into separate channels rather than adding them, so that
constant survives as its own channel for later layers to use. The preview here sums the
dilated branches to keep them comparable in one image, and leaves the pooling branch out,
because adding a constant to every pixel would change nothing you could see.
Five
選 · 択
Squeeze-and-excitation, then CBAM
A convolutional layer treats all its channels as equally important. They are not. SE asks a
simple question — averaged over the whole image, how strongly did each channel fire? — and
uses the answer to turn channels up or down. CBAM then adds the same idea across space:
which locations matter.
Channels before gating
Channels after gating
Squeeze — each channel's global average
Excitation — the learned gate
Where the gate comes from. In a real SE block the excitation weights come
from a tiny two-layer network trained end to end. Training that here would take longer than
you want to wait, so the gate below is computed by a stand-in rule: how well does this
channel's response correlate with the target region? A trained SE block converges on
something similar, but do not mistake the stand-in for the real thing.
CBAM — the spatial half
Input
the scene
Spatial attention
avg & max pooled across channels
After spatial gating
Six
門
Attention on the skip connection
Lab 02 established that skip connections are the only route by which fine detail reaches the
decoder. That is also their problem: they forward everything at that resolution,
including a great deal of background the decoder has no use for. Attention U-Net puts a gate
on the skip — the decoder's coarse, semantically confident guess decides which parts of the
fine-grained skip are allowed through.
Skip features
fine, but indiscriminate
×
Gating signal
coarse, from deeper down
→
Attention α
=
Gated skip
only the relevant detail
This is the bridge to lab 05. A camouflaged animal is precisely the case
where the skip connection is carrying an overwhelming amount of background that looks
statistically identical to the target. Gating the skip is one of the main things SINet and
its descendants are doing, under various names. Push the coarse-guess quality down and watch
the gate confidently suppress the wrong region — a gate is only ever as good as the
semantics driving it.