01

07 · useful win

Grow the model’s receptive field

Use hierarchical grouping to deepen the network, manage tensor shapes, and see why architecture changes how context can interact.

Compute receptive field growth across hierarchical layers.
02

The invariant chain

token window becomes a testable update

token window
pairwise grouping
local features
deeper grouping
wide-context prediction
03

Why it works

Depth can aggregate context progressively

Instead of flattening the entire context at once, group adjacent positions, transform them, then group again. Each level summarizes a larger receptive field while preserving a structured path for local interactions.

1Depth can aggregate context progressively
2Shape discipline becomes architecture discipline
3local features becomes observable
04

Implementation compass

The WaveNet connection is conceptual

class FlattenConsecutive:
    def __init__(self, n): self.n = n
    def __call__(self, x):
        B, T, C = x.shape
        x = x.view(B, T // self.n, C * self.n)
        return x.squeeze(1) if x.shape[1] == 1 else x
ShapeWrite every semantic axis beside the tensor.
ReferenceCompare one output against the smallest trusted version.
Scale gateOverfit one controlled batch before a real run.
05

Build gate

Compute receptive field growth across hierarchical layers.

Draw the receptive field of one output after each grouping layer.
Print shapes after every module in the network.
Compare parameter count and validation loss against the flat-context MLP.
Stop and inspect

Treating a reshape as harmless when it changes which positions are adjacent

Claiming architectural equivalence from a shared receptive-field idea

BUILD → TEST → EXPLAIN