Keeping the Receipts

Deriving “new height = old height ÷ stretch”

Part 1's one rule was a slogan. Here is where it comes from — a four-line argument you can check yourself. Let X have density p_X, and push it through a smooth, increasing map f to get Y = f(X). The trick is to reason about accumulated probability (the CDF, F), because accumulation survives relabeling: Y is below y exactly when X is below f⁻¹(y).

Density is the derivative of accumulated probability, so differentiate both sides in y, using the chain rule on the right:

writing x = f⁻¹(y) and using the inverse-function rule (f⁻¹)′ = 1/f′. That fraction is the slogan: old height at the source point, divided by the local stretch f′. If f is decreasing instead, the same argument runs with flipped inequalities and delivers a minus sign — which the density absorbs as an absolute value. Hence the theorem in its adult form:

A matrix at every point

In one dimension, “local stretch” was a single number, f′(x). In d dimensions a smooth map T can stretch each direction differently and rotate and shear — so the local behavior at a point is not a number but a linear map: zoom in far enough and T looks like multiplication by a matrix. That matrix of partial derivatives is the Jacobian:

And the “stretch factor” is its determinant. Feed the linear map a tiny unit square; out comes a parallelogram; the parallelogram's area is |det J|. A negative determinant just means the map flipped orientation (a mirror image) — the area is still |det|, which is why the formula wears absolute-value bars. So the theorem generalizes with one substitution — derivative → Jacobian determinant:

Watch it happen. Below, a nonlinear warp acts on the plane (the slider morphs it in from the identity). The dashed blue square is a small patch at your probe point before the warp; the orange parallelogram is that same patch after. The matrix on the right is the Jacobian at the probe, computed live — click or drag anywhere on the canvas to move the probe, and watch the matrix and its determinant chase the local geometry.

Two things worth noticing in the readout. The parallelogram's measured area really is |det J| times the square's area — the determinant is not a metaphor, it is the area. And the map in this demo has a special form: its first output depends only on x₁. That makes the Jacobian triangular (top-right entry is exactly zero) — hold that thought for Section 4.

Receipts add

Now stack layers, as a real flow does: T = T_K ∘ ⋯ ∘ T₂ ∘ T₁. The chain rule says the Jacobian of a composition is the product of the layer Jacobians, and determinants multiply along with them. Products are awkward — a hundred numbers multiplied together overflow or vanish — so everyone works in logs, where the product becomes a sum:

The ledger metaphor is now literal: each layer appends one line — its local log-|det| — and the total is a sum. Nothing is approximated anywhere; that exactness is inherited by everything built on top.

Where does space actually expand and contract? Paint it. The heatmap below colors the plane by the local receipt log |det J| of the whole map: red where a patch grew, blue where it shrank, neutral where area was preserved. Try both maps.

The second map holds a small surprise. Its stretch layer has the same receipt everywhere (×1.7 · ×0.4 = 0.68 — a flat field), and its swirl layer's receipt is exactly zero: a radius-dependent rotation moves points along circles without changing any areas at all, so det = 1 identically. All that visual drama in Part 1's spiral, and the ledger records a single constant line. Rotations, permutations, and shears are “free” layers — expressiveness without receipts.

Triangular is the magic word

Here is the engineering problem the next part solves. For a d-dimensional layer, det J is a determinant of a d×d matrix — an O(d³) computation, done at every data point, every training step. For images, d is in the hundreds of thousands. Generic layers are unaffordable.

But determinants have a famous escape hatch: if a matrix is triangular — everything above the diagonal is zero — its determinant is just the product of the diagonal entries, O(d). And you can force a Jacobian to be triangular by architecture: make output 1 depend only on input 1, output 2 only on inputs 1–2, and so on. Each output may depend on earlier inputs in any wild, nonlinear way — those derivatives land below the diagonal, where the determinant never looks. The probe demo's warp was exactly this: its top-right entry stayed pinned at 0.00 no matter where you put the probe, and its determinant was always just J₁₁·J₂₂.

This is the design space of the whole field: layers whose Jacobians are triangular (cheap dets), orthogonal (det = ±1, free), or otherwise structured — composed into stacks expressive enough to bend a bell curve into anything. Building those layers well is Part 3.

Four questions