Make the Data Likely
Part 3's trainer descended a loss without asking where it came from. This part derives the losses, gives each its personality — including the failure you can watch happen live — and then points the same machinery at a second job: computing integrals nothing else can touch.
Why “make the data likely” is the right command
You hold examples x₁…xn from an unknown data distribution pdata, and a flow whose exact density qθ came from Part 2's ledger. What should training minimize? The principled answer is a distance from the truth — the Kullback–Leibler divergence of the model from the data:
The first term is the data's own entropy — θ appears nowhere in it, so minimizing the KL means maximizing the second term alone. Replace its expectation with the sample average, flip the sign, and the loss Part 3's trainer was descending appears verbatim:
When all you hold is a formula
Now the other situation from the sciences: no examples at all, only an unnormalized score — a Boltzmann factor f(x) = e−E(x), a posterior's prior×likelihood — whose true density is p* = f/Z with Z unknown and unaffordable. Train anyway: sample from the flow itself, and measure the divergence pointing the other way:
And there is the famous cancellation: log Z is the same constant at every x and every setting of the knobs, so it shifts the whole loss surface without moving its minimum. You can descend a loss containing the very constant you cannot compute. (Boltzmann generators call this training by energy.) A third option skips divergences and goes straight for what an integrator cares about — the spread of the importance votes f/q, minimized as a variance. All three losses share one global minimum, q ∝ f. Their personalities differ on the way down — and personality is destiny, as the lab below shows.
Three losses, one target, live
Below, three identical 1-D flows (14 knobs each, a monotone stack of tanh bends) train simultaneously on the same two-bump unnormalized target — one under each loss, all by gradient descent in your browser. The orange shape is the target; blue is each model. The crucial experiment is the start: begin blind — a narrow model sitting on the big bump — and the two self-sampled losses never discover the second bump, because they only learn from where their own samples land. The forward-KL model cannot be fooled this way: its samples come from the data, and the data knows about both bumps.
Run it both ways. From the wide start, all three succeed (reverse KL habitually skimps on the smaller bump — its mode-seeking accent). From the blind start: forward KL walks out and claims both bumps; reverse KL polishes the one bump it can see; and the variance model reaches its most dangerous state — flat votes around the wrong answer, reporting Ẑ ≈ 2.40 with total confidence while the truth is 3.53. Flat votes certify that q matches f where q goes; they say nothing about the places q never visits. That is why real systems anneal, mix objectives, or inject defensive wide samples — and why “training by example” and “training by energy” are combined in Boltzmann generators rather than trusted alone.
Point the samples where it counts
Away from generative modeling, flows have a quieter superpower: computing integrals — the total area under a complicated curve. In serious applications that curve lives in dozens or thousands of dimensions: the odds of particle-collision outcomes, the light arriving at a camera pixel, the energies of a molecule's configurations. No grid can cover that much space, so the one method that survives is sampling — drop random points, average what you find.
But random points are wasteful. When nearly all of the area hides in a couple of narrow bumps, samples from a plain bell curve mostly land where there is nothing, and the estimate wobbles badly. The classic fix, importance sampling, says: draw your samples from a distribution q shaped like the thing you're integrating, then undo the favoritism by dividing each sample's contribution by q's height there. The catch: you must be able to draw from q and know its exact, honestly-normalized height at any point. Very few flexible models can do both. A flow can — sampling is its forward direction, and exact heights are its stretch receipts.
But what is a vote, really? A single sample can only see one thing — the integrand's height where it landed. A height becomes an area only when multiplied by a width, so each sample speaks for the stretch of axis around it. And the width of that stretch is set by the crowding: where q is tall, samples pack densely and each owns a thin sliver; where q is low, a lone sample must represent a wide stretch — width about 1 / (n·q). So a sample's honest contribution is its patch of area: height × the width it speaks for. The vote f ÷ q is exactly that patch scaled up n-fold — the sample saying, “if everyone's patch looked like mine, the total would be this.” Averaging the votes stitches the patches back into the whole area. Dividing by q is no trick; it converts a height into that sample's fair share of the area.
Check the vote by hand — a two-zone toy you can do in your head
Two zones, each 1 wide: zone A has height 3, zone B has height 1. True area = 4. Use a matched sampler that visits A three-quarters of the time (q = 0.75 in A, 0.25 in B). A sample in A votes 3 ÷ 0.75 = 4. A sample in B votes 1 ÷ 0.25 = 4. Every vote says 4 — each sample's patch (thin-but-tall in A, wide-but-short in B) is an equal share of the area.
Now use a 50/50 sampler instead: the votes become 3 ÷ 0.5 = 6 and 1 ÷ 0.5 = 2. Half say 6, half say 2 — the average is still 4, but now it is right by cancellation, and a small batch can get unlucky. That, in miniature, is the whole demo below.
Try it. The orange shape below is an integrand f; we want its area. The blue curve is the proposal q — the very same bendable bell curve from Part 1. Two hundred samples are drawn from q, and each casts a vote of f ÷ q, drawn as a stem: tall stem, loud vote. Tick “show each vote's patch” to see the patches themselves — the tinted rectangles' combined area literally is the estimate, and at the sweet spot they tile the orange shape almost seamlessly. Bend the proposal until it hugs the integrand, and watch the votes even out while the estimate locks onto the true area.
Why is bend = 0 so bad? Because the votes are wildly unequal. A sample landing at x = 0 — where the plain bell curve loves to put samples but the integrand is low — votes about 0.7. A sample landing out on a bump votes about 17. The average becomes a tug-of-war: dozens of samples muttering “0.7” against the rare one shouting “17!”. Over enough samples these exactly balance — the estimate is unbiased — but any single batch of 200 depends on how many shouters it happened to catch; hence the ±30% wobble. Near the sweet spot every sample casts nearly the same vote, so the same 200 points pin the area to under a percent. And since sampling error shrinks only with the square root of the sample count, matching the shapes buys what brute force would pay for with roughly a thousand times more samples.
Why the blue curve never reaches the orange one — and why that's the point
The proposal q is a probability distribution, so the area under it is locked at exactly 1. The area under the integrand f is the unknown answer — here, 2.363. Two curves with different total areas can never lie on top of each other, so the best q can possibly be is a perfect shrunk copy of f: the same shape, exactly 2.363× shorter everywhere.
Now watch what a vote does. Land anywhere — say where f = 0.40. The shrunk copy has q = 0.40 ÷ 2.363 = 0.169 there, so the vote is 0.40 ÷ 0.169 = 2.363. Land somewhere thinner, where f = 0.10: there q = 0.042, and the vote is 0.10 ÷ 0.042 = 2.363 again. Dividing f by its own shrunk copy recovers the shrink factor — the same number everywhere — and that factor is the ratio of the two areas: 2.363 to 1. The vote is the answer. With a perfectly matched proposal, the very first sample announces the area and the other 199 just agree. Real matches are imperfect, which is why the sweet spot reads ±0.8% rather than ±0.
This exact recipe — with learned bends, in many dimensions — is how flows do serious integrals. In particle physics the integrand is the rate of a collision outcome, summed over every way it can happen (flows are the neural descendants of the classic VEGAS method). In film rendering it is a pixel's brightness, summed over every path light can take to reach it. In chemistry it is a molecule's free energy, summed over its possible configurations. Each time, a flow is trained to be the shrunk copy of a monstrous integrand — and then a modest number of samples does an impossible-looking integral.
The flow brings the pdf: flexible in shape yet exactly normalized by construction — area 1, receipts and all — and usable in both directions, sampling and scoring. Importance sampling brings the number: it weighs the unnormalized integrand f against that area-1 proposal, and the average vote reads off the ratio of their areas — f's missing normalization factor. Sanity check: if f were already normalized, the votes would average to exactly 1.
How is this different from the classic VEGAS method?
It's the same game. VEGAS (1978) also bends a simple distribution toward the integrand, then importance-samples. Its bending is, in modern language, a tiny normalizing flow: each coordinate axis gets its own invertible, adaptive stretch — a row of histogram bins that narrow wherever the integrand is large, and narrow bins mean crowded samples.
The limitation hides in the word each. VEGAS bends every axis independently, so its proposal is always a product of 1-D shapes — in 2-D, a patchwork of axis-aligned rectangles whose density in any cell is forced to be (row weight) × (column weight). Aim it at two peaks placed diagonally and it must also light up the two empty corners of their rectangle — “ghost peaks” it cannot avoid. And no adjustment of rows and columns ever makes a grid hug a tilted ridge, or the spiral from Part 1.
A neural flow removes exactly that constraint. Its layers bend coordinates based on each other — Part 1's swirl rotated each point by an amount depending on its radius, which is coordinates talking to one another — so the proposal can be tilted, curved, and correlated. The price: a flow trains a neural network by gradient descent, where VEGAS adapts its bins with a cheap, robust little iteration. That's the real trade. VEGAS remains nearly free and excellent when the integrand is roughly separable in well-chosen coordinates (physicists pick their variables cleverly to make this true); flows earn their keep when the coordinates are genuinely tangled. Modern integrators often combine the two.
Why integrals are the prize
It is worth saying what those integrals buy. In statistical physics the partition function Z turns into free energy, F = −kBT log Z — the quantity that decides whether a protein folds or a material changes phase; flows that estimate Z (Boltzmann generators) attack it head-on. In Bayesian statistics the same integral is the evidence, ∫ p(data|θ) p(θ) dθ — the number that compares whole models against each other. Both fields spent decades on ladders of intermediate distributions (thermodynamic integration, annealed importance sampling) precisely because no flexible, exactly-normalized q existed. A trained flow is that q, and the ladder collapses to one healthy importance-sampling step — when, and only when, the proposal has truly covered the target. Which is exactly what the lab above taught you to distrust, and to verify.