Singular learning theory

A from-scratch walk through why counting parameters overstates how complex a neural network is, and what to count instead. No statistics background assumed.

Step 1. The question

You have some data and two candidate models. Say one is a small network and the other is a big one. Which should you trust?

Training loss cannot answer this. The bigger model can always fit the training data at least as well as the smaller one, including the noise. You need a score that rewards good fit and also charges for size. The whole story below is about what the right charge is.

Step 2. Score a model by its average fit

Some vocabulary first. It is all we need.

• The parameters \(w\) are the weights. A model with \(d\) weights has \(w \in \mathbb{R}^d\).
• The loss \(L_n(w)\) is the average error per data point over \(n\) data points when the weights are \(w\). Think cross-entropy.
• The fit of \(w\) is \(e^{-n L_n(w)}\). This is the probability the model assigns to the whole dataset, so low loss means high fit. Statisticians call it the likelihood.
• The prior \(\varphi(w)\) is the range of weight settings you were willing to consider before seeing any data. Picture a uniform spread over a big box.

The Bayesian way to score a model is to ignore its single best weight setting and instead ask how well it fits on average across every setting in the prior.

\[ Z_n = \int \varphi(w)\, e^{-n L_n(w)}\, dw \]

A big model has an enormous weight space, and almost all of it fits the data terribly. That drags its average down. It only wins if its good region fits so much better that it pays for all the wasted space. So a penalty for size appears without anyone adding one by hand.

People usually work with \(F_n = -\log Z_n\), so that lower is better, like a loss. It is called the free energy, a name borrowed from physics. Read it as the model's overall score.

Step 3. Data pins parameters down

How big is the penalty? Start with one parameter. As you collect more data, fewer and fewer values of that parameter still fit well. The instrument below shows this.

With the curved minimum, 100 times more data shrinks the average fit by a factor of 10. That is \(\sqrt{100}\). In general the good region has width about \(1/\sqrt{n}\), so the average fit scales as \(n^{-1/2}\).

Now take \(d\) parameters and suppose each one behaves this way. The good fraction of weight space is the product over all of them.

\[ \left(n^{-1/2}\right)^{d} = n^{-d/2} \]

Taking \(-\log\) turns this into a penalty of \(\tfrac{d}{2}\log n\). Add the loss at the best weights \(\hat w\) and you get the classic approximation to the score.

\[ F_n \approx n L_n(\hat w) + \frac{d}{2}\log n \]

This formula is called BIC, the Bayesian information criterion (Schwarz, 1978). The first term is fit. The second is the charge for size, and it is \(\tfrac{1}{2}\log n\) per parameter. When earlier explanations say a model with \(d\) parameters has complexity \(d/2\), this is all they mean.

Before moving on, switch Instrument 1 to the flat minimum. Now 100 times more data shrinks the average fit by only about 3.2, which is \(100^{1/4}\). A flatter minimum is harder for data to pin down, so it is charged less. The exponent changed from \(1/2\) to \(1/4\). Hold on to that.

Step 4. The hidden assumption

The \(d/2\) rule assumed that around the best weights, the loss curves upward like a bowl in every one of the \(d\) directions. Models where this is true are called regular. Models where it fails are called singular.

Neural networks are singular. Here is the smallest example. Take a network with one neuron

\[ f(x) = a \tanh(b x) \]

with two weights \(a\) and \(b\), and suppose the right answer is \(f(x) = 0\) for every input. Setting \(a = 0\) gives a perfect fit whatever \(b\) is. So does setting \(b = 0\) whatever \(a\) is. The best weights are not a single point. They form two crossing lines, and along those lines the loss is completely flat.

Real networks are full of this. You can swap two hidden units, or scale one layer up and the next one down, or switch a neuron off in several ways, and the network computes exactly the same function. Every such freedom is a direction the data never has to pin down. Charging \(\tfrac{1}{2}\log n\) for it is overcharging.

Step 5. Count volume instead of parameters

Here is the fix. Define the excess loss \(K(w)\) as the loss at \(w\) minus the best achievable loss, so \(K = 0\) at the best weights. Then ask how much of weight space is within a tolerance \(\varepsilon\) of the best.

\[ V(\varepsilon) = \text{volume of } \{\, w : K(w) < \varepsilon \,\} \]

Watanabe proved that for small \(\varepsilon\) this volume always shrinks like a power of \(\varepsilon\).

\[ V(\varepsilon) \approx c\, \varepsilon^{\lambda} \]

The exponent \(\lambda\) is the learning coefficient. You will also see it called the RLCT, short for real log canonical threshold, which is its name in pure mathematics. Small \(\lambda\) means the good region shrinks slowly as you get stricter. Good weights are plentiful.

For a bowl in \(d\) dimensions the good region is a ball of radius \(\sqrt{\varepsilon}\), whose volume is proportional to \(\varepsilon^{d/2}\). So a regular model has \(\lambda = d/2\) and nothing changes. For other shapes \(\lambda\) is smaller. All five shapes below have two parameters, so the old rule would say \(d/2 = 1\) for each of them.

One detail for completeness. When several flat sets cross at a point, as in the cross shape, the volume picks up an extra logarithmic factor, \(V(\varepsilon) \approx c\,\varepsilon^{\lambda}(-\log \varepsilon)^{m-1}\). The integer \(m\) is called the multiplicity. It is a small correction and you can ignore it on a first pass.

Step 6. The corrected score

Go back to the average fit \(Z_n\). With \(n\) data points, only weights whose excess loss is below roughly \(1/n\) contribute to the average. So the size of \(Z_n\) is set by \(V(1/n) \approx n^{-\lambda}\). Taking \(-\log\) gives Watanabe's formula.

\[ F_n \approx n L_n(w_0) + \lambda \log n \]

Compare it with BIC in Step 3. It is the same formula with \(d/2\) replaced by \(\lambda\). BIC is the special case where every direction is a bowl. (The full statement also has a \(-(m-1)\log\log n\) term from the multiplicity.)

The same \(\lambda\) controls test performance. If you predict by averaging over all good weights, the expected gap between your test loss and the best possible test loss is about \(\lambda / n\). So \(\lambda\) plays the role that the parameter count was supposed to play, as the model's effective complexity.

Step 7. Why this predicts sudden jumps in learning

Read the formula as a price list. Any region of weight space pays \(n\) times its loss, plus \(\lambda \log n\) for its complexity. Suppose there are two candidate solutions. Solution A is simple (small \(\lambda\)) but slightly inaccurate. Solution B is accurate but complex.

With little data, the complexity charge dominates and A is preferred. The loss charge grows in proportion to \(n\) while the complexity charge only grows like \(\log n\), so beyond some amount of data B becomes cheaper. The switch is abrupt. Physicists call this kind of switch a phase transition.

Step 8. Measuring \(\lambda\) on a real network

Working out \(\lambda\) exactly with pen and paper is very hard and has only been done for a handful of small model families. What made SLT usable in deep learning is the local learning coefficient (LLC), introduced by Lau, Furman, Wang, Murfet and Wei. It asks the volume question in the neighbourhood of one trained checkpoint \(w^*\), and it can be estimated numerically.

The idea is simple. Jiggle the weights randomly around \(w^*\) and watch how much the loss rises on average. If it barely rises, many nearby weights are just as good, so the good region is large and \(\lambda\) is small. If it rises sharply, \(\lambda\) is large.

\[ \hat\lambda(w^*) = \frac{n}{\log n} \Big( \text{average loss of jiggled weights} - \text{loss at } w^* \Big) \]

The jiggling is done with SGLD (stochastic gradient Langevin dynamics). That is ordinary SGD with random noise added at each step, plus a gentle pull back toward \(w^*\) so the weights explore the neighbourhood without wandering off.

The best-known application is called developmental interpretability. Hoogland and coauthors tracked the LLC during training for a small language model (about 3 million parameters) and for a small transformer trained to do linear regression from examples in its prompt (about 50 thousand parameters). The LLC curve divided training into distinct stages, and most of those stages matched changes in what the model was doing internally. The language model, for example, first learned pairs of adjacent words, then longer word sequences, then a circuit for copying patterns seen earlier in the prompt.

Where the theory stops

These are the places where people most often read too much into SLT.

1. The theory is about averaging over all good weights. Training gives you one set of weights. Everything above describes the Bayesian average from Step 2. SGD does not compute that average, and nobody has proved that it behaves as if it did. The link between \(\lambda\) and what SGD does is an experimental finding in small models.
2. The formulas are for large amounts of data. They become exact as \(n\) grows with the model held fixed. Instrument 2 shows the cost of stopping early. On the cross, the measured slope is about 0.34 when the exact answer is 0.5, because the logarithmic correction has not faded yet.
3. Curvature does not determine \(\lambda\). A popular way to measure flatness is the Hessian, the matrix of second derivatives of the loss. It only sees bowl-like curvature. Each curved direction contributes \(1/2\) to \(\lambda\). Each direction the Hessian calls flat contributes somewhere between \(0\) and \(1/4\), and the Hessian cannot tell you where in that range. Compare the cross with the flat-bottomed bowl in Instrument 2.
4. LLC estimates depend on how you jiggle. Step size, noise level and the strength of the pull toward \(w^*\) all shift the number. Most papers therefore compare LLC values across checkpoints of one training run and avoid reading a single value as absolute.
5. Small \(\lambda\) does not guarantee your trained network tests well. The \(\lambda/n\) result in Step 6 is for predictions averaged over all good weights.

Glossary

What to read next

• Liam Carroll, Distilling Singular Learning Theory. A blog sequence that goes through the volume picture and the free energy formula with small examples. Start here.
• Wei, Murfet and coauthors, Deep learning is singular, and that's good. A paper-length argument for why this matters for neural networks.
• Lau and coauthors, The Local Learning Coefficient. Defines the LLC and how to estimate it.
• Hoogland and coauthors, Loss Landscape Degeneracy and Stagewise Development in Transformers. The training stages result.
• Sumio Watanabe, Algebraic Geometry and Statistical Learning Theory (2009). The original source. It needs a lot of pure mathematics, so read it last.