Looped Mixture of Experts: Two Efficiency Tricks That Finally Stack

TL;DR: A paper called “Scaling Laws for Looped Mixture of Experts” shows that recurrence (looping a model’s layers) and sparsity (Mixture of Experts) are not competing tricks but complementary ones, and when you combine them with the right ratio a looped MoE matches a roughly 2x larger non-looped MoE on reasoning benchmarks at the same training compute.

For years the two cheapest ways to make a transformer smarter without paying full freight have been studied in separate rooms. One room: Mixture of Experts, where you have a huge pile of parameters but only fire a small fraction per token. The other room: looped or recurrent transformers, where you reuse the same layers multiple times to get more computational depth without adding weights. Both save money. Nobody had a clean theory for what happens when you run them together.

That is the gap “Scaling Laws for Looped Mixture of Experts” (arXiv, posted across cs.AI, cs.CL, and cs.LG) sets out to close. The headline claim is specific and testable: the two efficiency axes stack, and the paper gives you a formula to predict how much.

What problem does this actually solve?

Think about the two bottlenecks a model trainer fights. One is active compute, the FLOPs you burn per token at inference. The other is total parameters, the memory footprint you have to store and move around. MoE attacks the first: you can have a trillion parameters of capacity but only light up a few billion per token. Looping attacks the second: you can get the reasoning depth of a deeper model without the parameter count, because you are running the same block again instead of stacking a new one.

The paper’s authors argue these address different constraints, so you should be able to spend on both. The problem is that until now you had no principled way to decide how much to loop versus how much to grow the expert pool. You were guessing. Existing scaling laws modeled recurrence or sparsity in isolation, which is like having a mileage chart for city driving and a separate one for highway but nothing for a mixed commute.

What is a “Loop Scaling Law” and why should I trust it?

The contribution is what they call Loop Scaling Laws, the first scaling law to jointly model recurrence and sparsity alongside the usual suspects of model size and data. The mechanism at the center is a “bounded, sparsity-conditional recurrence mapping.” Unpacked: looping gives you an effective-parameter gain, as if you had a bigger model, and that gain is bounded (you cannot loop forever and get free intelligence) and it depends on sparsity (the more sparse your MoE, the more you get out of each loop).

Two things make this more than a curve fit. First, the law predicts held-out loss more accurately than prior alternatives. That is the real test of a scaling law: does it forecast the loss of models you did not fit it on. Second, it recovers the standard dense and MoE scaling laws as special cases. When you set the loop count to one, you should get the known MoE law back, and you do. A law that collapses to the established results at its boundaries is a law that is probably describing something real rather than overfitting to one lab’s runs.

I would still want to see independent replication before treating the exact coefficients as gospel. All three arXiv listings here are the same paper cross-posted, not three separate confirmations, so this is one result from one group. But the structure of the argument is the kind that holds up or breaks loudly, and that is worth something.

How big is the payoff in practice?

The numbers the authors report are concrete. Sparsity delivers roughly 3x active-parameter efficiency, meaning you get the quality of a model firing 3x as many parameters per token. Recurrence yields roughly 2x total-parameter efficiency on reasoning, meaning you match a model with twice the stored weights. And joint scaling pushes the frontier further than either alone.

The result that should make infrastructure people sit up: at trillion-token scale, at matched training compute, a looped MoE with law-derived recurrence matches a roughly 2x larger non-looped MoE on reasoning benchmarks. Same compute budget, half the parameter footprint, equal reasoning performance. That is not a toy-scale curiosity. Trillion-token training is the regime real frontier models live in.

There is a bonus that the paper flags and that I think gets undersold. Because the model loops, you can scale at test time through recurrence. You can spend more loops at inference on a hard problem and fewer on an easy one. That is a different flavor of test-time scaling than the long chain-of-thought approach that reasoning models popularized. Instead of generating more tokens, you run the same weights more times over the same tokens. For latency-sensitive or cost-sensitive serving, having a depth dial you can turn per request is genuinely useful.

What is the catch?

The word “bounded” is doing quiet, important work. Looping is not a money printer. The effective-parameter gain saturates, so the second loop buys you less than the first and the tenth buys you almost nothing. The whole value of the law is telling you where that curve bends so you do not waste compute past the point of return. If you ignore the bound and just loop more, you get the cost without the benefit.

The second catch is operational. Looping trades parameters for sequential compute. You store fewer weights, but you run them more times, which means higher latency per token unless you engineer around it. For memory-constrained deployment (think smaller GPUs, edge, or anywhere VRAM is the wall), that trade is great. For throughput-maximizing serving where latency is already tight, looping can work against you. The paper gives you the efficiency frontier, not a free lunch, and which side of the frontier you want depends entirely on whether memory or latency is your binding constraint.

The third caveat is the one I keep coming back to: reasoning benchmarks are where recurrence shines in this work. The 2x total-parameter claim is specifically “on reasoning.” Whether the same ratio holds for retrieval-heavy, factual, or multilingual workloads is not something these abstracts establish, and I would not assume it transfers cleanly.

Practitioner’s take: if you are not training frontier models you cannot act on this directly, but you can use it to read the next wave of model releases. When a lab ships a model that punches above its parameter count on reasoning while staying cheap to store, looping is now a plausible reason why, and you should ask whether that advantage holds on your actual workload or just on the reasoning evals. If you are fine-tuning or serving open models, watch for looped MoE architectures showing up in open weights, because they let you hit a given reasoning quality on less VRAM at the cost of more sequential passes, which is exactly the trade you want on a constrained box and exactly the trade you do not want when latency is your problem. The thing most readers will miss: the headline 2x is a matched-compute reasoning result from a single paper, so treat it as a strong hypothesis about architecture design, not a settled discount you can bank on for your use case.