AI's progress may be stalling outside maths and code, researchers warn

What's the deal? AI researcher François Chollet has raised the possibility that AI gains are concentrated in maths and code, while progress elsewhere may be slowing. He argues these "verifiable" domains can be pushed "arbitrarily far," but other areas stay "bottlenecked by human generated data."

The debate: Chollet, the creator of KerasDealroom has a profile for this one. Try Dealroom → and founder of NdeaDealroom has a profile for this one. Try Dealroom →, posed the question on X. Economist Luis Garicano, a London School of Economics professor and former Member of the European Parliament, called it a "great observation" and agreed.

Why it matters? At stake is whether recent model improvements reflect genuinely higher intelligence or simply more human data being fed into training. Chollet notes performance in "non-verifiable areas has kept improving steadily, albeit much slower than for math and code." Much depends on which explanation is correct.

Why now? Garicano argues frontier labs are "almost doing no pretraining," likely because that approach is "bottlenecked by data — the world is running out of data." Labs have instead pivoted to reinforcement learning, which offers clear reward signals in maths and code.

What could go wrong? Reinforcement learning has "much narrower applicability outside verifiable domains," Garicano wrote. Where reward signals are murky, progress "may depend on finding new sources of human-generated data or new training paradigms altogether."

The signal: The exchange sharpens a growing industry worry: that the data wall could cap AI gains in everything but the fields where answers are easy to check. If so, the next leap forward may require a new method rather than more scale.

Image credit: hackNY