The science of whether AI reasons is not settled

Quanta walks through why chain-of-thought results keep pointing in opposite directions, from accuracy collapse to Olympiad gold.

Quanta Magazine published a feature by John Pavlus on 31 July 2026 examining what large reasoning models are actually doing when they produce chains of thought — the streams of intermediate text these systems emit before answering. The piece frames the question as genuinely open rather than close to resolution.

The evidence it assembles points both ways. A team of researchers at Apple argued that chain-of-thought reasoning is an "illusion of thinking" subject to complete accuracy collapse under surprisingly simple conditions. Against that, reasoning models have taken gold medals at the International Mathematical Olympiad, and in May 2026 a general-purpose reasoning model from OpenAI solved a long-open mathematical research problem in a single shot. Both observations are real. Neither settles what mechanism produces them.

This matters for anyone forecasting from AI capability. Almost every argument that expertise — medical, legal, engineering — is about to get very cheap runs through an assumption that these systems reason in a way that generalises beyond their training distribution. If they instead succeed through something narrower that happens to cover the benchmarks, the capability curve is still real but the extrapolation from it is not. That is a difference in error bars, not in direction.

The honest caveat is that this is a survey of an unresolved argument, not a result. It reports no new experiment and settles nothing. Its value is in showing how much of the confidence on both sides is currently running ahead of the science.

Source: Quanta