Audit questions the uniformly rosy research record on AI in schools

All 39 meta-analyses of AI in education report benefits, but an audit of 14 found serious flaws, and a cleaner early estimate averages near zero.

A forensic audit by Patrick O'Neill, a professor at Ivy Tech's Bloomington, Indiana campus, has found serious conceptual and statistical flaws in 14 of the 39 meta-analyses on AI in education. All 39 report substantial benefits. The 14 audited papers appeared in journals from Springer Nature, Wiley, SAGE and Taylor & Francis, have more than 3,000 Google Scholar citations, and gain about 200 more each month. Jill Barshay of The Hechinger Report reported the findings on October 26, 2026.

O'Neill described impossible statistics, hallucinated references and overstated conclusions, some visible from the abstracts alone. Some underlying studies cited generative-AI papers that predate the technology's public release. Many did not measure AI's effect on learning at all: one compared nursing students given three hours a week of AI-using instruction with peers who had no study sessions. One meta-analysis, published in a Nature journal in May 2025, was retracted nearly a year later after two Norwegian scholars complained. O'Neill also said two-thirds of calls for papers on AI in education specifically invited research on benefits.

This matters for access to knowledge. If AI tutoring really can make learning cheaper and more widely available, schools need to find the uses that work. Inflated evidence sends money and policy toward the wrong tools and slows that search. O'Neill's own preliminary meta-analysis of 44 well-designed experiments found outcomes from very helpful to very harmful, averaging roughly zero. That does not mean AI is harmless or useless. Effects depend on how it is used.

Watch for better-specified reviews. Beth Tipton of Northwestern called a single broad meta-analysis of all AI uses 'crazy' and said the use must be defined first. Susanna Loeb's Stanford team screened almost 900 studies and found only 20 good enough for its AI Hub for Education summary, with 32 more since. She says it is too early for a useful meta-analysis, since few high-quality studies are in US K-12 schools. The audit covered only 14 of the 39 papers, and the source text is cut off near its end.

Source: Hechinger Report