Many MindedWriting in Burmese or Khmer costs more AI tokens than writing in English
A study from the Coalition for Language Equity in AI Research, released in September and covered by Rest of World on 26 October 2026, measures how many tokens AI models spend on different languages. Tokens are the chunks of text a model processes, and a tokenizer splits raw input into them. The researchers call the extra burden on some languages a "tokenizer tax."
Among the 20 most popular internet languages, the costliest used 1.5 to 5 times as many tokens as the cheapest. When the study widened to 200 languages, the gap in one model grew 17 times. Script matters as well as rarity: Asturian has few speakers and no Common Crawl data, yet benefits from Latin script and related languages like Italian. Burmese, Tibetan, Tifinagh, Khmer, Lao and Ol Chiki, written in scripts tokenizers rarely saw, were consistently the most disadvantaged across nearly all models analysed.
This matters because more tokens mean higher bills, more latency, and less room in the context window for documents and conversation history. The study says the effect hits hardest where compute and connectivity are limited. A key knowledge tool becomes pricier and less capable for the people who already have the least access, and it narrows who can use AI to communicate and learn in their own language.
Watch whether labs design tokenizers for under-served scripts. Maya Research's Dheemanth Reddy argues better data alone won't fix this, and Activate AI's Pratyush Choudhury wants every major lab to publish a language-by-language cost scorecard. Reddy also expects falling token prices to soften the direct cost over time. Those are industry opinions, not study findings. The 17-times figure applies to one model, and Choudhury concedes perfect language equality may be unrealistic.
Source: Rest of World