Speakers of Less Common Languages Pay More for AI

Speaking a rare language costs more

When people use a less common language, AI models use more tokens than they do for English or other popular languages. This leaves less room for documents and chat history. It also makes each answer take more work. As a result, these users are at a disadvantage.

Key numbers

• A study by CLEAR, a group for language equality in AI research, found the most expensive languages among the 20 most common online languages need 1.5 to 5 times more tokens than the cheapest ones.
• When the study covered 200 languages, the gap in token cost within one AI model grew to 17 times.
• CLEAR calls this extra cost the "hidden tokenizer tax."

Why tokens matter

A tokenizer breaks text, images, or audio into small pieces called tokens. The context window is the maximum amount of information an AI can handle in one session, and it is usually measured in tokens. Writing systems matter too. Languages that use scripts the tokenizer rarely sees, such as Burmese, Tibetan, and Khmer, are at a big disadvantage with almost every AI model.

Why it matters

Higher token use means higher costs, slower replies, and less space for the AI to remember earlier parts of a conversation. This affects startups and users who do not speak English. Experts say perfect equality may not be possible, but large gaps should be explained and reduced. One expert suggests that AI labs publish cost comparisons for each language.

What to watch

• Whether big AI labs publish language-by-language cost reports.
• Whether companies make measurable improvements for the languages that are most affected.
• Whether companies stop using token use as a measure of employee productivity.
• Whether the tactic of pairing expensive models for hard tasks with cheaper models for simple tasks spreads to more businesses.