Yes. The same text needs about 37% more tokens in French than in English on current generation tokenizers, and it needed 61% more on the previous generation. At identical price and identical emission factor, writing in French therefore costs mechanically more, in euros as in carbon. The extra cost lands on the prompt reading phase, the cheaper of the two.
The gap comes from the corpus used to build the split, not from the language. And it is far wider for poorly served languages: Burmese needed 11.6 times more tokens than English two years ago.
The measurement, and its protocol
No published source covered both French and low-resource languages on the 2025-2026 tokenizers. The measurement below was made for this course, on 9 September 2026, and its protocol is given so you can run it again.
Corpus: FLORES-200 devtest, 1,012 parallel sentences translated by professionals. Splitting done with tiktoken for the OpenAI encodings and Hugging Face's tokenizers library for the rest. The ratio is the total token count for the language divided by the English count on the same sentences.
Validation: the values obtained on cl100k reproduce the reference table from Petrov et al. (NeurIPS 2023) to within 2%, and match to the decimal Removable and Irreducible: A Token-Cost Ledger for the Multilingual Tokenization Tax (arXiv:2609.00378, July 2026), which publishes on the same corpus and the same encoding.
| Language | cl100k (GPT-4) | o200k (current generation) |
|---|---|---|
| English | 1.00 | 1.00 |
| Portuguese | 1.48 | 1.23 |
| Spanish | 1.54 | 1.32 |
| German | 1.59 | 1.31 |
| French | 1.61 | 1.37 |
| Simplified Chinese | 1.87 | 1.25 |
| Swahili | 1.95 | 1.49 |
| Arabic | 3.03 | 1.38 |
| Hindi | 4.76 | 1.57 |
| Amharic | 7.59 | 5.78 |
| Burmese | 11.62 | 3.16 |
| Santali | 12.74 | 13.70 |
And the pair you need to turn a volume of text into a volume of tokens:
| Tokenizer | tokens per word, English | tokens per word, French | characters per token, French |
|---|---|---|---|
| cl100k | 1.24 | 1.71 | 3.61 |
| o200k | 1.23 | 1.44 | 4.28 |
What changed, and what did not
Going from cl100k to o200k doubled the vocabulary size, and the gain is real. It is not spread evenly.
The languages that gain most are the ones that started furthest back. Burmese goes from 11.62 to 3.16, a drop of 73%. Hindi from 4.76 to 1.57, Arabic from 3.03 to 1.38. French, already well served, gains only 15%.
So the reduction in inequality between 2023 and 2026 barely benefits French at all. Santali actually goes backwards, from 12.74 to 13.70: the enlarged vocabulary was filled with other languages.
One detail not to pass on wrongly: the o200k_harmony encoding produces exactly the same token count as o200k_base across the 27 languages tested. Both share the same merge rules and differ only in special formatting tokens, so there is no compression gain.
Question
A French-speaking customer service team handles 400 million tokens a month. They are considering switching their system prompts to English to cut the bill. What do you tell them?
Choisissez une réponse pour voir l'explication.
The equity problem, in numbers
The mechanics turn political here. Better to put it in figures than in outrage.
A Burmese speaker asking the same question as an English speaker, of the same model, pays 3.16 times more and burns 3.16 times more prefill energy. On models still served in cl100k, it was 11.62 times.
Beyond the overcharging, it cuts into real capacity. With an identical 200,000 token context window, the English speaker fits about 976,000 characters in, the French speaker about 857,000, the Burmese speaker a fraction of that. At the same technical limit, the service is not the same depending on the language.
The African Language Tax (arXiv:2606.24460, June 2026) covers 22 African languages and 11 tokenizers on FLORES-200+, and ranks the splits by average extra cost: Gemma 4 leads at 2.43 times English, Llama 4 at 2.52, cl100k last at 3.40. The worst case recorded is N'Ko at 10.87 tokens per word, an extra cost of 8.92 times English on o200k_base.
What a consultant does with this
For a multilingual service, choosing a model is also choosing a tokenizer, so choosing a cost and a footprint per language. That dimension appears in no provider comparison grid, and it takes half a day to measure on the client's real logs.
One methodological caveat, worth stating if you reuse these figures. FLORES-200 is translated encyclopaedic text: no conversation, no code, no system prompts. Conversational French full of elisions and technical French stuffed with anglicisms will not give exactly these ratios. The characters per token ratio holds up better than the tokens per word ratio, because splitting on spaces inflates the French word count.
FAQ
Does French consume more tokens than English?
Yes, about 37% more for the same text on current o200k-style tokenizers, and 61% more on the previous cl100k generation. Measured on FLORES-200 devtest, 1,012 parallel sentences, in September 2026, validated against the reference table from Petrov et al. published at NeurIPS 2023.
Why do some languages cost far more in tokens?
Because the subword split is built statistically on a large corpus that is overwhelmingly English. Sequences that are frequent in that corpus get their own token, the others have to be broken into fragments. A poorly represented language is therefore written in many more tokens, with nothing to do with its structure.
Have the new tokenizers fixed the gap between languages?
Partly, and very unevenly. Going from cl100k to o200k gains 54% for Arabic, 67% for Hindi and 73% for Burmese, but only 15% for French, which started off well served. Santali goes backwards, from 12.74 to 13.70 times English: the enlarged vocabulary was filled with other languages.
How many tokens is a word in French?
About 1.44 tokens per word on current tokenizers, against 1.23 in English, so roughly 4.3 characters per token in French against 4.9 in English. These values come from translated encyclopaedic text and do not transfer as they are to conversation or to code.
Should you write your prompts in English to cut the bill?
The gain exists but it is smaller than it looks. A prompt written in English saves about 27% of tokens on the portion concerned, yet input is the cheapest item in energy terms, roughly 35 times less than an output token at batch size one, and it is often cached. The real lever sits on the length of the answers, not on the language of the instructions.