The Default Human

In 2021 I was given £36,500 by the Global Challenges Research Fund to work on a problem that sounded, at the time, faintly eccentric: could a machine automatically redraw a children’s picture book for a different culture?1 Not translate it. Redraw it. A child in Kerala reading a story set in a British seaside town does not know what a fish-and-chip shop is; a child in Yorkshire does not know a dosa stall. Publishers already solve this by hand, with an illustrator, one book at a time, which is why it reaches almost none of the world’s languages.

Five years later the same question has a fashionable name — cultural alignment — and everyone has an opinion about it. So here is mine, which is less comfortable than the one you would expect from someone who has spent six years on this.

The tax you pay for your alphabet

Start with the least arguable part, because it is arithmetic rather than politics.

Language models do not read characters. They read tokens, and the tokenizer that produces them is fitted to whatever the model saw most. English got the efficient encoding. Everything else got what was left. Petrov and colleagues measured this properly and found that the same text, translated, can take up to 15 times as many tokens depending on the language it is written in.3 A later study across more than 200 languages found non-Latin scripts and morphologically rich languages routinely paying three to five times the English rate.4

That is not an aesthetic complaint. Tokens are the unit of billing, the unit of latency, and the unit of memory. So a speaker of an unlucky language pays more money for the same answer, waits longer to receive it, and can fit less of their problem into the model’s context before it starts forgetting the beginning.3 The same query, the same model, the same day — and a structurally worse service, priced higher, for the people least able to absorb either.

See it for yourself. I measured this: Article 1 of the Universal Declaration of Human Rights, in 19 languages across 11 writing systems, run through two of OpenAI’s tokenizers. The same sentence costs 33 tokens in English and 305 in Amharic. Open the interactive → — the data and the script that produced it are published with it.

I find this the most clarifying fact in the whole debate, because nobody chose it. No committee decided that Telugu speakers should pay a premium. It fell out of an engineering decision about compression, made by people optimising for the corpus in front of them. Which is exactly how this kind of bias usually arrives: not as prejudice, but as a default that nobody had a reason to question.

Data, not speakers, decides who the model serves

The obvious assumption is that big languages are well served and small ones are not. It is wrong, and the counterexample is instructive.

Stanford’s work on the language divide points out that Swahili has roughly 200 million speakers but too little digitised text for models to learn from properly, while Welsh — with a fraction of that number — does comparatively well, because it has been documented, digitised and institutionally preserved for decades.5 Models work well for the 1.52 billion people who speak English, less well for 97 million Vietnamese speakers, and worse again for the 1.5 million who speak Nahuatl.5

So the variable is not how many people speak your language. It is how much of your language someone bothered to write down in a machine-readable form, and whether anyone funded that. Which means the map of who AI serves well is, to an uncomfortable degree, a map of historical investment in linguistic infrastructure — colonial, national, academic. The model inherits that map without ever being told it is a map.

Whose values got called “alignment”

Now the harder claim. A model trained overwhelmingly on one part of the world does not represent that part as a culture. It represents it as the absence of culture — the neutral case, from which everything else is a departure requiring a modifier. Ask for a wedding and you get one specific wedding. Ask for a family, a breakfast, a professional, a beautiful house, and the defaults have a postcode.

The literature calls this WEIRD — Western, Educated, Industrialised, Rich, Democratic — borrowing a term Henrich, Heine and Norenzayan coined in 2010, when psychology noticed that its findings about “human nature” rested on samples drawn overwhelmingly from Western university undergraduates — among, as they put it, “the least representative populations one could find for generalizing about humans”.96 The parallel is close enough to be embarrassing. A discipline generalising from an unrepresentative sample, and calling the result universal, is not a new failure mode. We have simply automated it and put it behind an API.

What makes the word alignment worth pausing on is that it names a destination without naming it. Aligned to what? The honest answer, most of the time, is: to the preferences of the annotators who were hired, under the guidelines they were given, at the companies that could afford to run the process. That is a legitimate thing to optimise for. It is not a neutral thing, and the vocabulary invites us to forget the difference.

The finding that ruins the easy version of this argument

Here is where I part company with most people who agree with me so far.

If the problem is Western defaults, the obvious fix is less Western default. Zhou, Constantinides and Quercia tested that directly across GPT-3.5, GPT-4, Llama-3, BLOOM and Qwen. Models with lower alignment to WEIRD values did produce more culturally varied responses — and were also 2–4% more likely to generate output that violated human rights, particularly on gender and equality. The examples are not subtle: endorsements of the ideas that a man who cannot father children is not a real man, or that a husband should always know where his wife is.6

I do not think this vindicates the current defaults, and I want to be careful here, because that is exactly how the finding will be used. Human rights are not a Western cultural preference, and treating them as one is a move with a long and disreputable history. But it does demolish the lazy formulation — that cultural representativeness is straightforwardly good and Western skew is straightforwardly bad. Sometimes a model reproducing local majority opinion faithfully is a model reproducing something a great many people in that locality are fighting against.

Anyone selling you a clean answer to that has not read the tension carefully. I certainly do not have one. What I have is the conviction that the trade-off should be visible and argued about, rather than resolved silently in a guidelines document by whoever happens to be writing it.

And we cannot currently tell whether any of it works

The methodological news is worse than the political news.

Most cultural alignment evaluation works by giving the model survey questions — typically from instruments like the World Values Survey — and comparing its answers to a country’s human responses. Khan, Casper and Hadfield-Menell examined the three assumptions this rests on, and found all three fail.7 Cultural alignment is not stable: response shifts caused by trivial changes in question formatting frequently exceed the actual differences between cultures. It is not extrapolable: alignment measured on a few dimensions predicts alignment on held-out dimensions about as well as chance. And it is not steerable: prompting a model to adopt a culture produces erratic patterns that no human population resembles.

Read that again, because it undercuts a great deal of published work, including work I am sympathetic to. If reformatting your question moves the result more than swapping Japan for Germany does, then you are not measuring a property of the model. You are measuring your own questionnaire. A field cannot claim to have reduced cultural bias using an instrument that unreliable, and the honest position is that many confident claims in this area — in both directions — are currently unsupported.

Self-awareness, not substitution

Six years on the same problem has moved me from the position most people start at to one that sounds duller and is, I think, correct.

The goal is not to find the right culture to encode. There isn’t one, the attempt to pick one is how we got here, and the evidence suggests that swapping the default out wholesale trades one set of harms for another. The goal is for the system to know that it is standing somewhere — that its defaults are defaults, that the person in front of it may not share them, and that this is a thing to be established rather than assumed. That is the argument behind CALM: Culturally Self-Aware Language Models, which we published at NeurIPS last year,2 and it is the same instinct as the picture-book problem: not “what should this child’s story be”, but “does the system know whose story it is telling”.

Three things follow, and none of them are technical.

• The tokenizer tax is the easiest win and nobody is taking it. It is a measurable, unambiguous, purely engineering inequity that makes AI more expensive for the world’s poorer speakers. It requires no philosophy to fix. It persists because the people paying it are not the people in the room.
• Digitisation is cultural policy, and it is being done by accident. Whether your grandchildren’s language works with these systems is being decided now, by who funds corpus-building. Swahili and Welsh should be the standing example in every ministerial briefing on AI. It almost never is.
• Fix the measurements before trusting any claim, including mine. On the current evidence, “our model is culturally aligned” is not a verifiable statement. Until the evaluations are robust, that phrase belongs in scare quotes — on marketing copy, on benchmark leaderboards, and in my own papers.

When UNESCO's statistics institute last put a headline number on this, in 2017, it counted 617 million children and adolescents not reaching minimum proficiency in reading and mathematics.8 It is an old figure and I quote it with that caveat, but it is the one that was in front of me at the start, and it is why I began on picture books rather than benchmarks, and it is the thing I try to keep in view when this subject becomes an argument about definitions. Somewhere in that number is a child who would read more if the story looked like somewhere they had been. Whether the technology of this decade reaches them is not going to be settled by a leaderboard.