Models Mathematically Find It More Difficult to Navigate Various Languages

Some languages cost more to run through a model than others, and the usual explanation is that they are harder. The pre-registration behind this piece asks whether they are harder, or whether we are just representing them badly.

One figure makes the case for asking. Sanskrit encodes 5.07 characters per token under a tokenizer aligned to it, and 1.13 characters per token under GPT-4's. That is more than a fourfold swing in efficiency, on the same language, with nothing changed but the encoding.

If a language's cost moves that much when you change the representation, then a cross-lingual efficiency gap measured under one tokenizer is not a fact about the language. It is a fact about the pair.

Same move, one level down

This is the programme's recurring claim, applied below the agent. The substrate a worker runs on decides more than the worker: the environment moved the cost of a coding task while the model already had the skill. Here the substrate is the representation layer, and the question is whether it is doing the same thing to whole languages.

The design is five languages, three scripts and four tokenizers, and it is pre-registered: the hypotheses, the method, the metrics and the falsifier are on the record before the results, which is the only way a result like this is worth anything.

One idea kept firmly in the speculative column

Because Sanskrit is so rule-based and dense, it is tempting to imagine routing meaning through it as a latent interlingua, more efficient than the implicit English-centric hubs models use today. The note flags this as a lovely idea it has no evidence for, and files it as speculation rather than a finding.

Which is the right instinct, and worth naming: the interesting speculation is exactly the thing a pre-registration exists to stop you quietly promoting into a result.

The design, the four ways this comparison goes wrong, and the evidence tiering are in the research note.