SEA-LION v4.8 keeps a tokenizer that spells Khmer out in bytes

What AI Singapore released on 18 September 2026

AI Singapore announced Nemotron-SEA-LION-v4.8 on 18 September 2026, built with NVIDIA on the Nemotron 3 family. Four checkpoints under the MIT licence: a 30B-A3B and a 120B-A12B, each as a continued-pretrained base and a post-trained model. Funded by the National Research Foundation Singapore. The technical report is arXiv:2609.18310. It is the first mixture-of-experts architecture in the SEA-LION line and the first built on Nemotron.

The training bill is in Table 3 of the report. The 30B model saw 150B continued-pretraining tokens on 32 H200s for 114.23 hours; the 120B saw 33.5B tokens on 32 H200s for 186.65 hours. The mixture gives SEA-Instruct data a sampling weight of 10 each for Indonesian, Malay, Burmese, Tamil and Vietnamese, and parallel data a weight of 2 each for Indonesian, Khmer, Lao, Malay, Burmese, Tamil, Thai, Vietnamese and Chinese.

Section 3.2 of the report contains the sentence that matters: "We retain the original tokenizer of each Nemotron 3 foundation model without modification." The report then names the consequence directly, saying that for Khmer, Lao and Tamil the tokenizer produces approximately five times as many tokens as the previous SEA-LION tokenizer. We measured it.

Our measurement: Khmer costs 14.7 times what English costs

We ran four tokenizers over the FLORES-200 devtest split, 1,012 parallel sentences per language, and counted. The tokenizers are the one SEA-LION v4.8 inherits (Nemotron 3, vocabulary 131,072), the Gemma 3 vocabulary behind SEA-LION v4 (262,145), the Llama 3.1 vocabulary behind SEA-LION v3 (128,256), and the custom SEABPE tokenizer from the original 2023 SEA-LION 7B (256,000).

Fertility is tokens per character; lower is more compact. FLORES-200 devtest, 1,012 sentences per language. Our measurement with tokenizers 0.22.2 and sentencepiece, 21 September 2026. Tokenizer files pulled from the Hugging Face repositories on the same date.

The report's five-times claim holds for two of the three languages it names, and understates them. Against the original SEABPE tokenizer, Khmer is 7.75x and Lao is 6.48x. Against the Gemma 3 vocabulary of SEA-LION v4 it is 5.52x and 4.96x, which is where "approximately five times" comes from. Tamil does not reproduce. It is 1.64x against SEABPE and 1.52x against Gemma 3, nowhere near five, on either comparison. We do not know which previous tokenizer the report's figure used, and the figure labels are not in the extractable text, so we report both.

Why five times is the wrong frame: Khmer is at the byte floor

A ratio against another tokenizer hides what is actually happening. Khmer text in UTF-8 averages 2.828 bytes per character. The Nemotron 3 tokenizer emits 2.671 tokens per character for it. That is 94.4% of the raw byte count. Lao is 2.708 against 2.759, or 98.2%. The tokenizer is not compressing Khmer or Lao. It is spelling them out in bytes with a handful of merges on top.

Burmese is the control that makes the point. Burmese characters are also about 2.842 bytes each, and the tokenizer emits 0.548 tokens per character, 19.3% of the byte count. Burmese is in the vocabulary, just inefficiently. Khmer and Lao are not in it in any meaningful sense.

This is the difference between a problem continued pre-training can fix and one it cannot. Adding Burmese data teaches the model to use Burmese tokens it already has. Adding Khmer data teaches it to model a byte stream. The report says as much in Section 3.4, and its own numbers show the cost: a language at the byte floor burns roughly fifteen times the context window and fifteen times the inference cost per sentence, before any question of quality.

The scorecard, and the two languages that are not on it

SEA-HELM point estimates, gathered 15 September 2026 per the report's own note. The suite runs eight independent runs per model and 2,000 bootstrap resamples, but Table 5 prints point estimates only, so none of the deltas below can be called significant from the published numbers. Source: arXiv:2609.18310, Table 5.

Three things in that table are worth separating out.

The headline gain depends on which baseline you pick. The arXiv abstract says the 30B model improves the overall SEA score "from 46.06 to 51.57", a gain of 5.51. Section 6.1 of the same paper says "from 46.89 to 51.57", a gain of 4.68. Both are in Table 5: 46.06 is Nemotron 3.5 Lightning, 46.89 is Nemotron 3 Nano, and the model was initialised from Nano. AI Singapore's own launch blog uses the 46.89 figure and the smaller gain. The abstract uses the other one.

Thai does not improve over the model's own parent. Nemotron 3 Nano scores 62.90 on Thai; SEA-LION v4.8 30B scores 62.60. Indonesian moves 65.36 to 65.87. The report writes that "improvements are observed across all seven evaluated SEA languages", which is true against Nemotron 3.5 Lightning and not true against Nano on Thai. With point estimates only we cannot say the 0.30 gap is real, and we are not claiming a regression. We are saying the sentence is baseline-dependent and the table shows it.

Khmer and Lao are in the training mixture and not in the evaluation. Both carry parallel-data sampling weight 2. SEA-HELM covers Burmese, Filipino, Indonesian, Malay, Tamil, Thai and Vietnamese. The two languages the inherited tokenizer fails hardest on are the two the scorecard never touches, so the 63.44 headline carries no information about them either way.

Across those seven, fertility and score run in opposite directions: Spearman rho is -0.929 for the 120B model and -0.607 for the 30B. Drop Thai, the one language with high fertility and a good score, and the 30B correlation goes to -0.829. Seven points is seven points, and fertility is confounded with how much of each language was on the internet in the first place. But the ordering is hard to miss: Burmese has the worst fertility of the seven at 0.548 tokens per character and the worst score at 31.35, and Malay and Indonesian have the best fertility and the best scores.

What this means if you are shipping in Southeast Asia

Budget by tokens, not by language count. "Supports 7 SEA languages" is not a cost statement. The same paragraph costs 1.32x English in Indonesian and 3.22x in Burmese on this tokenizer. If your product has a Khmer or Lao surface and you route it through a Nemotron-derived model, you are paying roughly 14x English per sentence and eating the context window at the same rate.

Check the tokenizer before you commission the data. AI Singapore put 150B tokens and 114 GPU-hours of H200 time behind this adaptation and its own report concludes that tokenizer inefficiency still constrains how much the model learns from the data it was given. Commissioning more Khmer corpus against an unchanged byte-fallback vocabulary buys less than the token count suggests.

A benchmark that omits a language is not evidence about that language. SEA-HELM is a careful suite: natively written where possible, native speakers involved at every stage, prompts in the target language, scores normalised against a random baseline. That is exactly why its silence on Khmer and Lao matters. Nothing in the 63.44 speaks to them.

Judgement, marked as judgement: the interesting comparison for v4.9 is not another base model but a vocabulary transplant with re-initialised embeddings. The report names the constraint, and keeping the tokenizer is what made the continued-pretraining track cheap. Whether that trade was worth it for Khmer and Lao is untested, because nobody tested those languages.

Check it yourself

The fertility table takes about two minutes end to end and needs no GPU.

pip install tokenizers sentencepiece
curl -sL https://tinyurl.com/flores200dataset -o flores200.tar.gz
tar xzf flores200.tar.gz

curl -sL https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16/resolve/main/tokenizer.json -o nemotron3.json
curl -sL https://huggingface.co/aisingapore/sea-lion-7b/resolve/main/tokenizer.model        -o sealionv1.model

python3 - <<'EOF'
from tokenizers import Tokenizer
import sentencepiece as spm
hf = Tokenizer.from_file('nemotron3.json')
sp = spm.SentencePieceProcessor(model_file='sealionv1.model')
for code, name in [('eng_Latn','English'), ('ind_Latn','Indonesian'), ('tha_Thai','Thai'),
                   ('tam_Taml','Tamil'), ('mya_Mymr','Burmese'),
                   ('khm_Khmr','Khmer'), ('lao_Laoo','Lao')]:
    lines = open(f'flores200_dataset/devtest/{code}.devtest').read().splitlines()
    chars = sum(len(l) for l in lines)
    byts  = sum(len(l.encode('utf-8')) for l in lines)
    n_hf  = sum(len(hf.encode(l, add_special_tokens=False).ids) for l in lines)
    n_sp  = sum(len(sp.encode(l)) for l in lines)
    print(f'{name:<11} nemotron={n_hf:>7,}  sealion_v1={n_sp:>7,}  '
          f'ratio={n_hf/n_sp:5.2f}x  tok/char={n_hf/chars:.3f}  bytes/char={byts/chars:.3f}')
EOF
# Khmer   nemotron=409,170  sealion_v1= 52,765  ratio= 7.75x  tok/char=2.671  bytes/char=2.828
# Lao     nemotron=352,230  sealion_v1= 54,374  ratio= 6.48x  tok/char=2.708  bytes/char=2.759
# Tamil   nemotron= 60,618  sealion_v1= 36,953  ratio= 1.64x  tok/char=0.393  bytes/char=2.724

The SEA-HELM numbers are in Table 5 of arXiv:2609.18310 and on the live SEA-HELM leaderboard, which the report warns may have moved since 15 September 2026. Model weights: aisingapore/Nemotron-SEA-LION-v4.8-120B-A12B, MIT licence.

What would prove this wrong

A Khmer or Lao SEA-HELM column. If AI Singapore adds either language to SEA-HELM and SEA-LION v4.8 120B scores above 50 on it with the tokenizer unchanged, the byte-fallback ceiling is not binding on downstream quality and our framing is too strong. Our prediction: both land below Burmese's 31.35 at the 120B scale.

The Tamil five-times claim. If the report's authors publish the tokenizer and corpus behind Figure 3 and Tamil really is 5x on it, then our 1.64x means we compared against the wrong baseline tokenizer, not that the claim is wrong. Either the comparison corpus differs from FLORES devtest or the reference tokenizer is one we did not test.

The fertility correlation. Our rho of -0.929 uses seven languages and one model. If SEA-LION v4.9 keeps the same tokenizer, lifts Burmese above 55, and leaves Malay and Indonesian where they are, fertility stops ordering the scores and the relationship was training data all along.

Sources

1. AI Singapore and NVIDIA, SEA-LION-v4.8: A Technical Report, arXiv:2609.18310, announced 16 September 2026. 41 authors.
2. SEA-LION, Uplifting AI in Southeast Asia: Announcing Nemotron-SEA-LION-v4.8 in Collaboration with NVIDIA, 18 September 2026.
3. AI Singapore, Nemotron-SEA-LION-v4.8-120B-A12B model card, MIT licence, card last updated 18 September 2026.
4. NLLB Team, Meta AI, FLORES-200, the 1,012-sentence devtest split used for our token counts, CC BY-SA 4.0.
5. AI Singapore, SEA-HELM leaderboard, the live version of the Table 5 numbers.

Related BLOMEGA guides: The per-language token ceiling · Omnilingual MT by resource tier · Hours of audio against BLEU in low-resource speech translation

FAQ

What is Nemotron-SEA-LION-v4.8?

A family of four open-weight models released by AI Singapore with NVIDIA on 18 September 2026 under the MIT licence: 30B-A3B and 120B-A12B mixture-of-experts models built on NVIDIA Nemotron 3, each as a continued-pretrained base and a post-trained variant. The 30B saw 150B continued-pretraining tokens on 32 H200s over 114.23 hours; the 120B saw 33.5B tokens over 186.65 hours.

How much does the Nemotron 3 tokenizer cost per Southeast Asian language?

On the same 1,012 FLORES-200 devtest sentences, relative to English: Indonesian 1.32x, Malay 1.35x, Vietnamese 1.43x, Filipino 1.93x, Tamil 2.18x, Thai 2.37x, Burmese 3.22x, Lao 12.66x and Khmer 14.70x. That is our own measurement, 21 September 2026.

Why are Khmer and Lao so expensive to tokenize?

Because the Nemotron 3 vocabulary has almost no coverage of either script, so the tokenizer falls back to bytes. It emits 2.671 tokens per character for Khmer against 2.828 UTF-8 bytes per character, and 2.708 against 2.759 for Lao, which is 94% and 98% of the raw byte count. Burmese, by contrast, needs 0.548 tokens per character against 2.842 bytes, so Burmese is genuinely in the vocabulary.

Did SEA-LION v4.8 improve every Southeast Asian language?

Against Nemotron 3.5 Lightning, yes for all seven scored languages. Against Nemotron 3 Nano, the base the 30B model was actually initialised from, Thai goes 62.90 to 62.60 and Indonesian only 65.36 to 65.87. Table 5 prints point estimates without confidence intervals, so the Thai gap cannot be called significant either way.

Which SEA languages does SEA-HELM evaluate?

Burmese, Filipino, Indonesian, Malay, Tamil, Thai and Vietnamese. Malay and Burmese were added in this update. Khmer and Lao are in the continued-pretraining mixture at sampling weight 2 but are not evaluated, so the overall SEA score of 63.44 says nothing about them.