NVIDIA's 2.8 Trillion Token Indic Corpus Is Translated English, Not Scraped Indic Text

A 2.8 trillion token corpus built by translation, not collection

AI4Bharat has spent years assembling Indic-language text the conventional way: crawling the web, cleaning what comes back, and publishing corpora such as the earlier AI4Bharat-IndicNLP collection. Meta's No Language Left Behind project and the newer Omnilingual MT line of work follow a similar pattern for other low-resource languages, mining and aligning whatever text already exists in the target language. Nemotron-pretraining-Indic-v1 skips that search. NVIDIA's NeMo Data Designer documentation describes a pipeline that starts from 15 subsets of Nemotron-CC-v2, an English web corpus covering general web text, question-answering data, mathematics and code, and translates each document directly into a target Indian language rather than looking for an equivalent document that already exists in that language.

The resulting corpus breaks down into 2,601.0 billion native-script tokens spanning all 22 languages listed in the Eighth Schedule of India's constitution, plus a further 224.6 billion romanized, Latin-script tokens covering 12 of those 22: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu and Urdu. Native and romanized output together span 34 language-script configurations. The remaining 10 languages, Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali and Sindhi, get native-script text only, and NVIDIA's documentation says rewriting is switched off for them.

What the numbers add up to

Several independent write-ups of the October 16, 2026 announcement converge on the same figures, which is some reassurance that they trace back to the same NVIDIA source material, even though the dataset itself was not yet downloadable when they were published. The table below lists the recurring numbers with where each one comes from, so it is clear which details come from NVIDIA's own pipeline documentation and which are reported by outlets that covered the release.

The rewrite loop recovered 438.4 billion tokens, already folded into the 2.8 trillion total rather than added on top. That is roughly one in six tokens in the corpus: translations that first scored below NVIDIA's 75-out-of-100 quality bar, got rewritten, and passed on a later attempt. Without that loop, the usable portion of the corpus would have been meaningfully smaller than the headline number suggests.

How a document gets translated, scored, and sometimes rewritten

NVIDIA's dev notes lay out the pipeline step by step. An English document from one of the 15 Nemotron-CC-v2 subsets is translated into a target language by one of two models, google/gemma-4-31B-it for general translation or the specialist bodhan-ai/indic-translate. An LLM judge then scores the output from 0 to 100 against five dimensions, Faithfulness, Fluency, Terminology, Style and Format Preservation, tagging any problem Minor, Major or Critical. A score of 75 or above keeps the document in the corpus. Below that, and if a rewrite budget remains, the document goes back through translation and judging again.

Rewriting is disabled for the 10 lower-resource languages and for windowed translation, where a long document is split into chunks rather than translated whole; Santali, for instance, uses a 2,048-token translation window according to the documentation. Script choice changes the token bill independently of language choice: romanized output costs roughly 1.5 times the tokens of the same text in native script, and Meitei Mayek, the script used for Manipuri, costs roughly 9 times the token count of the English source. That cost difference is a plausible reason romanization is offered for only 12 of the 22 languages, and why the more expensive rewrite loop is reserved for languages where NVIDIA judged the extra compute worthwhile.

What this means for someone doing this work

Translated text is not a stand-in for what AI4Bharat has published before. Coverage of the release is explicit that the corpus is mostly machine-translated synthetic data and does not replace text originally written in Indian languages. Anyone evaluating a model trained on it should treat gains on this corpus as gains on translated English content reformatted into 22 languages, not gains on native Indic writing.

The headline benchmark gain is not an ablation of this corpus alone. NVIDIA's own continued-pretraining run for Nemotron 3.5 Super VL used 10 billion Indic tokens, a mixture of this corpus's 12-language output and other, naturally occurring Indic text, and reported gains of 4.98 to 9.82 points across five Indic benchmarks: MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy and Indic GSM8K. NVIDIA itself notes the comparison does not isolate what the synthetic corpus contributed versus the natural data it was mixed with.

The gain came with a tradeoff. Against the prior Nemotron 3 Super generation, five of six reported tasks improved, but IFEval 1.5, an instruction-following benchmark, fell by 2.04 points. A corpus built mostly from translated web, QA, math and code text does not automatically carry over instruction-following quality.

Token cost scales with script, not just language. A training budget planned purely by token count undercounts the real cost of a romanized or Meitei Mayek-heavy mix by roughly 1.5 times and 9 times respectively, which matters for anyone pricing out a comparable pipeline for a different language family.

Announced open and actually downloadable are different dates. NVIDIA and AI4Bharat announced the corpus as open on October 16, 2026, but as of an October 22, 2026 update to one outlet's coverage, the dataset was still listed as coming soon rather than live on Hugging Face under its announced name.

Check it yourself

A reader can check the dataset's live status and cross-read the documentation directly rather than taking any outlet's word for it.

# Confirm whether the dataset is live on Hugging Face yet
curl -s https://huggingface.co/api/datasets/nvidia/Nemotron-Pretraining-Indic-v1 | head -c 400

# Read NVIDIA's own pipeline documentation
open https://docs.nvidia.com/nemo/datadesigner/dev-notes/nemotron-indic-translation

# Cross-check the reported token breakdown against independent coverage
open https://blog.pebblous.ai/blog/ai4bharat-indic-corpus-translated-from-english/en/
open https://runtimewire.com/article/nvidia-ai4bharat-indic-pretraining-data-pipeline

What would prove this wrong

If, by March 1, 2027, NVIDIA and AI4Bharat have not published a controlled ablation that measures this synthetic corpus's contribution separately from the natural Indic text it was mixed with, and the dataset still has not appeared as a downloadable, versioned release on Hugging Face under the Nemotron-pretraining-Indic-v1 name, the October 2026 announcement should be read as a roadmap rather than a measured, released dataset. Either a published ablation isolating the corpus's own effect, or a working public download, would settle the question; the absence of both past that date confirms the gap this piece flags.

FAQ

How do you make a large language model multilingual when a language has almost no training text available?

One route is continued pretraining on machine-translated text: take a large, already-curated corpus in a high-resource language, translate it document by document into the target language, filter the output with an automated quality judge, and mix the result into a later training stage. NVIDIA and AI4Bharat demonstrated this with Nemotron-pretraining-Indic-v1, while noting the translated text does not replace text originally written in the target language.

Where do labs get training data for low-resource languages such as Santali or Manipuri?

For languages with too little native web text to scrape at scale, Nemotron-pretraining-Indic-v1 translates from an English source corpus instead of searching for native documents, using a chunked translation window for very low-resource languages like Santali and disabling the more expensive rewrite loop for languages including Manipuri, Sanskrit and Sindhi. That keeps compute costs down for the hardest cases at the price of a lighter quality check than the 12 higher-resource languages receive.

How do you evaluate whether a multilingual training run actually improved the model?

NVIDIA measured Nemotron 3.5 Super VL on five Indic benchmarks, MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy and Indic GSM8K, reporting gains of 4.98 to 9.82 points after adding 10 billion Indic tokens to continued pretraining. Because that stage mixed synthetic translated text with naturally occurring Indic text, the gain alone cannot isolate which ingredient did the work, which is exactly the kind of isolation a careful evaluation still owes a reader.

Sources

1. NVIDIA, NeMo Data Designer developer documentation, "Creating the Nemotron Indic Corpus" dev note, accessed October 2026: docs.nvidia.com/nemo/datadesigner/dev-notes/nemotron-indic-translation
2. Pebblous AI, "AI4Bharat Indic Corpus: Translated From English," October 18, 2026: blog.pebblous.ai/blog/ai4bharat-indic-corpus-translated-from-english/en
3. RuntimeWire, "NVIDIA and AI4Bharat announce a 2.8T-token corpus for 22 Indian languages," published October 15, 2026, updated October 22, 2026: runtimewire.com/article/nvidia-ai4bharat-indic-pretraining-data-pipeline
4. Unite.AI, coverage of the NVIDIA/AI4Bharat Nemotron-pretraining-Indic-v1 announcement, October 2026: unite.ai