NVIDIA and AI4Bharat announce Nemotron-pretraining-Indic-v1: 2.8T tokens across 22 Indian languages

NVIDIA and AI4Bharat, the Indian language research lab at IIT Madras, have announced Nemotron-pretraining-Indic-v1, a synthetic pretraining corpus of approximately 2.8 trillion tokens spanning the 22 languages listed in India's Constitution. The dataset is "releasing soon," according to the announcement, together with the open, GPU-accelerated pipeline used to build it. A release date, license terms and download location were not stated in the source material.

What is in the corpus

According to NVIDIA, the corpus holds 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, about 2.83 trillion combined. Counts use the Gemma-4 tokenizer. NVIDIA notes that token counts depend on the tokenizer and describe the retained corpus, not the token budget of any training run. Records also include translations and script variants of shared sources, so they are not counts of unique documents.

The data covers 34 language-script configurations in two groups:

• Native script and romanized (12 languages): Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu, Urdu.
• Native script only (10 languages): Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali, Sindhi.

All of it is translated from English. Romanized records are Indic-language text in Latin script, not English translations. For the twelve dual-script languages, romanization covers a subset of the accepted native-script translations. The two forms can express the same underlying content, which NVIDIA says matters when building a training mixture.

How it was built

The source material comes from 15 Nemotron-CC-v2 subsets covering web text, question-answer data, mathematics, code and other specialized content. The pipeline runs on NeMo Data Designer, NVIDIA's open-source synthetic data framework. Each record keeps a source identifier, so outputs can be traced to the original document.

For the twelve dual-script languages, the workflow has four stages:

1. Translation. Gemma-4-31B-it translates full documents, with prompts that separate prose from content to preserve, such as code, equations, URLs and formatting.
2. Quality assessment. The same model scores each output against its English source using a rubric inspired by Multidimensional Quality Metrics (MQM). The rubric covers meaning, fluency, terminology, style and formatting. Only outputs scoring at least 75 out of 100 are kept. NVIDIA stresses this is a rubric score, not an accuracy percentage.
3. Repair. Low-scoring outputs are rewritten using the assessment feedback, then re-scored. This recovered 438.4 billion tokens, already counted in the totals.
4. Romanization. A subset of accepted translations is converted to Latin script and checked against the native-script input.

NVIDIA reports that in an initial human comparison, converting from native script produced fewer hallucinated additions and less residual English than generating romanized text directly from English. The remaining ten languages use a separate workflow built on a document-level translator called Indic Translate, adapted for long-document quality and script-stability issues. The source text is cut off before describing it fully.

Downstream use

Continued pretraining of Nemotron 3.5 Super VL added 10 billion Indic tokens, drawn from a twelve-language subset of this corpus plus naturally authored Indic text. NVIDIA points to a separate technical report on that model and says it plans to use the corpus in future Nemotron models.

What this means

The corpus is large for the Indic languages, and the open pipeline may matter as much as the data. Teams can swap in their own source documents, prompts and quality gates. Provenance tracking and per-output validation against the source address a common complaint about synthetic data, though the quality claims rest on NVIDIA's own rubric and sampled human review, not independent evaluation.

The limits are clear. Every record is translated from English, so the corpus carries English-centric content and does not capture local knowledge or culture. NVIDIA itself says naturally authored text remains essential. Quality for the ten lower-resource languages, such as Bodo, Santali and Dogri, will be the real test once the data is public. Until the release lands with a license and benchmarks, treat the scale and quality figures as the developers' own claims.