NVIDIA Releases Nemotron-3-Embed-1B-BF16: 1.14B Parameter Multilingual Embedding Model

NVIDIA has released Nemotron-3-Embed-1B-BF16, a 1.14 billion parameter text embedding model optimized for retrieval and semantic similarity tasks across 34 languages.

Model Specifications

Architecture: The model is based on Ministral-3-3B-Instruct-2512 and uses a transformer architecture with bidirectional attention masking. It produces 2048-dimensional embedding vectors through average pooling of token-level representations.

Context Window: 32,768 tokens maximum sequence length. Longer inputs must be chunked or truncated.

Parameter Count: Approximately 1.14 billion parameters with a hidden size of 2048.

Language Support: The model was evaluated across 34 languages including English, Arabic, Chinese, Japanese, Korean, Hindi, Spanish, French, German, Russian, and 24 additional languages spanning European, Asian, Middle Eastern, and African languages.

Training Methodology

NVIDIA employed a two-stage structured pruning approach using ModelOpt's minitron Neural Architecture Search (NAS). The parent Ministral-3B model was first trained as an embedding model, then pruned to 2B parameters. The NAS process searched across hidden width, FFN size, attention heads, and depth using a 50,000-sample in-domain calibration corpus.

The 2B model was distilled from Nemotron-3-Embed-8B-BF16 using combined cosine distance loss and mean squared error loss on multilingual retrieval data. This pruning-and-distillation cycle was repeated to produce the final 1.14B model.

Features and Capabilities

Dynamic Embedding Sizes: The model supports dimension reduction by slicing the 2048-dimensional vector from the start (e.g., first 1024 or 512 dimensions). Sliced embeddings remain functional when re-normalized with L2 normalization.

Retrieval Optimization: Uses query: prefix for queries and passage: prefix for documents. Embeddings are L2-normalized, making dot product and cosine similarity equivalent.

Commercial License: Released under the OpenMDW License Agreement version 1.1, making it available for commercial deployment.

Technical Requirements

The BF16 checkpoint requires Transformers 5.2.0 or higher and a CUDA-enabled PyTorch installation. NVIDIA tested the model in the pytorch:26.06-py3 container with FlashAttention-2 support. The model can be used with Sentence Transformers 5.4.1 or higher for simplified inference.

What This Means

Nemotron-3-Embed-1B-BF16 represents NVIDIA's entry into the sub-2B embedding model space, competing with models like Alibaba's GTE and Jina AI's embedding models. The structured pruning approach demonstrates how larger models can be efficiently compressed while maintaining multilingual performance. At 1.14B parameters, it offers a practical size for RAG applications requiring broader language coverage than English-only alternatives, though NVIDIA has not disclosed specific benchmark scores or pricing for API access. The 32K context window matches or exceeds most competing embedding models in this parameter range.