Nvidia Releases Nemotron 3 Diarization, a Free 100M-Parameter Model That Tracks 8 Speakers in Real Time

What happened

Nvidia released Nemotron 3 Diarization, a freely available AI model that identifies which speaker is talking at any given moment in an audio stream. The model has approximately 100 million parameters, and its weights are available on Hugging Face.

The model can distinguish up to eight speakers in a single conversation and detect overlapping speech, when multiple people talk simultaneously. It works on both pre-recorded audio and live audio streams, according to Nvidia.

Benchmark performance

On the VoiceArena Diarization Benchmark v1, Nemotron 3 Diarization currently ranks first with a Diarization Error Rate (DER) of 14.7%, ahead of the next-best system at 19.3%. Nvidia claims this represents a 41% average error reduction over its predecessor, Streaming Sortformer, across eight test scenarios when using a 1.04-second audio buffer.

The benchmark applies strict scoring: overlapping speech counts toward errors, and even small misalignments at speaker transitions are penalized. Users can configure the audio buffer at four settings, ranging from 30.4 seconds down to 0.32 seconds. Shorter buffers reduce latency but generally increase error rates, and performance degrades further with more speakers, heavy background noise, or reverberation, according to Nvidia.

How it fits into a pipeline

Nemotron 3 Diarization does not transcribe speech itself. It only labels who is speaking and when, using anonymous tags such as "speaker_2" rather than identifying real names. When paired with a speech recognition system like Nvidia's Parakeet, it can produce full transcripts with speaker labels attached to each line of text.

Confirmed facts vs. claims

The parameter count (~100 million), open weight availability, support for up to eight speakers, and the four buffer-length options are confirmed specifications. The 14.7% DER score and the 41% improvement over Streaming Sortformer come from the VoiceArena Diarization Benchmark v1 and Nvidia's own reporting; independent verification across broader datasets has not been published. Pricing is not applicable since the model is free to download and self-host; Nvidia has not disclosed specific hosting or API costs for enterprise deployment.

What this means

Diarization has typically been a bottleneck in transcription pipelines, especially in multi-speaker settings like meetings, call centers, and podcasts, where existing open models often lag well behind proprietary offerings. A 100M-parameter model that tops a public benchmark while remaining small enough to run in real time suggests diarization is becoming cheap and commoditized rather than a premium feature bundled into closed transcription APIs.

For teams building voice AI products, the practical value is in the combination: pairing a lightweight diarization model like Nemotron 3 with an ASR system such as Parakeet gives a full speaker-labeled transcript without licensing a closed end-to-end stack. The tradeoffs are real, though — accuracy drops with more speakers, noisy environments, and shorter buffers, meaning production deployments will need to tune buffer length against latency requirements. The benchmark's strict scoring of overlapping speech also means real-world error rates in noisy, multi-person settings will likely exceed the reported 14.7%.