NVIDIA Releases Audex-30B-A3B: Unified Audio-Text Model With 1M Token Context and Speech Generation

NVIDIA released Audex-30B-A3B, a unified audio-text language model that handles speech and audio tasks while maintaining text reasoning capabilities. The model is built on Nemotron-Cascade-2-30B-A3B, a 30B parameter mixture-of-experts (MoE) architecture with 3B activated parameters.

Core Specifications

• Architecture: 30B MoE with 3B active parameters
• Context window: 1M tokens
• Modalities: Text, speech, and general audio input/output
• Reasoning modes: Thinking and instruct (non-thinking) modes
• License: NVIDIA Oneway Noncommercial License
• Released: June 8, 2026 (according to model card)
• Base model: Nemotron-Cascade-2-30B-A3B

Technical Architecture

Audex-30B-A3B extends its text-only backbone with discrete audio token vocabularies for speech and audio outputs, plus an audio encoder for inputs. The model uses the ChatML template and encloses reasoning content within <think> and </think> tags. To activate non-thinking mode, the system prepends <think></think> to assistant responses.

Inference requires vLLM 0.20.0 or transformers >= 4.53.0. Audio functionality needs additional packages (mamba-ssm, causal-conv1d) and audio codecs not bundled in the base vLLM container.

Capabilities

The model handles six primary audio tasks:

1. Audio understanding: Question-answering about audio content (top_p=0.9, temperature=0.7)
2. Speech recognition: Transcription using greedy sampling
3. Speech translation: Cross-lingual speech conversion
4. Text-to-speech (TTS): Uses standalone Audex causal speech decoder or XCodec2
5. Text-to-audio (TTA): General audio generation using XCodec1 with optional 48kHz enhancement VAE
6. Speech-to-speech: Direct audio-to-audio conversion with text reasoning

NVIDIA claims the model preserves the "reasoning, alignment, knowledge, long-context, and agentic capabilities" of its text-only backbone with "marginal or no regression."

Inference Options

The model supports multiple inference paths:

• vLLM (recommended): Offline batch processing and OpenAI-compatible server with audio_url support
• Hugging Face/transformers: Requires transformers >= 4.53.0
• Text-only reasoning: Can convert checkpoint to pure text by removing audio vocabularies

Audio understanding and speech recognition use transformers 4.53.3 or Megatron-LM's native inference for benchmark results. Text-to-audio uses XCodec1 with enhancement VAE, while text-to-speech uses the original non-streaming XCodec2 decoder.

What This Means

NVIDIA's release adds another entry to the growing unified multimodal model category, following similar efforts from other labs. The 1M token context window matches recent long-context releases, while the MoE architecture with only 3B active parameters should enable efficient inference. The noncommercial license restricts commercial deployment, limiting this to research and development use cases. The model's ability to maintain text reasoning while adding audio capabilities addresses a common trade-off in multimodal models, though independent verification of the "no regression" claim on text tasks would be needed. The requirement for multiple inference setups depending on task type adds implementation complexity compared to unified inference pipelines.