NVIDIA Releases Audex-30B-A3B: Unified Audio-Text Model With 1M Token Context and Speech Generation
NVIDIA released Audex-30B-A3B, a unified audio-text model built on the Nemotron-Cascade-2-30B-A3B backbone. The model handles audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech while supporting up to 1M token context length.
NVIDIA Releases Audex-30B-A3B: Unified Audio-Text Model With 1M Token Context and Speech Generation
NVIDIA released Audex-30B-A3B, a unified audio-text language model that handles speech and audio tasks while maintaining text reasoning capabilities. The model is built on Nemotron-Cascade-2-30B-A3B, a 30B parameter mixture-of-experts (MoE) architecture with 3B activated parameters.
Core Specifications
- Architecture: 30B MoE with 3B active parameters
- Context window: 1M tokens
- Modalities: Text, speech, and general audio input/output
- Reasoning modes: Thinking and instruct (non-thinking) modes
- License: NVIDIA Oneway Noncommercial License
- Released: June 8, 2026 (according to model card)
- Base model: Nemotron-Cascade-2-30B-A3B
Technical Architecture
Audex-30B-A3B extends its text-only backbone with discrete audio token vocabularies for speech and audio outputs, plus an audio encoder for inputs. The model uses the ChatML template and encloses reasoning content within <think> and </think> tags. To activate non-thinking mode, the system prepends <think></think> to assistant responses.
Inference requires vLLM 0.20.0 or transformers >= 4.53.0. Audio functionality needs additional packages (mamba-ssm, causal-conv1d) and audio codecs not bundled in the base vLLM container.
Capabilities
The model handles six primary audio tasks:
- Audio understanding: Question-answering about audio content (top_p=0.9, temperature=0.7)
- Speech recognition: Transcription using greedy sampling
- Speech translation: Cross-lingual speech conversion
- Text-to-speech (TTS): Uses standalone Audex causal speech decoder or XCodec2
- Text-to-audio (TTA): General audio generation using XCodec1 with optional 48kHz enhancement VAE
- Speech-to-speech: Direct audio-to-audio conversion with text reasoning
NVIDIA claims the model preserves the "reasoning, alignment, knowledge, long-context, and agentic capabilities" of its text-only backbone with "marginal or no regression."
Inference Options
The model supports multiple inference paths:
- vLLM (recommended): Offline batch processing and OpenAI-compatible server with audio_url support
- Hugging Face/transformers: Requires transformers >= 4.53.0
- Text-only reasoning: Can convert checkpoint to pure text by removing audio vocabularies
Audio understanding and speech recognition use transformers 4.53.3 or Megatron-LM's native inference for benchmark results. Text-to-audio uses XCodec1 with enhancement VAE, while text-to-speech uses the original non-streaming XCodec2 decoder.
What This Means
NVIDIA's release adds another entry to the growing unified multimodal model category, following similar efforts from other labs. The 1M token context window matches recent long-context releases, while the MoE architecture with only 3B active parameters should enable efficient inference. The noncommercial license restricts commercial deployment, limiting this to research and development use cases. The model's ability to maintain text reasoning while adding audio capabilities addresses a common trade-off in multimodal models, though independent verification of the "no regression" claim on text tasks would be needed. The requirement for multiple inference setups depending on task type adds implementation complexity compared to unified inference pipelines.
Related Articles
Mistral AI Releases Shieldstral-1.0-3B, a 3B-Parameter Policy-Adaptive Safety Classifier
Mistral AI has released Shieldstral-1.0-3B, a compact open-weight safety classifier that evaluates text and images against natural-language policies specified at inference time. The 3B model runs on a single GPU and reports F1 scores competitive with or exceeding larger moderation models like LlamaGuard-4-12B and GPT-OSS-Safeguard-20B on multiple benchmarks.
NVIDIA Releases Nemotron VoiceChat 11B, an Open Full-Duplex Speech Model with Live Tool Calling
NVIDIA has released NemotronLabs VoiceChat 11B, an 11-billion-parameter end-to-end full-duplex speech model that unifies streaming speech understanding and generation in one architecture. The model claims to be the first open full-duplex system to support live tool calling during natural conversation, with ~450ms turn-taking latency.
Mistral Releases Shieldstral, a 3B Open-Weights Safety Classifier That Matches Models 7x Its Size
Mistral has released Shieldstral, a 3B open-weights safety classifier that reframes content moderation as a policy-adaptive question-answering task. The model claims to match or outperform guard models up to 7x its size on text safety and multimodal benchmarks, and runs on a single 16GB GPU.
LG AI Research Releases K-EXAONE 2.0, a 750B-Parameter Open-Weight MoE Model with 262K Context
LG AI Research has released K-EXAONE 2.0, a 750-billion-parameter mixture-of-experts language model with 37B active parameters, a 262,144-token context window, and support for 10 languages. The model is open-weighted under Apache 2.0 and claims competitive results against Qwen3.5, GLM-5.1, and DeepSeek-V4 Pro on reasoning, coding, and long-context benchmarks.
Comments
Loading...