Mistral Releases Voxtral TTS: 4B Parameter Text-to-Speech Model at $0.016 per 1k Characters

Mistral AI has released Voxtral TTS, a 4B parameter text-to-speech model supporting 9 languages with voice cloning capabilities from as little as 3 seconds of audio.

Technical Specifications

The model consists of three components:

• 3.4B parameter transformer decoder backbone (built on Ministral 3B)
• 390M parameter flow-matching acoustic transformer
• 300M parameter neural audio codec

Voxtral TTS achieves 70ms model latency for typical inputs (10-second voice sample, 500 characters) with a real-time factor of approximately 9.7x. The model natively generates up to 2 minutes of audio, with the API handling longer generations through smart interleaving.

Language Support and Capabilities

Voxtral TTS supports 9 languages: English, French, German, Spanish, Dutch, Portuguese, Italian, Hindi, and Arabic. The model can adapt to custom voices using reference samples as short as 3 seconds, capturing voice characteristics including accent, inflections, intonations, and disfluencies.

The model demonstrates zero-shot cross-lingual voice adaptation despite not being explicitly trained for it. For example, it can generate English speech using a French voice prompt, producing natural-sounding French-accented English.

Architecture Details

Voxtral TTS uses a transformer-based, autoregressive, flow-matching architecture. The in-house codec processes audio causally using semantic VQ (8,192 vocabulary) and acoustic FSQ (36 dimensions, 21 levels) latent representations, producing them at 12.5Hz frame rate. The flow-matching transformer runs 16 function evaluations per audio frame to produce acoustic latents.

Performance Benchmarks

According to Mistral AI, human evaluations by native speakers show Voxtral TTS achieves superior naturalness compared to ElevenLabs Flash v2.5 while maintaining similar time-to-first-audio. The company claims performance parity with ElevenLabs v3 in quality, with support for emotion-steering.

In zero-shot custom voice evaluations across 9 languages, Mistral conducted side-by-side preference tests with 3 annotators per language pair, measuring naturalness, accent adherence, and acoustic similarity. The company claims Voxtral TTS outperformed ElevenLabs v2.5 Flash in this multilingual custom voice setting.

Pricing and Availability

Voxtral TTS is available via API at $0.016 per 1,000 characters. The model can be tested in Mistral Studio and Le Chat. An open-weight version with reference voices is available on Hugging Face under CC BY-NC 4.0 license.

What This Means

Mistral's entry into text-to-speech with a compact 4B parameter model and competitive pricing positions it against established players like ElevenLabs. The 70ms latency and 3-second voice cloning capability make it viable for real-time voice agent applications. The open-weight release under a non-commercial license follows Mistral's hybrid approach of offering both commercial API access and community model weights. Cross-lingual voice adaptation without explicit training is a notable capability that could simplify speech-to-speech translation pipelines.