Microsoft Releases VibeVoice-ASR-BitNet: 1.58GB Speech Recognition Model Runs Real-Time on CPU, No GPU Needed

Microsoft Research has released VibeVoice-ASR-BitNet, a compressed automatic speech recognition model designed to run in real time on CPUs without requiring a GPU. The model shrinks the original VibeVoice-ASR from 4.62 GB to 1.58 GB — a 2.9x compression — while delivering 1.6 to 2.3 times faster inference than Whisper.cpp, according to Microsoft's technical report published alongside the release.

The model is available now on Hugging Face under an MIT license, with accompanying C++ inference code (VibeASR.cpp) built on the ggml framework used by llama.cpp and Whisper.cpp.

Technical Details

VibeVoice-ASR-BitNet uses heterogeneous quantization across two components. The VAE tokenizer is quantized with an I8_S scheme, compressing from 1.31 GB to 0.65 GB (2.0x). The LM decoder combines I2_S and Q6_K quantization, dropping from 3.32 GB to 0.92 GB (3.6x). Total model size lands at 1.58 GB, with 0.3B parameters in F32 tensor format for the underlying safetensors checkpoint (10.7 GB uncompressed).

Microsoft reports real-time factor (RTF) benchmarks on an AMD EPYC 7V13 processor (AVX2+FMA) using 20-second audio clips. At 3 CPU threads, the model hits an RTF of 0.77 — under the 1.0 threshold required for real-time transcription — while running 1.86x faster than Whisper.cpp. Performance scales further with more threads: RTF drops to 0.42 at 8 threads, a 1.55x speedup over Whisper.cpp. Custom SIMD kernels for both ARM (NEON) and x86 (AVX2) platforms underpin these gains.

The model supports multiple languages including English, Chinese, French, Italian, Korean, Portuguese, and Vietnamese.

Accuracy Tradeoffs

Compression comes with a measurable accuracy cost. On the MLC-EN benchmark, word error rate (WER) rises from 7.82% for the full VibeVoice-ASR-7B model to 8.25% for the BitNet variant — still beating Parakeet (8.40%), Whisper (13.57%), SenseVoice (12.39%), and FunASR (11.36%) on this test.

The gap widens on harder benchmarks. On AISHELL4, a Mandarin meeting-transcription dataset, WER jumps from 19.83% (full model) to 27.45% (BitNet), trailing FunASR's 20.41%. On Libri-other, VibeVoice-ASR-BitNet scores 6.27% WER versus 3.13% for Parakeet and 3.60% for Whisper — indicating the compression hurts more on noisier, accented English speech than on controlled multilingual benchmarks.

Availability

The model ships as two ready-to-use GGUF files: a 0.65 GB VAE encoder and a 0.92 GB LM decoder. It is not yet deployed by any Hugging Face Inference Provider. Microsoft researchers can be reached at VibeVoice@microsoft.com for questions. The accompanying paper (arXiv 2607.21075) was published six days before this release.

What this means

This release targets a specific gap: speech recognition on edge devices — laptops, embedded systems, low-power servers — where GPU access is unavailable or impractical. The 2.9x size reduction and thread-efficient real-time performance make on-device transcription viable for applications like offline dictation, meeting transcription on commodity hardware, or voice interfaces on resource-constrained devices.

The accuracy tradeoff is real but bounded on clean audio (MLC-EN, Fleurs-en) while degrading more noticeably on noisy multi-speaker settings (AISHELL4, AliMeeting). Teams choosing this model should benchmark against their actual audio conditions rather than relying solely on the reported averages. The MIT license and open weights lower the barrier for integration into existing ggml-based toolchains already used by Whisper.cpp and llama.cpp deployments.