vLLM

IBM has released Granite-Switch-4.1-8B and Granite-Switch-4.1-30B, models that bundle a Decision adapter, a Guardian safety adapter and a speech-to-text model behind a single endpoint. IBM says the combined model uses roughly the same GPU VRAM as the base model alone. Benchmark claims are IBM's own, and context window and pricing were not disclosed.

AWS published a deployment guide for running Alibaba's Qwen3-TTS-12Hz-1.7B-Base voice cloning model as a real-time SageMaker inference endpoint. The model clones a speaker's voice from a short audio clip and generates speech in 10 languages, including cross-lingual cloning, without retraining.

Alibaba's Qwen team released Qwen3.8-2.4T-A95B on August 12, 2026, the open-weight version of Qwen3.8-Max and the first Qwen-Max-class model made publicly available. The 2.4 trillion-parameter mixture-of-experts model activates only 95 billion parameters per token and supports context windows up to 1 million tokens.

Amazon SageMaker AI has launched bidirectional streaming support for real-time inference, enabling WebSocket-based voice applications through vLLM integration. The feature uses HTTP/2 on port 8443 to bridge client connections with vLLM's Realtime API, allowing audio to stream in while transcription streams back simultaneously over a single persistent connection.

NVIDIA released Nemotron 3 Nano Omni, a 31B parameter (30B active, 3B per token) multimodal model supporting video, audio, image, and text inputs. The model features a 256K token context window, reasoning mode with chain-of-thought, and tool calling capabilities.