AWS launches real-time voice agent framework combining Stream Vision Agents with Nova 2 Sonic

Amazon has released Stream's Vision Agents, an open-source Python framework for building production-ready voice AI agents that integrates with Amazon Nova 2 Sonic through Amazon Bedrock. The system targets end-to-end latency under 500 milliseconds for real-time voice interactions.

Technical architecture

The framework combines three components: Stream's Vision Agents handles orchestration and provides client SDKs for React, iOS, Android, Flutter, and React Native; Amazon Nova 2 Sonic provides speech-to-speech processing through Bedrock's real-time API; and Stream's Edge Network delivers the real-time transport layer with typically sub-500ms join times and under 30ms audio latency.

The system separates concerns by keeping Amazon Nova Sonic running in the customer's AWS account while Stream's infrastructure handles real-time media transport. Audio flows through Stream's globally distributed SFU (Selective Forwarding Unit) nodes, which terminate WebRTC connections and forward audio tracks to Vision Agent worker processes.

Audio pipeline details

Incoming speech is decoded to raw PCM by Vision Agent workers and streamed to Nova 2 Sonic via Bedrock's real-time API. Response audio frames are re-encoded, packetized as RTP, and delivered back through the SFU to client devices. The framework uses voice activity detection (VAD) in the worker to detect speech boundaries and barge-in events, while browser-based echo cancellation prevents the agent's output from retriggering the VAD loop.

Audio is transmitted as RTP over UDP rather than TCP to ensure predictable low latency and avoid head-of-line blocking. Stream's SFU handles bandwidth estimation, simulcast, and NAT traversal.

Capabilities and integration

Vision Agents provides a plugin-based architecture with 25+ integrations and supports function calling for API-driven actions. The framework includes automatic reconnection logic, session management, and graceful degradation for production deployment. Nova 2 Sonic handles the full speech-to-speech pipeline within a single model, eliminating the need for separate speech-to-text and text-to-speech services.

The system supports multilingual voice interactions and maintains full conversational context during barge-in scenarios. Developers can use Stream's global edge network or integrate their preferred real-time communication provider through a decorator-based interface.

Deployment model

The architecture maintains clear account boundaries: customer AWS accounts handle business logic, orchestration, and Bedrock integration, while Stream's AWS infrastructure manages the WebRTC/SFU media plane, TURN/STUN services, and signaling. Vision Agent runtime processes run as worker processes that terminate WebRTC as robot peers and bridge the customer's Bedrock integration.

What this means

This release addresses the infrastructure complexity of building production voice agents by providing an open-source framework that handles WebRTC management, audio streaming, and session lifecycle. The sub-500ms end-to-end latency target makes conversational AI interactions feel natural, while the separation between Stream's transport layer and customer-controlled Nova 2 Sonic deployments addresses data sovereignty concerns. The framework's 25+ integrations and multi-platform SDKs lower the barrier to deploying voice agents across web, mobile, and desktop applications.