SageMaker

Amazon SageMaker AI now supports serverless fine-tuning for NVIDIA Nemotron 3 Nano (30B parameters, 3B active) and Nemotron 3 Super (120B parameters, 12B active). The integration includes supervised fine-tuning, reinforcement learning with verifiable rewards (RLVR), and reinforcement learning from AI feedback (RLAIF).

Amazon SageMaker HyperPod has launched infrastructure updates for enterprise inference workloads. The platform now captures inference data at three points—endpoint, load balancer, and model pod—with configurable sampling and S3 storage. Teams can deploy models directly from Hugging Face Hub without pre-staging weights, with support for gated access across vLLM, TGI, and SGLang runtimes.

Hugging Face and Amazon Web Services have integrated a one-click workflow that takes developers from model discovery on Hugging Face directly into AWS SageMaker Studio. The integration eliminates manual setup steps by automatically provisioning domains with pre-configured IAM permissions and displaying GPU quota availability inline.

NVIDIA Nemotron 3 Ultra is now available on Amazon SageMaker JumpStart with 550 billion total parameters and 55 billion active parameters. The model features a hybrid Transformer-Mamba Mixture-of-Experts architecture and supports context windows up to 1 million tokens, targeting agentic AI workloads.

Amazon SageMaker AI has launched bidirectional streaming support for real-time inference, enabling WebSocket-based voice applications through vLLM integration. The feature uses HTTP/2 on port 8443 to bridge client connections with vLLM's Realtime API, allowing audio to stream in while transcription streams back simultaneously over a single persistent connection.

Amazon Web Services has released agent-guided workflows in SageMaker AI that use AI coding agents to automate model customization. The feature includes nine pre-built skills covering use case definition, data preparation, fine-tuning technique selection (SFT, DPO, RLVR), evaluation, and deployment to Amazon Bedrock or SageMaker endpoints.

Amazon SageMaker AI now supports capacity-aware instance pools that automatically try alternative GPU instance types when primary choices lack capacity. The feature works across endpoint creation, autoscaling, and scale-in operations, eliminating the manual retry cycles that previously left endpoints stuck in failed states.

NVIDIA has launched Nemotron 3 Nano Omni on Amazon SageMaker JumpStart, a multimodal model with 30 billion total parameters (3 billion active) that processes video, audio, images, and text in a single inference pass. The model features a 131K token context window and uses a Mamba2 Transformer Hybrid MoE architecture combining three specialized encoders.

Amazon Web Services published a practical guide for fine-tuning Amazon Nova models using the Nova Forge SDK's data mixing capabilities. According to AWS, blending customer data with Amazon-curated datasets preserved near-baseline MMLU scores while delivering a 12-point F1 improvement on a Voice of Customer classification task spanning 1,420 leaf categories.

AWS has enabled fine-tuned Amazon Nova Micro models to run on Bedrock's on-demand inference for text-to-SQL generation. According to AWS testing, a sample workload of 22,000 queries per month costs $0.80 monthly using the serverless approach, compared to higher costs with persistent model hosting. The solution uses LoRA fine-tuning on the sql-create-context dataset containing over 78,000 SQL examples.