HyperPod

Amazon rolled out 13 new SageMaker AI inference capabilities in 2026 across managed endpoints and HyperPod Inference, spanning automated benchmarking, instance-pool fallback, OpenAI-compatible APIs, and container caching. AWS claims container caching cut endpoint startup latency by 51% and an inference-recommendation feature doubled GPT-OSS-20B throughput at equal latency.

Amazon SageMaker HyperPod has launched infrastructure updates for enterprise inference workloads. The platform now captures inference data at three points—endpoint, load balancer, and model pod—with configurable sampling and S3 storage. Teams can deploy models directly from Hugging Face Hub without pre-staging weights, with support for gated access across vLLM, TGI, and SGLang runtimes.

Amazon Web Services published a practical guide for fine-tuning Amazon Nova models using the Nova Forge SDK's data mixing capabilities. According to AWS, blending customer data with Amazon-curated datasets preserved near-baseline MMLU scores while delivering a 12-point F1 improvement on a Voice of Customer classification task spanning 1,420 leaf categories.