MLOps

Amazon rolled out 13 new SageMaker AI inference capabilities in 2026 across managed endpoints and HyperPod Inference, spanning automated benchmarking, instance-pool fallback, OpenAI-compatible APIs, and container caching. AWS claims container caching cut endpoint startup latency by 51% and an inference-recommendation feature doubled GPT-OSS-20B throughput at equal latency.

AWS has launched model caching for Amazon SageMaker Inference on HyperPod, pre-loading model weights and container images onto cluster nodes to cut inference pod startup times from tens of minutes to seconds. The feature targets large models like DeepSeek-R1, which previously took 30+ minutes to become servable during scale-out events.

Amazon Web Services released AgentCore Optimization in preview, introducing an automated performance loop that generates configuration recommendations from production traces, validates them through batch evaluation and A/B testing, and enables continuous agent optimization. The system targets the quality drift problem where AI agents degrade as models evolve and user behavior shifts.

Amazon Web Services has released agent-guided workflows in SageMaker AI that use AI coding agents to automate model customization. The feature includes nine pre-built skills covering use case definition, data preparation, fine-tuning technique selection (SFT, DPO, RLVR), evaluation, and deployment to Amazon Bedrock or SageMaker endpoints.