AWS Overhauls Bedrock AgentCore Runtime, Cuts Cold Starts to Flat 2 Seconds Regardless of Image Size

Amazon has shipped a new version of the Amazon Bedrock AgentCore runtime, the managed compute layer for deploying AI agents, with two changes aimed directly at production workloads: flat cold-start latency and usage-based memory billing.

The numbers

According to AWS, the company tested an empty echo agent (one that returns input with no model or tool calls, isolating platform overhead) across five image sizes from 200 MB to 2 GB, sending 5,000 cold invocations per agent per runtime version. Results, measured client-side from an EC2 instance in us-west-2 calling agents in us-east-1 over the public internet:

• New runtime: P75 cold start latency of approximately 2 seconds, flat across all image sizes from 200 MB to 2 GB.
• Original runtime: latency scaled with image size, from roughly 5.4 seconds up to nearly 30 seconds.

AWS says the improvement comes from a snapshot-based startup model: the platform prepares an environment once, snapshots it, and restores that snapshot for each new instance rather than repeating a full boot-and-initialize sequence on every cold start.

On the memory side, AWS says the new runtime starts each session from a smaller baseline memory profile and pages in additional memory on demand as the workload requires it, rather than holding a full provisioned footprint for the session's duration. Based on AWS's stated analysis of allocation patterns across billions of sessions, the runtime now reclaims memory once it goes cold and is unlikely to be reused, instead of holding it until the session ends. AWS says billing now tracks these reclamation events over a session's lifetime rather than tracking the peak memory watermark, as the original runtime did.

Why AWS made these changes

AWS frames the update around a shift in how agents are used: from short chat-style exchanges to long-running, ambient agents that operate for hours with intermittent activity, sometimes triggered by other agents or events rather than direct user requests. AWS says this shift exposed two gaps in the original AgentCore runtime — memory billed at peak usage even when idle, and cold-start latency that grew with image size and worsened under bursty traffic, which AWS says forced customers to build workarounds like pre-warmed spare environments to avoid delays.

The underlying serverless model is unchanged: pay-per-use billing with scale-to-zero when agents are idle. AWS states the goal is to keep that consumption model intact while extending it to cover both quick interactive sessions and longer, more autonomous agent runs. Pricing for the new runtime was not disclosed in AWS's announcement.

What this means

Cold-start latency and memory over-provisioning are the two costs that make serverless agent infrastructure expensive to run at scale, and both have been cited by developers as reasons to build custom orchestration layers instead of relying on managed platforms. AWS's benchmark — a flat ~2-second P75 cold start independent of image size — is a meaningful claim if it holds under real production traffic with actual model and tool calls, rather than the empty echo agent used in AWS's own test. Independent verification against real-world agent workloads, and disclosure of updated pricing, will determine whether this closes the gap with self-managed alternatives or simply shifts complexity from customers to AWS's infrastructure.