Amazon Bedrock adds programmatic tool calling to reduce latency and token usage in multi-step workflows

Amazon Web Services has introduced programmatic tool calling (PTC) for Amazon Bedrock, enabling language models to generate Python code that orchestrates multiple tool invocations in a single inference cycle rather than requiring sequential round trips.

How it works

Traditional tool calling requires models to invoke tools one at a time, with each call requiring a full inference round trip. For a query like "Which engineering team members exceeded their Q3 travel budget?", a model using standard tool calling would need to make 20+ separate API calls to retrieve team members and expense records, passing thousands of intermediate data points through its context window.

With PTC, the model generates Python code once that handles all tool calls, data processing, and filtering within a sandboxed execution environment. Using asyncio.gather(), the code can execute tool calls in parallel. Only the final processed result returns to the model's context.

According to AWS, this approach reduces both latency and token consumption for workflows involving multiple tool calls, data aggregation, or numerical calculations.

Three implementation options

AWS provides three ways to implement PTC on Bedrock:

Self-hosted Docker sandbox on Amazon ECS: Offers maximum control with model-agnostic support for Claude, Qwen, MiniMax, Llama, Nova, and other Bedrock models. Developers can customize the sandbox environment, install domain-specific Python packages, and keep code execution within their AWS account. The architecture uses an orchestrator (ECS task or Lambda) that calls the InvokeModel API via Boto3 and manages Docker sandbox lifecycle.

Managed solution via Amazon Bedrock AgentCore Code Interpreter: A fully managed sandbox environment that handles execution without requiring custom infrastructure.

Anthropic SDK-compatible proxy: Designed for teams already using Anthropic's SDK who want PTC functionality while maintaining their existing developer workflow.

System prompt engineering

The self-hosted implementation relies on injecting tool definitions into the system prompt rather than using the standard tool_config parameter. The prompt instructs models to write Python code with specific rules: each execute_code call runs in a fresh stateless environment, tool calls must use await, and all operations must complete in a single code block.

The orchestrator intercepts tool calls through IPC over stdin/stderr, executes them externally, and injects results back into the sandbox.

What this means

PTC addresses a legitimate bottleneck in agentic workflows where sequential tool calling creates compounding latency. The model-agnostic self-hosted option is particularly significant—it extends a pattern originally introduced by specific providers to any model available on Bedrock. This matters for enterprises already committed to AWS infrastructure who want to avoid vendor lock-in at the model level.

The technique works best for workflows involving data aggregation, filtering operations, or scenarios where intermediate data shouldn't enter the model's context for privacy reasons. However, it requires models capable of generating correct async Python code and adds complexity around sandbox security and resource management.