Enterprises that are beginning to deploy autonomous AI agents often hit a familiar wall: scaling thousands of short‑lived tool calls without over‑provisioning hardware, while still keeping operational overhead low. Google’s recent open‑source release, Agent Substrate, paired with the Agent eXecutor (AX) runtime, promises to address that gap by layering agent‑centric controls on top of existing Kubernetes clusters. In this post we’ll break down what the projects deliver, why they matter for large‑scale, bursty workloads, and how you can start experimenting with them today.

Why a New Layer on Kubernetes?

Kubernetes has become the de‑facto platform for container orchestration, but it was built around long‑running services, not the frantic, sub‑second interactions typical of AI agents that call external tools, fetch data, and adjust their own state on the fly. Developers face two main challenges:

1. Resource inefficiency – Traditional pod scheduling can waste CPU/GPU cycles when agents idle between calls.
2. Control‑plane chatter – Managing millions of rapid lifecycle events (create, pause, resume, destroy) stresses the default Kubernetes API, leading to latency spikes.

Agent Substrate introduces a minimal control plane that is purpose‑built for “the chatter of millions of sub‑second tool calls.” It abstracts the agent lifecycle into a lightweight custom resource, letting the underlying cluster handle the heavy lifting while keeping the API surface lean.

Core Components

Agent Substrate

• Custom Resource Definition (CRD): Defines an Agent resource that captures the agent’s code bundle, runtime config, and desired state.
• Scheduler Extension: Hooks into the kube‑scheduler to make placement decisions based on predicted burst patterns, improving hardware utilization by up to 97 % according to Google’s benchmark.
• Lifecycle Controller: Handles rapid start‑stop cycles, automatically scaling pods up or down in response to incoming tool calls.

Agent eXecutor (AX)

• Distributed Runtime: Executes the agent code across a fleet of lightweight workers. Each worker pulls the latest agent bundle from the substrate store and runs it in an isolated sandbox.
• Message Bus: Provides a low‑latency channel for agents to send/receive tool‑call results, bypassing the kube‑APIServer for the hot path.
• Observability Stack: Emits OpenTelemetry‑compatible traces and metrics, enabling engineers to monitor per‑call latency, error rates, and resource consumption.

Both projects are released under the permissive Apache‑2.0 license, encouraging contributions from the broader community.

Real‑World Use Case: Burst‑Heavy Customer Support

Imagine a global e‑commerce platform that uses AI agents to handle live chat, process refunds, and query inventory systems. During a flash‑sale, the number of simultaneous agent invocations can skyrocket, but the average conversation length remains under a second. Traditional Kubernetes would spin up a pod for each request, quickly saturating the cluster and inflating costs.

With Agent Substrate, the platform can:

1. Pre‑warm a pool of AX workers that stay idle in a low‑power state.
2. Dispatch incoming tool calls directly to the pool via the message bus, avoiding pod creation latency.
3. Scale out only when the burst exceeds the pre‑warmed capacity, leveraging the custom scheduler’s predictive placement.
4. Collect telemetry to fine‑tune the pre‑warm size, achieving near‑optimal hardware usage.

The result is a dramatic reduction in compute spend while maintaining sub‑100 ms response times.

Getting Started

Prerequisites

• A Kubernetes cluster (v1.27+ recommended)
• kubectl configured for the cluster
• Docker or a compatible OCI runtime

Installation Steps

Defining Your First Agent

Create a simple Agent manifest that runs a Python script echoing the incoming request:

Apply it with kubectl apply -f echo-agent.yaml. AX workers will automatically pick up the agent and start handling requests.

Sending a Tool Call

AX exposes an HTTP endpoint for inbound requests. A quick curl example:

The response will contain the echoed payload, demonstrating the end‑to‑end flow.

Community and Contribution

Both repositories include CONTRIBUTING guides, issue templates, and a Slack channel for real‑time discussions. Since the projects are in very early development, there’s a high demand for:

• Benchmarks on different hardware (GPU vs. CPU)
• Support for additional language runtimes (Node.js, Rust)
• Security hardening of the sandbox environment

If you’re interested in shaping the future of agent orchestration, consider filing a PR, joining a design sprint, or sponsoring the project through Google’s Open Source Programs Office.

Key Takeaways

• Agent Substrate adds a lightweight, agent‑aware control plane on top of Kubernetes, targeting bursty, sub‑second workloads.
• The paired AX runtime provides a distributed execution environment with a low‑latency message bus and built‑in observability.
• Early benchmarks claim up to 97 % better hardware efficiency compared to naïve pod‑per‑request approaches.
• The stack is open source under Apache‑2.0, ready for community extension and real‑world experimentation.

Conclusion

The AI agent landscape is moving from isolated demos to production‑grade services that must scale like any other cloud workload. Google’s Agent Substrate and AX aim to give developers the same reliability and elasticity that Kubernetes provides for microservices, but tuned for the hyper‑dynamic nature of agentic AI. By leveraging these tools, teams can avoid reinventing orchestration logic, cut operational costs, and focus on building smarter agents.

---

Source: With Agent Substrate, Google chases a Kubernetes moment for AI agents