A Million AI Agents Packed into K8s: Sleep When Idle, Wake in Under 500ms
Agent Substrate treats agents as untrusted, mostly idle stateful workloads: it snapshots them offline when idle and restores them within 500ms when a request arrives. It runs on standard K8s, is not tied to an SDK, and aims to increase density on the same hardware by 10x.
Shubham Saboo called Agent Substrate the Kubernetes moment for AI agents, and Jaana Dogan reposted it. Let's set the marketing aside and look at what it actually solves.
Agent workloads look different from ordinary web services. Most of the time, they are waiting for a model to return, waiting for a tool to execute, or waiting for a human to confirm. But platforms often reserve resources as if they were running all the time. Agent Substrate assumes the opposite: agents are stateful, untrusted actors that are idle most of the time. It maps a large pool of actors onto a small pool of warm workers. Idle actors are snapshotted out of memory and restored when a request arrives. The official numbers are: restore time under 500ms, more than 500 activations per second, 10x higher density than standard containers on the same hardware, and 10x to 30x oversubscription in the demo.
The key to this scheduling model is state. It restores not just the process but also memory and filesystem state. In the project demo, about 250 stateful actors are multiplexed onto 8 physical pods without losing session or tool state. When a request hits a full worker pool, the router parks the request until a worker frees up instead of immediately throwing a 503. That detail says more about its focus on real long-running sessions than the claim of supporting a million agents.
The security model is also worth a look. It treats every actor as untrusted and mutually hostile. Ordinary Linux containers are out of scope; only gVisor or Cloud Hypervisor microVMs are supported. Each active actor gets a short-lived SPIFFE X.509 certificate by default, and the egress gateway can issue a per-actor JWT on demand. API keys and cloud tokens are not placed inside the sandbox; the egress gateway injects them into approved outbound requests. Networking is deny-by-default: only the ingress router can get in, and there is no lateral access between actors. Egress policy points are pluggable, so you can connect your own authorization, DLP, prompt injection scanning, or cloud IAM. For an agent platform running untrusted code, this is a tougher requirement than simply running more concurrent workloads.
It is not tied to an Agent SDK. It runs standard OCI images, so ADK, LangChain, Claude Code, Codex, Antigravity, and MCP servers can all go inside. Under the hood it is still Kubernetes Pods, autoscaling, and NetworkPolicies, with extra actor scheduling, snapshotting, identity, and network mediation layered on top. The project is open source under Apache 2.0 and has reached 1.0, supporting the latest stable K8s and the previous minor version. The official demo video is here.
To try it locally, you need Go, kubectl, and Docker, then use kind to spin up a cluster, install ate-system, and deploy the counter demo. After creating an actor, forward requests with kubectl ate proxy and curl it to see state survive suspend and restore. There are also installation guides for GKE.
One assessment: the hard part of agent infrastructure may not be how many inferences you can run at once, but how to keep a large number of sleeping sessions alive cheaply while preventing untrusted code from touching credentials. Agent Substrate has written that assumption into its scheduling and isolation layers. Whether it can actually sustain a million is another matter, but the direction is more correct than simply piling up containers.
发布时间: 2026-11-17 22:34