AWS, Azure, Google agree: sessions are the new unit of cloud

Key Takeaways

• AWS, Microsoft, and Google now treat the session, not the request, as the fundamental compute unit for AI agents
• Each provider takes a different approach to session isolation: sandboxing, containerization, or new runtime primitives
• Engineering teams must evaluate these divergent isolation models before committing to an agent platform

AWS, Microsoft Azure, and Google Cloud have reached an unusual consensus: the session is now the core unit of cloud computing for AI agents. But that's where agreement ends. Each hyperscaler is betting on a different isolation strategy, and the choice you make now will shape your agent architecture for years.

The shift marks a departure from the request-response model that dominated cloud computing for two decades. AI agents don't fire one-off requests. They hold conversations, maintain state, use tools, and run for minutes or hours. That makes the session, not the API call, the natural boundary for resource allocation, security, and billing.

Why sessions matter more than requests for agents

A traditional web request hits a server, gets a response, and disappears. Statelessness was a feature. It made horizontal scaling simple. But an AI agent that books your travel, writes code, or manages a customer support thread can't be stateless. It needs memory of what it's done, what the user wants, and what tools it has called.

This isn't just a convenience problem. It's a security problem. When an agent has access to your email, your calendar, and your code repository, the blast radius of a compromised session is enormous. A malicious prompt injection in one turn could exfiltrate data or take actions on your behalf. The session boundary is where you contain that risk.

All three cloud providers recognize this. They've all moved toward session-aware runtimes in their agent platforms. The disagreement is over how to isolate sessions from each other and from the underlying infrastructure.

How AWS, Azure, and Google isolate agent sessions

AWS Bedrock Agents leans on sandboxing. Each agent session runs in a constrained execution environment that limits what code can do, even if an attacker manages to inject malicious instructions. The sandbox approach favors speed over flexibility. You get fast cold starts, but your agents operate within tighter guardrails.

Microsoft Azure AI Agent Service takes a containerization approach. Sessions run in lightweight containers that provide stronger isolation at the cost of more overhead. If you're already deep in the Kubernetes ecosystem, this maps to familiar primitives. But it means heavier resource requirements per session.

Google's Vertex AI agents push toward new runtime primitives designed specifically for agentic workloads. Google is betting that neither sandboxes nor traditional containers are the right abstraction. Instead, they're building purpose-built isolation mechanisms. The risk: these are newer and less battle-tested.

What this means for your agent architecture

The isolation model you choose affects more than security. It determines latency, cost, and portability. A sandbox-first approach like AWS keeps per-session costs low but limits what your agents can do. Containerized sessions on Azure give you more flexibility but burn more compute. Google's custom primitives promise the best of both but lock you into a less portable stack.

For teams building production agent systems, the practical question is: how sensitive is your workload? If your agents handle financial data or medical records, stronger isolation is worth the overhead. If you're building a coding assistant for internal use, speed and cost might trump maximum isolation.

The second question is multi-cloud. If you want agents that can run across providers, you need to abstract the session layer yourself. None of these isolation models are compatible. That's a significant engineering investment, but it's also insurance against vendor lock-in.

The stakes of getting isolation wrong

Agent security incidents are coming. When one happens at scale, the industry will split into teams that isolated sessions properly and teams that didn't. The hyperscalers know this. That's why they're racing to define the session boundary before the market does it for them.

The convergence on sessions as the unit of compute is real. The divergence on isolation is where competitive differentiation lives. For engineering leaders, the takeaway is simple: understand your provider's isolation model before you ship agents to production. The architecture you inherit today becomes the security posture you defend tomorrow.

Frequently Asked Questions

What is a session-aware agent runtime?

A runtime that treats the entire conversation or task sequence with an AI agent as a single compute unit, rather than handling each request independently. This enables state persistence, better security isolation, and accurate resource accounting.

Why does agent session isolation matter for security?

AI agents often have access to sensitive systems like email, code repositories, and databases. If a session is compromised via prompt injection, isolation limits the blast radius and prevents attackers from accessing other users' sessions or underlying infrastructure.

Which cloud provider has the best agent session isolation?

It depends on your priorities. AWS sandboxing is fastest but most constrained. Azure containerization offers stronger isolation for compliance-heavy workloads. Google's custom primitives may offer the best balance but are newer and less proven.

Can I run agents across multiple cloud providers?

Yes, but you'll need to build your own session abstraction layer. The isolation models from AWS, Azure, and Google are not compatible, so multi-cloud agent architectures require significant custom engineering.

Source: The New Stack / Janakiram MSV