CNCF: agentic AI should run on Kubernetes, not new infra

Key Takeaways

• CNCF argues agentic AI systems should run on existing cloud native infrastructure rather than entirely new platforms
• Kubernetes provides the orchestration, resilience, and multi-cloud consistency that long-running AI agents require
• Observability tools like OpenTelemetry must evolve to trace AI reasoning paths, not just latency metrics

The Cloud Native Computing Foundation published a technical analysis arguing that agentic AI systems should run on Kubernetes and existing cloud native tooling rather than purpose-built infrastructure. The argument: AI agents are fundamentally distributed systems with reasoning capabilities, and the cloud native ecosystem already solves their core operational challenges.

This position matters because enterprises are rapidly moving beyond experimental chatbots toward autonomous agents that invoke tools, coordinate with other agents, and make operational decisions for hours or days at a time. The infrastructure question, until now, has been open.

Why treat AI agents as distributed systems?

The CNCF analysis centers on a real implementation: a multi-agent cybersecurity platform running on Kubernetes that detects and responds to runtime threats. Rather than replacing traditional security tooling, the AI agents extend an existing cloud native foundation. The platform combines Kubernetes for orchestration, OpenTelemetry for observability, Dapr for workflows, SPIFFE for workload identity, Falco for runtime security, and Kafka for event streaming.

The authors argue this pattern generalizes. Agentic systems face the same problems microservices faced a decade ago: securing identities, coordinating long-running workflows, managing state, ensuring observability, and recovering from failures. The cloud native ecosystem spent ten years solving these problems. Rewriting that infrastructure from scratch for AI seems wasteful when extension works.

Kubernetes specifically provides what multi-agent systems need: resilience through automatic rescheduling, consistent orchestration across hybrid and multi-cloud environments, and declarative configuration that fits GitOps workflows. An agent that runs for 72 hours coordinating with external services needs this kind of operational foundation.

Observability has to explain decisions, not just measure latency

The analysis identifies observability as a defining requirement that separates AI systems from traditional applications. Standard metrics like latency and throughput miss the point. AI agents make probabilistic decisions, invoke external tools dynamically, and adapt to changing context. When something goes wrong, operators need to understand why an agent reached a particular decision and how that decision propagated across the system.

OpenTelemetry is evolving to meet this need. The goal is tracing not just service interactions but reasoning paths, tool invocations, execution contexts, and multi-agent collaboration. This is harder than tracing a REST call chain, but it builds on the same distributed tracing primitives.

For engineering teams already using OpenTelemetry, the implication is clear: instrument your AI agents the same way you instrument your services, but extend the semantic conventions to capture reasoning steps and tool calls.

Workload identity becomes critical when agents access production systems

Security is the third major theme. As AI agents gain access to sensitive APIs, databases, and business processes, the question of identity becomes urgent. Who authorized this agent? What permissions does it have? Can we prove its execution history remains intact?

The analysis points to SPIFFE and SPIRE as examples of cloud native identity frameworks that provide cryptographically verifiable identities for autonomous workloads. This aligns with broader industry efforts. Dapr 1.18 recently added Verifiable Execution capabilities. The Linux Foundation launched the Akrites security initiative. The pattern is consistent: future AI systems must prove what decisions they made, who made them, and under what authority.

For teams deploying agents in production, this means treating agent identity with the same rigor applied to service accounts and API keys. The difference is that agents make decisions, so their identity credentials should reflect their decision-making scope, not just their technical access.

The limiting factor shifts from model capability to operational reliability

The broader message in the CNCF analysis deserves attention: successful agentic AI depends less on increasingly capable models than on disciplined systems engineering. As enterprises move from chatbots to autonomous workflows, the bottleneck shifts. A brilliant agent that crashes, loses state, or cannot be audited is useless in production.

This perspective runs counter to the common assumption that better models automatically yield better AI systems. Models matter, but so does everything around them. The teams that treat AI agents as first-class distributed systems, with proper orchestration, observability, and identity, will ship reliable autonomous workflows. The teams that bolt agents onto ad-hoc infrastructure will spend their time debugging failures they cannot trace.

What should engineering teams do now?

If you are building agentic systems, the CNCF analysis suggests a practical checklist. Run agents on Kubernetes rather than standalone processes. Instrument them with OpenTelemetry from day one, capturing reasoning traces. Assign cryptographic identities using SPIFFE or equivalent. Treat agent state as a distributed systems problem, with explicit persistence and recovery logic.

None of this requires new infrastructure. It requires applying existing cloud native practices to a new workload type. The learning curve is lower than building from scratch, and the operational maturity is higher.

Frequently Asked Questions

Why does CNCF say agentic AI should run on Kubernetes?

AI agents face the same operational challenges as distributed microservices: orchestration, observability, identity, resilience, and state management. Kubernetes and the cloud native ecosystem already solve these problems, so building on existing infrastructure is more efficient than creating new platforms.

How does observability differ for AI agents compared to traditional services?

Traditional observability tracks latency, throughput, and error rates. AI agent observability must also trace reasoning paths, tool invocations, and decision propagation across multi-agent systems. OpenTelemetry is being extended to capture these additional dimensions.

What role does workload identity play in agentic AI security?

AI agents that access production systems need cryptographically verifiable identities. Frameworks like SPIFFE and SPIRE provide this capability, allowing organizations to prove what decisions an agent made, under what authority, and whether execution histories remain intact.

Do I need new infrastructure to run AI agents in production?

According to CNCF, no. Existing cloud native tooling including Kubernetes, OpenTelemetry, Dapr, SPIFFE, and Kafka provides the foundation. The work is applying these tools to agent workloads, not replacing them.

Source: InfoQ