/aks-cluster-architecture

Agent Runtime Bundle

Deploying AI agent runtimes on AKS: kars (Agent Reference Stack for Kubernetes, an open-source Microsoft reference stack, not an officially supported product), Kata Containers pod sandboxing, KAITO / AI Runway model serving, AKS MCP server patterns, and security isolation for untrusted agent code.

Load this bundle for: deploying agent workloads that execute untrusted code, choosing between isolation primitives, designing agent-specific node pools, and wiring MCP servers to AKS-hosted agents.

kars. Agent Reference Stack for Kubernetes

kars (Azure/kars) is an open-source, Kubernetes-native Agent Reference Stack from Microsoft's Azure Cloud Native team for running AI agents on AKS. It is a reference implementation, not an officially supported Microsoft product (no SLA or support contract). It treats every agent as untrusted code with defense-in-depth isolation:

Microsoft Agent Framework (MAF) is one of kars's supported agent runtimes, and the router includes an MCP gateway that brokers calls to external MCP servers with OAuth and per-tool allowlists. Running kars node pools with AKS Pod Sandboxing (Kata Containers) optionally adds a per-pod guest kernel on top of the container-and-router model.

When to Use kars

Architecture

┌──────────────────────────────────────────────────┐
│  AKS Cluster (Automatic or Standard)               │
│                                                    │
│  ┌────────────────────────────────────────────┐    │
│  │  Sandbox pod (one agent per pod)           │    │
│  │  ┌───────────────┐   ┌───────────────────┐ │    │
│  │  │ Agent         │   │ Inference router  │ │    │
│  │  │ container     │─► │ (Rust)            │─┼──► governed egress
│  │  │ UID 1000      │   │ UID 1001          │ │    │  (L7 allow/blocklist)
│  │  │ no network    │   │ holds credentials │ │    │
│  │  └───────────────┘   └───────────────────┘ │    │
│  │  NetworkPolicy: default-deny               │    │
│  │  egress-guard init container (iptables)    │    │
│  └────────────────────────────────────────────┘    │
│                                                    │
│  AgentMesh (Signal Protocol) · A2A Gateway         │
│  MCP gateway (in router) ─► external MCP servers   │
└──────────────────────────────────────────────────┘

Deployment Checklist

Before deploying kars agents to production:

• Node pool: Dedicated agent node pool with runtimeClassName: kata-vm-isolation or AKS Pod Sandboxing enabled.
• VM SKU: Confidential VM (CVM) SKUs or v5+ family with nested virtualization support for Kata.
• Identity: Per-agent Workload Identity (azure.workload.identity/client-id annotation on ServiceAccount). Zero static credentials.
• NetworkPolicy: Default-deny per namespace; explicit allow rules for DNS, MCP endpoints, and approved egress.
• ResourceQuota: CPU, memory, and ephemeral storage limits per agent namespace.
• Pod Security: restricted Pod Security Admission with explicit securityContext.
• Observability: ContainerLogV2, Managed Prometheus, and agent-specific metrics.
• MCP transport: Streamable HTTP (preferred) or stdio via sidecar pattern. Do not expose MCP servers on public IPs.
• Lifecycle: Agent pods should be stateless; checkpoint state externally. Use terminationGracePeriodSeconds for clean shutdown.

Agent Isolation Primitives Comparison

kars's baseline isolation is the no-network agent container plus the inference router. To add a per-pod guest kernel on AKS, enable Pod Sandboxing (Kata Containers) with runtimeClassName: kata-vm-isolation, which is the AKS-native, officially supported (GA) way to get kernel isolation per agent. gVisor is not an AKS-supported runtime class, so treat that row as a general comparison, not an AKS option. Confidential Containers on AKS (kata-cc-isolation) is a separate, retiring preview, so do not build new work on it.

apiVersion: v1
kind: Pod
metadata:
  name: agent-pod
  namespace: agent-<id>
spec:
  runtimeClassName: kata-vm-isolation
  serviceAccountName: agent-sa
  containers:
  - name: agent
    image: <agent-image>
    resources:
      requests:
        cpu: "500m"
        memory: "512Mi"
      limits:
        cpu: "1"
        memory: "1Gi"
    securityContext:
      runAsNonRoot: true
      seccompProfile:
        type: RuntimeDefault
      allowPrivilegeEscalation: false
      capabilities:
        drop: ["ALL"]

MCP Server Sidecar Pattern

When the agent requires MCP tool access, run the MCP server as a sidecar in the same pod (or use a shared cluster service):

containers:
- name: agent
  image: <agent-image>
  env:
  - name: MCP_ENDPOINT
    value: "http://localhost:8080/mcp"
- name: mcp-sidecar
  image: <mcp-server-image>
  ports:
  - containerPort: 8080

For shared MCP servers (tool registry, knowledge base access), deploy as cluster services in a shared namespace with NetworkPolicy allowing agent namespaces.

AKS MCP Server Deployment

For MCP servers that are NOT part of kars but serve AKS-hosted agents:

• Transport: Use Streamable HTTP (MCP 2026-07-28). SSE is deprecated for new deployments.

• Auth: Workload Identity + Entra ID. Do not use API keys in Kubernetes Secrets.

• Placement: System node pool (cluster-wide tools) or dedicated MCP node pool.

• Scaling: HPA based on HTTP request rate. Cold-start aware if using scale-to-zero.

• Session state vs replicas: FastMCP-style streamable-HTTP servers default to stateful sessions, which break behind a multi-replica Service with no session affinity - requests land on replicas that do not own the session. Run stateless mode (FastMCP(..., stateless_http=True)) so any replica serves any request, or pin to one replica / configure session affinity.

• Networking: Internal cluster IP only (ClusterIP service). Do not expose via LoadBalancer or Ingress unless external agents need access, and then only via authenticated gateway.

• Probes (required for all MCP server containers). Every MCP server container must declare both readinessProbe and livenessProbe. MCP server pods that lack probes will receive traffic before they are ready (causing tool call failures at startup) and will not be restarted when they hang or deadlock (causing silent agent failures). Minimum probe configuration:

  readinessProbe:
    httpGet:
      path: /health
      port: 8080
    initialDelaySeconds: 5
    periodSeconds: 10
    failureThreshold: 3
  livenessProbe:
    httpGet:
      path: /health
      port: 8080
    initialDelaySeconds: 15
    periodSeconds: 20
    failureThreshold: 3

  If the MCP server does not expose an HTTP health endpoint, use a tcpSocket or exec probe - many streamable-HTTP MCP endpoints require protocol headers on every request, so a plain httpGet probe returns 4xx even when the server is healthy. Absence of probes is a blocking finding in the readiness review.

Security Boundaries

• Agent → MCP server: Authenticate via Workload Identity token. MCP servers validate the azp or sub claim against known agent identities.
• Agent → external APIs: Route through egress firewall. Default-deny, explicit FQDN allow rules.
• Agent → storage: Use Workload Identity + RBAC on Storage/Key Vault. No access keys.
• Agent → other agents: NetworkPolicy isolation by default. Explicit cross-namespace allow rules only when agent-to-agent communication is designed.
• One ServiceAccount per agent component (non-negotiable). Every agent container (main agent, MCP server sidecar, tool executor) must have its own Kubernetes ServiceAccount with its own federated identity credential. A shared ServiceAccount across multiple agent components collapses blast radius: compromise of any component grants the combined Azure RBAC scope of all components. Scope each ServiceAccount to the narrowest Azure roles required by that specific component only.

Observability

• Agent health: Liveness/readiness probes on the agent pod's health endpoint.
• Tool call metrics: Track tool invocation count, latency, and error rate via OpenTelemetry from the agent SDK.
• MCP metrics: Request rate, latency, error rate on the MCP endpoint.
• Isolation health: Monitor Kata VM boot times; pods stuck in ContainerCreating may indicate VM provisioning failures.

Related Skills

• microsoft-agent-framework — Building agents with MAF, hosted agents, and Foundry integration.
• mcp-server-design — MCP server architecture, auth, and production hardening.
• azure-container-apps — ACA Express for simpler agent hosting (trusted code, lower isolation requirements).