/aks-cluster-architecture

Quality Review - aks-cluster-architecture

Review date: 2026-08-26 Stopping condition: validator fully green (0 errors, 0 warnings); 1 review round (3 concurrent reviewer angles, all Critical/High closed by fix packs); capability-delta ledger fully dispositioned; convergence contract met without a second round.

This document records the current review state. Earlier release history remains in CHANGELOG and below.

3.20.0 → 3.21.0 Review (Full-mode improve-skill 4.11.0 audit)

Summary

• Context: second consecutive daily full-mode audit. The 2026-08-25 baseline (<30 days old) carried forward as confirmed-correct; evidence work concentrated on post-2026-08-25 drift via change-feed probes (AKS releases API, MRC updates feed, azure-aks-docs commit feed, live spec listing).
• Driver: two GA capability deltas absent from every prior sweep (control-plane metrics GA Aug 2026; ACNS eBPF Host Routing GA June 2026) plus a Critical stale-flag CLI example in the fleet guide that survived three prior audits because no probe had diffed the example against the az CLI reference.
• Rounds: 1 (3 concurrent background reviewers: content-accuracy/skeptical, cross-reference/trusting, builder's-eye/skeptical). No material findings remained open after fix packs A–E; second-round trigger not met.
• Final status: STRONG.

Findings closed (8 + 1 rejected candidate)

Critical: F-102 fleet az fleet autoupgradeprofile create example used non-existent flags --upgrade-type/--update-strategy-name (corrected to --channel NodeImage + --update-strategy-id, verified against the live az CLI reference). High: F-101 dead citation updates-auto-upgrade-profiles (URL never existed under either docset root → replaced with verified update-automation); F-105 control-plane metrics GA absent from observability surfaces; F-106 eBPF Host Routing GA + Static-Egress-Gateway mutual exclusion absent from CNI/outbound surfaces. Medium: F-103 SecurityPatch channel nuance (live-patch-in-place, Windows exclusion — original wording verified substantively accurate vs Learn); F-104 Artifact Streaming Premium-ACR prerequisite + AKSNodeClass NAP path omitted; F-107 six spec properties missing from hidden-API-surface table (aiToolchainOperatorProfile, bootstrapProfile, httpProxyConfig, podIdentityProfile, diskEncryptionSetID, powerState — sourced from live 2026-06-01 spec diff); F-108 NAP decision-matrix dead-end concretized. Rejected: R2 anchor candidate (self-refuted — resolves correctly).

External Validation

• Official source: Azure/AKS releases API (newest still 2026-08-07 ✓); azure-rest-api-specs stable/2026-06-01 + preview/2026-06-02-preview re-listed live (identical to 2026-08-25 baseline); managedClusters.json property set diffed; az fleet autoupgradeprofile CLI reference fetched live.
• Official docs: control-plane-metrics-monitor (updated_at 2026-08-18), container-network-performance-ebpf-host-routing + how-to-enable-ebpf-host-routing, kubernetes-fleet/update-automation (updated_at 2026-08-21), auto-upgrade-node-os-image — all fetched live 2026-08-26.
• Structure-drift probe: 31/31 unique body-cited learn.microsoft.com URLs resolved HTTP 200 post-fix (one rot found and fixed).
• Change-feed: azure-aks-docs commits through 2026-08-25 triaged (CNL label fix below cut-line; cilium min-version bump no skill impact; NAP artifact-streaming PR adopted; private-DNS spoke-link fix no contradicting claim).

Capability Deltas (dispositioned)

Adopted: D-001 control-plane metrics GA (ops bundle + §8 route + Aug-26 stamp); D-002/D-003 eBPF Host Routing GA + verified incompatibility set incl. SEG mutual exclusion both directions (cluster-foundations + glossary + stamp); D-004 Artifact Streaming × NAP via AKSNodeClass (workload-platform + stamp); L-101..L-106 spec properties (hidden-surface table). Deferred with rationale: D-005 ACNS WireGuard/Cilium mTLS (transport detail below architecture cut-line); L-107 schedulerProfile (niche tuning). Out of scope: CNL label-selector doc fix; schedulerProfile. Recorded-not-adopted corrections: zero.

Token Efficiency

86,266 words (~112k tokens est.) post-run vs 84,980 baseline (+1.5%, net-additive correctness/capability work; no compression pass while hard gates were closing). M5 structural consolidation remains the known deferred opportunity.

Honest Unknowns

• Fleet "Security patch" channel: Preview per portal UX tab; exact GA date unconfirmed ([VERIFY] retained, WhereToCheck named: kubernetes-fleet/update-automation).
• Control-plane metrics Private Link support roadmap unconfirmed (currently unsupported).
• KubeBuddy check-ID mapping not re-enumerated this run (carried).
• No live Azure subscription: CLI shapes verified against command references and Learn text, not executed against a cluster.

Non-blocking recommendations (deferred)

• M1-residual: references/full-reference.md snapshot dated 2026-05-14 (explicit disclaimer stands; optional refresh debt).
• M5 structural consolidation (cluster-foundations overlap), agent-runtime scaffolding items — carried unchanged.
• Process note for future runs: delegated link-sweep agents produced unreliable verdicts once this run (all pages mislabeled REDIRECT-HOME); direct GET verification is the reliable method — prefer it or verify agent results before acting on them.

Historical reviews (3.19.0 and earlier)

3.19.0 → 3.20.0 Review (Full-mode improve-skill 4.8.0 audit)

Summary

• Driver: requested full-mode audit of SKILL.md + bundles. Phase 1 currency check found AKS Release 2026-08-07 (published 2026-08-11) missing entirely - the 3.19.0-era "August 11 research pass" still asserted no release newer than 2026-07-17.
• Rounds: 2 (fix-pack application + verification; reviewer perspectives executed manually after subagent spawn failures - recorded per execution-contract fallback).
• Angles: content accuracy + capability recency (skeptical); cross-reference quality + structural cohesion; builder's-eye / LLM-consumability.
• Final status: STRONG (see closure below).

Findings closed (14)

Critical: F-001 false "no newer release" claim (SKILL.md). High: F-002 stale absolute no-rollback negative → rewritten pool-scoped GA with constraints (SKILL.md + operations-resilience); F-003 automatic zone placement Preview→global (SKILL.md + cluster-foundations); F-004 link rot ×3 body files (ACNS CLI doc, ray-overview family, fleet cross-cluster networking; found via 65-URL structure-drift probe). Medium: F-005 Fleet Security patch channel added (#3738); F-006 hyperscale GiB units + 50% etcd review threshold aligned to ms.date 2026-08-19 doc; F-007 skew-policy collision softened (checklist + non-negotiable now name the sanctioned N-3 staged-upgrade exception); F-008 Kueue capacity-on-admission ProvisioningRequest pattern adopted; F-010 NDP coverage updated for K8s 1.37+ immediate-reimage triggers; F-012 bundle version stamps realigned to 3.20.0. Low: F-009/F-011/F-013/F-014 ecosystem stamps, VMSS surge cap check, MIG slice-width validation, ACL Entra SSH rejection.

External Validation

• Official source: Azure/AKS releases page (2026-08-25) incl. Release 2026-08-07 full notes; KubeDeckio/KubeBuddy releases API (v0.0.36, 2026-08-13); Azure/kaito releases API (v0.12.0, 2026-08-22); Azure/kars push activity.
• Official docs: supported-kubernetes-versions (ms.date 2026-08-05), hyperscale-configuration-aks (ms.date 2026-08-19), roll-back-node-pool-version, configure-kueue-with-cluster-autoscaler, use-advanced-container-networking-services, ray-overview, concepts-cross-cluster-networking (all resolved live 2026-08-25).
• Spec surface: azure-rest-api-specs TypeScript spec folder for Microsoft.ContainerService/aks - newest stable 2026-06-01 / preview 2026-06-02-preview; skill pins audited, one aged lesson anchor noted (2024-09-02-preview, self-aware lesson text).
• Community/release communications: azure-aks-docs commit feed through 2026-08-24; PixelRobots/GBB sweeps from 3.16.0–3.18.0 remain fresh (<30 days).

Capability Deltas (dispositioned)

Adopted: node-pool rollback GA (ops bundle + SKILL.md), zone placement global, control-plane-only LTS path, 1.37 reimage-trigger set, Fleet Security patch channel, Kueue capacity-on-admission, AGC add-on auto-alignment, VMSS surge cap, MIG slice-width validation, EiT×WI stamp, plus minor stamps (D-001..D-018 in audit register). Covered: PIS preview (already present). Out of scope: Flatcar retirement (never selectable for new designs). Recorded-not-adopted corrections: zero.

Token Efficiency

69,843 words (~90.8k tokens est., words × 1.3) across router + bundles + references; net-additive correctness work concentrated in expected surfaces. No compression pass warranted; M5 structural consolidation remains the known deferred opportunity (carried since 3.10.2).

Honest Unknowns

• No live Azure subscription in the audit environment: az aks command shapes (rollback/get-rollback-versions flags) verified against Learn article text, not an installed az CLI against a real cluster.
• Fleet "Security patch" channel CLI surface is docs-commit-level evidence ([VERIFY] retained in fleet bundle).
• KubeBuddy check-ID mapping (AKSBP*/AKSSEC*) not re-enumerated this run (v0.0.36 stamp only).

Non-blocking recommendations (deferred)

• M1-residual: references/full-reference.md snapshot dated 2026-05-14 - mitigated by its explicit "Snapshot, not source of truth" disclaimer; refresh remains optional debt.
• M5 structural consolidation (cluster-foundations overlap), L2-L4 agent-runtime scaffolding - carried unchanged.

Historical reviews (3.10.2 and earlier)

3.10.1 → 3.10.2 Review (Currency Pass)

Summary

• Driver: skill-improvement-metaprompt run (Phase 0-4). External-evidence pass against AKS releases 2026-07-17, 2026-06-19, 2026-05-29 and current Microsoft Learn.
• Rounds completed: 1 — evidence-driven; direct main-agent fixes (no fix-agent fan-out needed).
• Angles covered: Content accuracy (currency claims), Capability recency (AKS 2026-07-17 release items).
• Pre-fix score: 9.4/10 (unchanged from 3.10.0 — no correctness corrections needed).
• Post-fix score: 9.4/10 — currency pass only; skill remains at same quality bar with refreshed capability surface.

Findings

• Critical/High: None. No internal contradictions found; no must-fix correctness items.
• Capability deltas added (11 items): Artifact Streaming GA, Windows Server 2025 GA, In-place node pool resize (Preview), Automatic PDB Management (Preview), Azure Container Linux (ACL) GA, NVIDIA RTX PRO 6000 Blackwell GPU support, FIPS on Ubuntu 22.04 GA, enableCustomCATrust retirement (Sep 14, 2026), K8s 1.30 deprecated, K8s 1.33 LTS-only.
• Already covered: Node Disruption Policy (confirmed in AKS 2026-07-17, already in skill from July 18 platform updates); K8s 1.36 GA/LTS (already correct).

External Validation

• Official source: AKS Release 2026-07-17, 2026-06-19, 2026-05-29 (GitHub Azure/AKS).
• Official docs: Microsoft Learn — artifact-streaming, resize-node-pool, automatic-pod-disruption-budget-management, upgrade-windows-os, azure-container-linux-overview, supported-kubernetes-versions.
• Community / release notes: Azure/AKS #3928 (Artifact Streaming GA), #5826 (enableCustomCATrust retirement), #4447 (Windows Server 2025 GA).

Token Efficiency

• Net-neutral: ~+70 lines added to SKILL.md (new currency section). Partial-loading structure preserved — SKILL.md router + lazy-loaded bundles unaffected.
• No compression needed; additive-only pass.

Honest Unknowns

• No live Azure subscription available; az aks CLI examples remain verification gates.
• Artifact Streaming Microsoft Learn page not yet updated to reflect GA status (GH issue confirmed GA).
• enableCustomCATrust retirement date (Sep 14, 2026) is from GitHub issue; verify against official Azure Updates.

3.9.4 → 3.10.0 Review (Closed)

Summary

• Driver: skill-improvement-metaprompt run. Three parallel external-evidence agents — (A) currency/supersession/latent-capability vs authoritative sources, (B) local content-accuracy + structural cohesion, (C) veteran/beyond-quickstart/field-reality.
• Rounds completed: 1 (evidence-driven; direct main-agent fixes, no fix-agent fan-out needed).
• Angles covered: content accuracy, structural cohesion, cross-reference quality (Correctness); capability recency + supersession + latent capability (Production/currency); veteran/beyond-quickstart + field reality (Production Readiness); upgrade & breaking-change surface (Resilience/Ops).
• Pre-fix structural score: 6/10 — a mature, densely-linked skill carrying genuine self-contradictions on hard-to-reverse decisions. Post-fix: validator green, all Critical/High closed. Final score: 9.4/10 (see rationale).

Must-fix items closed (Critical / High)

• C1 (Critical) — SKILL.md claimed managedNATGateway is zone-redundant, contradicting cluster-foundations + full-reference and Azure docs (Standard NAT Gateway is zonal; only StandardV2/managedNATGatewayV2 is zone-redundant). Corrected; preserved the real userAssignedNATGateway single-template-Bicep preflight lesson; cross-linked the authoritative outbound section.
• H1 (High) — Arc Fleet Manager GA-vs-Preview split: fleet-management + full-reference said Preview (Nov 2025) while SKILL.md/cluster-foundations said GA. Verified GA at Build 2026 (member scale 200→1,000); reconciled to GA with capability-parity caution retained.
• H2 (High) — cluster-foundations node-OS channel used invalid NodeSecurityPatch + flag --node-image-channel; corrected to SecurityPatch + --node-os-upgrade-channel; added None row.
• H3 (High) — "Ubuntu is a legacy choice" (SKILL.md) vs "Azure Linux 3 or Ubuntu 24.04" (bundles). Reframed: both GA/first-class; dropped dated "(mariner)" gloss.

Should-fix items closed (Medium / Low)

• M2/M3 — cluster-foundations "built-in NPM"/"NPM vs Calico" (Cilium ≠ NPM; NPM deprecated) → Cilium Network Policy; removed NGINX-as-default ingress (contradicted the avoid-NGINX EOL non-negotiable).
• M4 — softened hard-coded SLA numbers in the first pricing table to [VERIFY] + cross-link the authoritative Pricing Tier and SLA section; noted LTS→Premium auto-routing cost.
• L1 — Confidential Containers retirement tense (production-workload-controls) future→past.

Capability Delta Register (dispositioned)

Supersession Register (closed)

• S-1 Azure Linux "(mariner)" gloss → Replaced (lineage-only label removed). S-3 "Ubuntu legacy" → Replaced (reframed first-class). S-4 containerd 1.x assumptions → Replaced (2.x section). S-5 target-K8s baseline → Confirmed current (skill already targets 1.36; baseline table is version-agnostic). Zero open S-001 items.

Disagreement Register

• Empty. The two web agents converged independently on containerd 2.0 as the top finding; the structural agent's contradiction findings were verified against primary sources before applying (no Type-1/Type-2 conflicts remained open).

External validation results

• Official docs: Microsoft Learn upgrade-os-version (2026-07-13), supported-kubernetes-versions (2026-07-15), whats-new-azure-linux-4. NAT Gateway zoning cross-checked against in-skill previously-verified content (reliability doc returned 404 at review time).
• Live source / release notes: Azure/AKS releases 2026-06-19; issues #4700 (containerd 2.0), #5772 (AZNFS), #5017 (Node Disruption Policy); containerd upstream #12808/#12612/#12636.
• Community: node_exporter #3331, GKE containerd-2 migration guide, Broadcom KB 401577 (deprecated-API gate), Ubuntu AppArmor userns spec.

Token Efficiency

• Total markdown words: 64,470 → ~65,229 (+759, ~+1k tokens est., wc -w × 1.3). Additive, concentrated in cluster-foundations (containerd/OS) and operations-resilience (upgrade gate). Partial-loading structure preserved — a typical task still loads the router + one or two bundles.
• No compression needed; corrections were net-neutral edits.
• Capability signals preserved: containerd 2.x runtime guidance, node-OS SKU decision, NAT/egress zone-redundancy, Fleet Manager GA, CVE/upgrade runbooks.

Non-blocking recommendations (deferred)

• M5 — cluster-foundations carries workload/ops overlap (Spot pools, PSA migration, node-OS channel, cost estimation) duplicating other bundles; a consolidation pass would reduce a 3+ file redundancy cluster. Structural, not correctness.
• M1 residual — references/full-reference.md is dated 2026-05-14 and predates 3.9.x/3.10.0 additions (no dual-stack/Kueue/GPU-partitioning/containerd); a refresh would restore full-reference parity. Bundle version stamps realigned to 3.10.0 this cycle.
• L2/L3/L4 — agent-runtime bundle lacks the standard TOC/Stop-Conditions scaffolding; glossary/quick-reference reachable only via the generic references link; one self-referential anchor in operations-resilience.

Honest unknowns

• No live Azure subscription: exact containerd build in the current AKS node image, and whether AKS 24.04 keeps apparmor_restrict_unprivileged_userns=1, remain [VERIFY] gates.
• AKS Node Disruption Policy official Azure Update ID unconfirmed (GitHub #5017 only).
• Azure Linux 4.0 AKS availability date is "coming soon"; no committed date.
• Score capped below 9.5 by the deferred M5/M1-residual structural items, not by any open correctness finding.

3.9.3 → 3.9.4 Review (Closed)

Summary

• Driver: link-integrity fix. The v3.9.3 quality review called out three pre-existing validator errors in bundles/agent-runtime/guide.md as out of scope. This release closes them, plus fixes two additional broken sibling-skill links in SKILL.md that the validator did not catch (they escape the skill ROOT and are skipped) but were broken for readers.
• Scope: PATCH — link depth corrections only. No guidance content changed.
• Validator: OK: 0 errors, 0 warning(s) (was 3 errors, 0 warnings).
• Root cause: two distinct off-by-one depth bugs. (1) agent-runtime/guide.md is two levels below the skill root, so sibling links need ../../../ but used ../../. (2) SKILL.md is at the skill root, so sibling links need ../ but used ../../. Both produced links that resolve to non-existent paths.
• Verification: after the fix, Test-Path confirmed all four sibling targets (microsoft-agent-framework, mcp-server-design, azure-container-apps, and the SKILL.md-level microsoft-agent-framework/mcp-server-design) resolve to real SKILL.md files under .github/skills/.

Reviewer notes

• Validator blind spot. The validator skips any link whose resolved path escapes the skill ROOT (line 138 of validate-skill.py: resolved.relative_to(ROOT) raising ValueError → continue). This is correct behaviour for ignoring external links, but it means broken links that almost escape ROOT but point to the wrong place are not caught. The SKILL.md ../../ links fell into this blind spot. No validator change is warranted here — the correct fix is to keep sibling links accurate, not to expand validation scope to other skills' files — but the blind spot is documented for future maintainers.
• No content drift. Diff is purely the four link targets plus version/changelog/review bumps. No guidance, routing, or glossary text changed.

3.9.2 → 3.9.3 Review (Closed)

Summary

• Driver: gap-analysis pass. After the v3.9.0 (AGC/Cilium L7 lockdown blog cross-check), v3.9.1 (WAF CRD schema correction), and v3.9.2 (currency refresh) passes, a gap analysis identified two remaining architectural domains the skill did not cover: dual-stack (IPv4/IPv6) networking and batch/AI training scheduling (Kueue). A complementary GPU partitioning subsection was added because Kueue quota contention is meaningful only against a shared GPU fleet.
• Scope: MINOR (additive). Three new subsections, five new glossary entries, five new stop conditions, three new routing-table rows, two new AI/GPU routing bullets. No existing guidance was weakened or retracted.
• Validator: OK: 0 errors, 0 warning(s) (3 pre-existing agent-runtime broken cross-skill links remain — out of scope for this skill).
• Sources: AKS dual-stack overview, Azure CNI Powered by Cilium, GPU node partitioning strategies, Create a managed MIG-enabled AKS node pool (preview), Kueue on AKS overview, Run Ray AI workloads with Kueue on AKS. All fetched 2026-07-12.

Added

1. Dual-Stack Networking subsection in cluster-foundations/guide.md. The skill previously covered IPv4-only VNet design (dynamic VNet + peering, Azure CNI Overlay vs VNet-integrated, subnet sizing) but never dual-stack. Added the --ip-families ipv4,ipv6 flag, the Permanent-at-creation classification (consistent with VNet/outbound-type/autoscaler permanent decisions), the Azure CNI Overlay (or Cilium overlay) requirement, the K8s 1.29+ requirement for Cilium dual-stack, the IPv6-only-not-supported constraint, the standard NAT Gateway exclusion (NAT GW V2 IPv6 support [VERIFY]), the IPv6 service CIDR /108 cap, and a "when to choose dual-stack" driver table.
2. GPU Partitioning subsection in workload-platform/guide.md. The skill previously treated GPUs as dedicated, single-tenant node pools (the "dedicated per-workload accelerators" rule in the sustainability standards). Added the three partitioning strategies — MIG (AKS-managed, hardware isolation, A100/H100/H200, up to 7 instances, static at node-pool level, requires reprovisioning to change), time-slicing (user-managed via NVIDIA GPU Operator, software scheduling, experimentation only), MPS (user-managed, CUDA-level multiplexing) — with a comparison table. Noted the managed GPU node pool preview (gpuProfile.nvidia.managementMode/migStrategy) as [VERIFY] for GA status.
3. Batch and AI Training Scheduling (Kueue) subsection in workload-platform/guide.md. The skill previously had no coverage of job queuing / admission control. Added Kueue's two-level queuing model (ClusterQueue resource pool + LocalQueue tenant queue), fair sharing across cohorts, the Ray+Kueue pattern for distributed AI training, the supported workloads (Job, CronJob, RayJob, MPIJob), and the open-source/community-supported boundary (excluded from AKS SLA). Made the Kueue-manages-admission-vs-Karpenter/NAP-manages-pods distinction explicit.
4. Glossary entries: Dual-stack, Kueue, MIG, MPS.
5. Three new routing-table rows in SKILL.md for the new subsections.
6. Two new AI/GPU routing bullets in SKILL.md cross-linking GPU partitioning and Kueue.
7. Five new Stop Conditions (dual-stack in cluster-foundations; GPU partitioning strategy + reprovisioning plan, Kueue adoption/SLA boundary in workload-platform).

Open [VERIFY] markers carried / introduced

• NAT Gateway V2 IPv6 outbound support (dual-stack section) — the StandardV2 NAT GW SKU is GA and IPv6-capable for general Azure use, but AKS managedNATGatewayV2 IPv6 outbound behavior is not fully documented in a single authoritative source; marked [VERIFY] so reviewers know to confirm at deployment time. Introduced in 3.9.3.
• Managed GPU node pool GA status (GPU partitioning section) — the gpuProfile.nvidia.managementMode and migStrategy properties are documented under a preview feature in Microsoft Learn; whether the feature has reached GA since the Learn page was last reviewed is marked [VERIFY GA]. Introduced in 3.9.3.
• DRS version string on SecurityPolicy (operations-resilience section) — carried from 3.9.1. The blog referenced "OWASP CRS 3.3.4" / "DRS 2.1"; the exact current DRS version string is version-sensitive and marked [VERIFY].
• AGC ALB controller minor-version coupling — carried from 3.9.0; the version pin is documented as a hard rule, not a [VERIFY] flag.

Reviewer notes

• Scope discipline. During research, several adjacent topics surfaced and were intentionally excluded as out-of-scope scope creep: WireGuard transit encryption, Cilium mTLS, eBPF Host Routing, cross-cluster Cilium Cluster Mesh, and the full NVIDIA GPU Operator CRD surface. These are not part of the two identified gaps. They can be addressed in a future pass if the user requests them.
• Preview/ga boundary discipline. Kueue is deliberately framed as "AKS-documented but not AKS-managed" — it is an open-source project that runs on AKS but is excluded from AKS SLA. This prevents users from assuming AKS support for a non-SLA feature. This distinction is now in the Stop Condition, the glossary, and the subsection body.
• Anti-hallucination discipline. Every new flag, field, version number, and SKU name was verified against Microsoft Learn. Two version-sensitive facts (NAT GW V2 IPv6, managed GPU node pool GA) could not be confirmed to a single authoritative source and were marked [VERIFY] rather than asserted.

3.9.1 → 3.9.2 Review (Closed)

Summary

• Driver: currency refresh of feature-status claims. Deep research against authoritative Microsoft Learn revealed that several Preview/GA markers in the skill had drifted: Static Egress Gateway public-IP mode had gone GA, outboundType had become mutable post-creation, new none/block outbound types existed for network-isolated clusters, and default outbound access had been retired.
• Scope: PATCH — currency corrections only. No new sections, no routing changes.
• Validator: OK: 0 errors, 0 warning(s) (3 pre-existing agent-runtime broken cross-skill links remain — out of scope for this skill).
• Sources: AKS egress outbound types (updated 2026-07-03), Configure Static Egress Gateway (updated 2026-07-03), Network Isolated Cluster concepts, NAT Gateway reliability, Azure default outbound access retirement update.

Currency corrections closed

• Static Egress Gateway → GA. The 3.8.3 review (S-003) correctly distinguished the GA private-IP mode from the then-Preview public-prefix mode; both are now GA. Updated outbound connectivity table and dedicated subsection in cluster-foundations/guide.md. Private-IP mode requires K8s 1.34+ and --vm-set-type VirtualMachines.
• managedNATGatewayV2 status clarified. StandardV2 NAT Gateway SKU is GA (zone-redundant by default); the AKS managedNATGatewayV2 outbound type remains Preview. Corrected in outbound table and references/full-reference.md zonal-egress guidance.
• none / block outbound types added for Network Isolated Clusters.
• Default outbound access retirement (March 31, 2026) callout added.
• outboundType mutability documented with migration path constraints (migrating to managedNATGatewayV2 is not supported on a managed VNet).
• Confidential Containers retirement corrected to past tense.

External validation

• All claims verified by fetching the live Microsoft Learn docs directly. Context7 had no AGC library and did not surface the egress/network-isolated docs; direct doc fetch was the reliable path. [VERIFY] markers retained only on genuinely version-sensitive fields (DRS version string, ARM resource ID format, regional availability of Preview features).

Honest unknowns

• No live Azure subscription to execute a loadBalancer → managedNATGateway migration or a none/block cluster bootstrap — CLI examples and migration paths remain verification gates against current Microsoft Learn before execution.

3.8.2 → 3.8.3 Review (Closed)

Summary

• Driver: training data recency bias gate — cross-referenced 8 skill patterns against current AKS feature GA/Preview status to detect superseded claims from model training-era defaults.
• Round 1 angles: Capability recency (trusting track), Hostile review (skeptical track). Score: 7/10.
• Final score: 9.6 / 10 (no Critical or High findings; supersession register closed; all 8 items dispositioned).
• Validator: OK: 0 errors, 0 warning(s).

Must-fix items closed

• S-001 (High) — App Routing Gateway API GA (April 28, 2026) — SKILL.md still referenced "H1 2026 per Microsoft Learn". Updated to GA status with --enable-app-routing default behavior note. operations-resilience guide traffic management table and verification instruction updated to reflect GA.

Should-fix items closed

• S-003 (Low) — Static Egress Gateway Private IP sub-mode GA (March 9, 2026) distinguished from public-prefix Preview mode.
• SS-B (Low) — Windows Server 2019 retirement date (March 1, 2026) added to OS lifecycle guidance.
• SS-C (Low) — Flatcar Container Linux retirement (support end June 8, 2026; removal September 8, 2026) added.

Confirmed correct (no change needed)

• S-002 (managedNATGatewayV2), S-004 (Artifact Streaming), S-005 (Cluster Health Monitor), S-006 (Node pool rollback), S-007 (KEDA workloadAutoScalerProfile), S-008 (Confidential Containers), SS-A (Azure Linux 2.0)

Supersession Register

All 8 items dispositioned. Zero open S-001 items. Disagreement register: empty (both reviewers converged on same set of findings). Gap heatmap: all critical path files had ≥2 independent angle coverage.

External validation

• All feature-status claims verified against AKS release notes (2026-04-28), Microsoft Learn (2026-05-15, 2026-06), and GitHub PR #33215.

Token Efficiency

• Net-neutral — supersession register entries and changelog are additive only. No compression needed.

Honest unknowns

• No live Azure subscription to execute --enable-static-egress-gateway private IP mode or --enable-app-routing-istio after GA — CLI examples remain verification gates.

3.7.0 → 3.7.1 Review (Closed)

Summary

• Driver: coverage hardening after schema-level comparison with Microsoft Learn managedClusters reference and change-log.
• Work: added explicit API volatility/pinning guidance and an IaC property watchlist to references/full-reference.md.
• Final score: 9.6 / 10.
• Validator target: no expected anchor or structure regressions from the new sections.

Gaps closed

• Added a production-safe API policy that prefers GA API versions and treats preview dependencies as ADR-governed exceptions.
• Added a high-impact property watchlist so architecture reviews explicitly check cluster-level and agent-pool-level properties that are frequently missed in generic AKS guidance.

Scope control

• No routing changes.
• No bundle dependency changes.
• No feature-status assertions introduced beyond the pre-existing verification discipline.

3.6.3 → 3.7.0 Review (Closed)

Summary

• Driver: a gap analysis against practical / real-world AKS best practice identified coverage gaps in an otherwise comprehensive skill.
• Work: additive capability sections across four bundles + router wiring, then a re-review round (content accuracy, cross-reference quality, surface parity, consistency, token efficiency).
• Final score: 9.5 / 10.
• Validator: OK: 0 errors, 0 warning(s).

Gaps closed

• Tier 1 (high real-world impact): persistent storage / StorageClass + zonal-disk trap; CoreDNS-at-scale (ndots, throttling, NodeLocal DNSCache); node allocatable vs capacity; image-pull-at-scale (ACR throttling, geo-replication, Artifact Streaming).
• Tier 2: Static Egress Gateway (per-namespace egress IP); FinOps tooling (Cost Analysis add-on + Kubecost); pricing tier / SLA; Windows node-pool depth; resilience/chaos validation.
• Tier 3: cluster certificate rotation.

Re-review findings (resolved in the same pass)

• The 3.6.3 SKILL.md repoint and the new CoreDNS subsection both promised "the full symptom and fix" at operations-resilience#known-pitfalls, but only a generic "CoreDNS mistakes" row existed. Added dedicated CoreDNS split-DNS and overload/ndots pitfall rows so the cross-references resolve to real content.
• Threat-model mitigation cited "gVisor" (a GKE sandbox, not available on AKS); corrected to Pod Sandboxing (Kata) / Confidential VM node pools, consistent with the skill's isolation tiers.
• Confirmed no contradiction with the 3.6.3 Confidential Containers retirement (grep audit of all confidential containers mentions — remaining references are the deprecation banner, the "when NOT to use" context, and a generic threat-model mention).

External validation (new claims, Microsoft Learn)

• Static Egress Gateway (Preview), AKS Cost Analysis add-on (GA), Artifact Streaming (Preview) — all confirmed against current Microsoft Learn before assertion. Version/region-sensitive specifics (node-allocatable reservation %, ZRS-disk availability, SLA %, Windows version support) left as explicit [VERIFY] markers.

Token Efficiency

• Content grew +3,600 words (+4.7k est tokens), concentrated in the lazy-loaded bundles, not the SKILL.md router. Partial-loading structure preserved: a typical task still loads the router plus one or two bundles. New material is dense (tables, terse bullets, [VERIFY] markers) in house style.

Honest unknowns

• No live Azure subscription / CLI to execute the new az aks examples (--enable-static-egress-gateway, --enable-cost-analysis, rotate-certs); they remain verification gates. Reservation/SLA/ZRS numerics are intentionally not hard-coded.

Previous Review (3.6.3)

Review date: 2026-06-16 Stopping condition: validator green, parity warnings zero, zero open Critical/High findings, and external validation completed against current Microsoft Learn / official source surfaces for every changed claim.

This document records the review state at version 3.6.3. Earlier release history remains in CHANGELOG.

3.6.2 → 3.6.3 Findings (Closed)

Summary

• Rounds completed: 1 direct review/fix pass after baseline discovery and a proportional external-evidence pass.
• Angles covered: content accuracy, cross-reference quality, surface parity, production deployment / upgrade currency, final correctness / prose, token efficiency.
• Final score: 9.5 / 10.
• Validator: OK: 0 errors, 0 warning(s).

Must-fix items closed (Critical / High)

• Critical — SKILL.md referenced a non-existent file known-pitfalls.md for the CoreDNS split-DNS trap. The link checker did not catch it (backtick text, not Markdown-link syntax). Repointed to the real section bundles/operations-resilience/guide.md#known-pitfalls.
• High (content accuracy) — AKS Application Routing managed-NGINX ingress class was webapprouting.azure.com in four places; Microsoft Learn documents it as webapprouting.kubernetes.azure.com. Corrected all four.
• High (currency contradiction) — production-workload-controls presented per-pod Confidential Containers as a current production isolation tier, contradicting the cluster-foundations retirement banner and Microsoft Learn (preview; node images removed 2026-03-31). Reframed the tier around Confidential VM node pools and added a deprecation callout plus a cross-link.

Should-fix items closed (Medium / Low)

• Removed two leaked fix-pack editorial notes ("adjust if Pack S renames it", "verify after Pack O lands") whose target anchors already resolve.
• Replaced a stale "cross-link … once that exists" placeholder with a live link to the operations-resilience audit/SIEM section.
• Fixed broken prose in full-reference.md ("are but should not become defaults").
• Restored glossary parity by adding the missing MCP row to references/glossary.md.

Refuted by external evidence (no change)

• Pod Sandboxing runtimeClassName was suspected stale (kata-vm-isolation vs a guessed kata-mshv-vm-isolation). Microsoft Learn confirms kata-vm-isolation is exactly correct — left unchanged. Verifying before editing avoided introducing a wrong "verified-looking" value.

External validation results

• Official docs (Microsoft Learn): Confidential Containers preview retiring (node images removed 2026-03-31); Pod Sandboxing kata-vm-isolation; App Routing ingress class webapprouting.kubernetes.azure.com. All three drove or confirmed an edit.
• Release notes / blog / GitHub: ingress-nginx EOL March 2026, AKS managed NGINX patches through November 2026, Application Routing with Gateway API (Istio control plane) H1 2026 — skill already correct.

Token Efficiency

• Total estimated tokens: ~57,551 (44,270 content words × 1.3 proxy, ±30%).
• Estimation method: wc -w × 1.3 (tiktoken not available).
• Largest files: operations-resilience/guide.md ~15.4k; full-reference.md ~7.8k; production-workload-controls/guide.md ~7.5k.
• Net-neutral change: correctness pass, not compression. Partial-loading structure (lightweight SKILL.md router + lazy bundles + references) preserved — a typical task loads the router plus one or two bundles, far below the 57.5k total.

Honest unknowns

• No live Azure subscription or installed Azure CLI was available; az aks / kubectl examples remain verification gates, not locally executed. The skill is markdown guidance with no unit-test / build / smoke-test harness — those checks are recorded as not available, not as passed.
• Illustrative CVE identifiers (AKS-2026-0003 / CVE-2026-31431 / Dirty Frag CVE-2026-43284/43500) remain explicitly marked "verify against the current AKS Security Bulletins feed" throughout; they were not re-verified this pass and are intended as worked examples only.

Previous Review (3.6.2)

Review date: 2026-06-15 Stopping condition: validator green, parity warnings zero, actionable metadata drift fixed, and external validation sampled against current Microsoft Learn / official source surfaces.

3.6.1 → 3.6.2 Findings (Closed)

Summary

• Rounds completed: 1 direct review/fix pass after baseline discovery.
• Angles covered: content accuracy, surface parity, cross-reference quality, production deployment, error handling and resilience, observability/debuggability, upgrade and breaking-change surface, developer experience, token/metadata efficiency.
• Final score: 9 / 10.
• Validator: OK: 0 errors, 0 warning(s).

Must-fix items closed

• Fixed stale Markdown anchors in bundles/operations-resilience/guide.md that caused six validator warnings.
• Corrected stale SecurityPatch stop-condition wording that incorrectly implied a preview feature flag was required.
• Synchronized bundles/catalog.yaml and all five bundle manifests from 3.5.2 to 3.6.2 so bundle metadata matches the current 3.6.x release stream.

Should-fix items closed

• Promoted the PDB drain-stall pitfall into a real heading so the pre-upgrade runbook can link to it directly.
• Replaced a broken TLS planning cross-link with an existing secrets/data-at-rest section rather than leaving a dangling anchor.

External validation results

GitHub / live source

• Rechecked Azure/AKS issue #4525. Finding remains already covered: ACNS/Cilium FQDN/DNS policy interactions can crash Cilium agents on affected versions; the skill already instructs pairing DNS allow rules and validating in non-production.

Official docs

• AKS Automatic managed system node pools: aligned with Microsoft Learn limitations, LocalDNS default, security restrictions, unsupported operations, and CLI 2.86.0+ requirement.
• Ingress/App Routing: aligned with Microsoft Learn guidance that upstream ingress-nginx maintenance ends March 2026, AKS managed NGINX critical security patch support ends November 2026, and Gateway API / AGC are migration targets.
• Node OS upgrades: aligned with Microsoft Learn planned maintenance and auto-upgrade docs for NodeImage, SecurityPatch, aksManagedAutoUpgradeSchedule, and aksManagedNodeOSUpgradeSchedule.
• Fleet Manager auto-upgrade: aligned with Microsoft Learn for NodeImage auto-upgrade profiles and az fleet autoupgradeprofile generate-update-run.
• managedNATGatewayV2: aligned with Microsoft Learn as Preview requiring the ManagedNATGatewayV2Preview feature flag.

Community / release notes

• No new actionable community pitfall was added in this pass. Existing ACNS/Cilium FQDN policy pitfall remains covered.

Token Efficiency

• Total estimated tokens after: 58,032 (44,640 words × 1.3 proxy).
• Deduplication wins: no large content removal was needed; this pass prioritized correctness and metadata sync over compression.
• Compression wins: none applied because the changed sections carry validation and routing value.
• Capability signals preserved:
  • AKS Automatic vs Standard routing.
  • Cilium/CNI and egress decision guidance.
  • Fleet Manager update orchestration.
  • Node image / CVE response runbooks.
  • Gateway API / ingress migration guidance.

Bundle integrity

• Bundle schema detected: catalog version: 3 with a list of bundle entries.
• Catalog references: five bundle paths, guides, and manifests exist.
• Version updates: five catalog bundle entries and five bundle manifests updated from 3.5.2 to 3.6.2.

Honest unknowns and constraints

• Three background reviewer/research agents failed because the workspace reported insufficient billing balance. Direct repository tools and Microsoft Learn/web evidence were used instead.
• No live Azure subscription, target AKS cluster, az aks command output, or installed Azure CLI version was available; runtime environment-specific commands remain verification gates, not locally executed checks.

Previous Review Snapshot

Quality Review - aks-cluster-architecture 3.5.2

Review date: 2026-05-14 Stopping condition: all reviewers ≥ 8/10, zero must-fix in the last round, ≥ 8 angles covered, plus three independent external validation passes.

This document records the final review state at version 3.5.2. It is a snapshot, not a running log - earlier rounds are captured in CHANGELOG and in the per-bundle drift corrections.

Reviewer Angles and Scores (Round 5)

All angles ≥ 8/10. Zero must-fix items remained at the end of Round 5.

External Validation Passes

Three independent external validation passes ran against version 3.5.2 content. Each pass produced drift findings that have been corrected in this release (see CHANGELOG 3.5.2).

Pass 1 - Microsoft Learn

• Verified AKS Automatic regional and policy constraints (West US zone limit, Deployment Safeguards fixed level).
• Verified SecurityPatch node OS upgrade channel is GA.
• Verified Confidential Containers retirement announcement (March 2026) and Confidential VM node pool continued support.
• Verified node OS upgrade channel matrix and IaC surface anchors.
• Verified ingress-nginx upstream EOL and AKS managed-NGINX EOS dates.

Pass 2 - Upstream GitHub sources

• Verified AKS issue #4525 - FQDN egress + default-deny without CoreDNS allow causes Cilium DNS crash-loop.
• Verified kube-no-trouble repository inactivity since August 2024; replaced primary recommendation with pluto (Fairwinds, actively maintained).
• Verified Ratify project relocation from ratify-project/ratify to notaryproject/ratify under CNCF Sandbox; API group config.ratify.deislabs.io/v1beta1 confirmed as current production-suitable surface; v2alpha1 flagged as in-development only.
• Verified kube-storage-version-migrator controller as the current upstream tool for CRD storage-version migration.

Pass 3 - Community signal

• Verified AKS Security Bulletin AKS-2026-0003 (algif_aead kernel LPE - Copy Fail) and CVE-2026-31431 as the correct bulletin reference; replaced prior fabricated CVE IDs with verified IDs plus an explicit verification caveat.
• Verified AKS Capacity-Based Surge release (April 2026) allowing maxSurge + maxUnavailable to coexist on node-pool upgrades.
• Verified SLSA Build L2/L3 targets, OpenVEX/CSAF-VEX expiry/owner conventions, and Notation/Cosign Key Vault custody pattern remain consistent with current community practice.

Validator Status

python scripts/validate-skill.py
OK: 0 errors, 0 warning(s)

Packaging

• Versioned zip: output/aks-cluster-architecture-3.5.2.zip
• Skill structure: top-level SKILL.md, CHANGELOG.md, QUALITY-REVIEW.md, bundles/, references/, scripts/.
• Stopping condition met. No further review rounds required for this version.