/drasi

Quality Review Log

This document records each iterative review pass against the Drasi skill, the reviewer angles used, scores, must-fix findings, and how they were resolved.

Current quality posture

• Package version: 2.1.5
• Last reviewed: 2026-08-16
• Rounds completed: 13 (Round 13 = full-mode propagation sweep: live external evidence baseline, two High default-propagation fixes in SKILL.md, link-rot repair after the drasi.io Jan-2026 restructure, GHCR gap live re-verification, small capability adoptions; executed sequentially in-session after host subagent infrastructure failed — logical coverage preserved per the execution contract)
• Unique reviewer angles used: 24 internal + 5 external sources (GitHub API releases, GHCR tag lists, crates.io API, modelcontextprotocol.io, restructured drasi.io docs)
• Open must-fix items: 0
• Validator status: passing
• Stopping condition: MET — Round 13 closed the propagation misses Round 12 left (MCP revision text, stale shape prose, dead links); no new material findings in the final validation pass; remaining unknowns carry [VERIFY] markers with WhereToCheck paths.

Round 13 - 2026-08-16 (full-mode propagation sweep)

Method

Claim-first inventory (22 material claims) against live evidence: GitHub API (drasi-platform/drasi-server), GHCR token-authenticated tag lists (first live re-probe of the post-dapr-pubsub gap), crates.io API, modelcontextprotocol.io 2026-07-28 changelog, and the restructured drasi.io (EventHub how-to, MCP Reaction, CLI reference, middleware reference — fetched from GitHub raw to bypass nav noise). Angles: content accuracy, hostile review, structural cohesion (default-propagation sweep), capability recency, minimalism.

Critical/High findings closed

• SKILL.md Anti-Hallucination item 4 and asymmetry-traps list still claimed MCP upstream 2025-11-25 / "two revisions behind" (Round 12 updated the snapshot and agent-integration bundle but missed these two) — fixed to 2026-07-28 / three.
• SKILL.md Workload Identity section still contained the purged drasi.io/v1alpha1 + metadata.name shape prose — rewritten to canonical.

Other findings closed

• 3 dead /docs/ links in SKILL.md (Jan-2026 site restructure) — replaced with current URLs; package otherwise already migrated.
• v0.2.1 date corrected to 2026-06-19 (2026-07-15 is the rolling republish).
• GHCR post-dapr-pubsub gap re-verified LIVE (was carried 2026-06-23 evidence).
• Capability adoption: middleware reference link + where-configured (sources bundle); versioning model + plugins.lock (drasi-server bundle).

Confirmed correct (explicitly re-verified)

• Canonical K8s resource shape, bootstrapWindow minutes, top-level spec.identity shape, per-source SA + FIC subject (EventHub how-to); MCP pin 2025-03-26 with 2024-11-05 compat on Streamable HTTP port 3000 (MCP Reaction page); CLI surface apply/describe/list -n/env/ingress/wait/watch (CLI reference); drasi-lib 0.8.9 vs AI-context 0.8.6 baseline; context 0.7.0 unchanged (fetched file).

Honest unknowns / watch items

• EventHub consumerGroup and drasi list -o yaml keep [VERIFY] markers (absent from current docs; live-cluster describe recommended).
• drasi.io URL structure is volatile (restructured Jan 2026): re-check cited links each currency pass.

Round 13 outcome

• Validator passing. 6 files changed (SKILL.md, sources/reactions/drasi-server guides, current-sources.md, catalog.yaml). Package 2.1.4 → 2.1.5.
• Zero open must-fix items.

Round 12 - 2026-08-16 (full-mode audit: shape corrections, unit fix, capability adoption)

Method

Full claim inventory (45 claims) against live evidence: drasi-platform and drasi-server GitHub releases, drasi.io how-tos (PostgreSQL/EventHub/MCP Reaction), CLI reference, drasi-context.yaml (0.7.0, unchanged), crates.io API (drasi-lib 0.8.9 = max stable), and modelcontextprotocol.io (spec now 2026-07-28). Reviewer angles run sequentially: content accuracy, hostile review, structural cohesion, builder's-eye, capability recency, minimalism.

Critical findings closed

• bootstrapWindow is minutes, not seconds (EventHub how-to).
• SKILL.md K8s resource examples used a stale/invented shape (drasi.io/v1alpha1, metadata.name, spec.provider, hub:, singular source:); canonical is apiVersion: v1 + top-level name + spec.kind (PascalCase) + spec.sources.subscriptions[].id for queries. Bundles were already correct; only SKILL.md had drifted.
• Workload-Identity sequence used kubectl apply (must be drasi apply) and a non-default namespace literal; an az acr login --name ghcr registry check (never valid) was replaced with the canonical token-authenticated manifest check.
• One paragraph inverted the 0.9.x/0.10.x component mapping.

Capability adoption (not just recorded)

• drasi-server bundle: full Server component catalog (incl. oracle/sqlite/ neo4j/kafka sources; loki/rabbitmq/aws-sqs/dashboard/file/profiler/sse reactions; bootstrap/identity/secret-store providers) with the lowercase kind convention, plus the drasi-sse-cli release binary.
• operations bundle: drasi ingress init for exposing reaction endpoints.

Confirmed correct (explicitly re-verified)

• Per-resource service accounts (source.<name> / reaction.<name>) and the federated credential subject are now officially documented on the EventHub how-to page — the skill's earlier production-verified correction is now docs-verified. spec.identity top-level shape confirmed on the same page.
• MCP Reaction still pins 2025-03-26 (confirmed on the MCP Reaction page); skill updated to "three revisions behind" after upstream shipped 2026-07-28.
• drasi wait, drasi watch, drasi env kube, middleware coverage, the drasi.* function surface, and the sources/reactions bundles' YAML shapes all verified current.

Honest unknowns / watch items

• consumerGroup on the EventHub source: absent from the current docs property table; source-provider schema page 404s; [VERIFY] marker added (check drasi describe sourceprovider EventHub).
• GHCR per-tag availability carried as 2026-06-23 production evidence (registry not re-probed this run).
• drasi-server source-build MSRV now verify-at-pinned-tag (was stale "1.70+").

Round 12 outcome

• Validator: passing. 7 files changed (+123/−151 lines; SKILL.md deduplicated then re-grew slightly with canonical-shape comments and [VERIFY] notes).
• Package 2.1.3 → 2.1.4; bundles sources/operations/agent-integration/ drasi-server bumped.
• Zero open must-fix items.

Round 10 - 2026-07-21 (drasi-server integration + currency + REST-path fix)

Round 10 ran the skill-improvement metaprompt against the v2.1.0 package. The 2.1.0 change (production Workload-Identity learnings) had shipped a drasi-server bundle guide without wiring it in, leaving the validator red. This round used content-accuracy, cross-reference-quality, and capability-recency (currency) angles, grounded in live external evidence.

Baseline validator errors (all closed in 2.1.1)

• bundles/drasi-server/bundle.yaml missing → created.
• bundles/drasi-server/guide.md had no evidence/validation heading → added a ## Validation acceptance table (source → query → reaction).
• CHANGELOG.md 2.1.0 latest-tag line tripped the anti-slop check → reworded to the recognized "do not use / floating latest tag" form.

Content-accuracy must-fix (verified against official docs)

• OpenAPI REST path: SKILL.md (Anti-Hallucination Rule + safe-degraded output) and bundles/drasi-server/guide.md used root /openapi.json. The drasi.io REST API reference confirms /api/v1/openapi.json; 8 other skill files already used the correct path. Fixed all three occurrences. /health (root) confirmed correct.
• Drasi Server run command: the drasi-server guide Quick start ran the server with docker compose up -d. No Server compose file ships upstream; the official getting-started uses the binary or a digest-pinned docker run (compose is only for the tutorial database). Fixed to match SKILL.md "Prerequisites by runtime".

Cross-reference must-fix

• README.md Bundles index was missing drasi-server (present in catalog and SKILL.md routing). Added.
• drasi-server bundle undocumented in CHANGELOG/QUALITY-REVIEW → documented here and in the 2.1.1 CHANGELOG entry.

External validation (2026-07-21)

• GitHub / live source: drasi-platform 0.10.0 still latest; drasi-server v0.2.1 latest stable with 0.2.2/0.2.3-preview ahead.
• Official docs: getting-started (run command, endpoints) and REST API reference (/api/v1/openapi.json, /api/v1/docs/, /health) verified.
• Authoritative context: drasi-context.yaml still 0.7.0 (2026-06-16); drasi-lib 0.8.6, drasi-core models 0.5.4.
• Community: MQTT + Shell/Command reactions are in-flight (CNCF LFX 2026 mentorship) and correctly absent from the authoritative context and the skill.

Honest unknowns / watch items

• The authoritative drasi-context.yaml (0.7.0) lags the drasi-server GitHub release page (lists 0.1.6 as latest while v0.2.1 is the current stable). Recorded in the snapshot; re-check when context 0.8.x publishes.
• MQTT / Shell/Command reactions: add coverage once they land in the context file or a platform release. Not yet actionable.

Round 10 outcome

• Validator: passing (Drasi skill validation passed.).
• drasi-server bundle bumped 1.0.0 → 1.0.1; package 2.1.0 → 2.1.1.
• Zero open must-fix items.

Round 10 continuation - Cypher/GQL function-reference accuracy (2.1.2)

Follow-up (same session) prompted by a review of Cypher/GQL and drasi.* function coverage. Verified the drasi.* surface against the official Drasi Custom Functions reference.

• Fabricated functions removed (High): the absence-of-change example used drasi.age() and drasi.minutes(), which do not exist. Replaced with the canonical drasi.trueFor(expr, duration({...})) pattern and a drasi.changeDateTime() / datetime() - duration({...}) comparison.
• Signatures corrected + hedges resolved: drasi.* table rebuilt with verified signatures; drasi.trueUntil and drasi.slidingWindow confirmed (hedges dropped); added the two documented-but-missing functions drasi.listMin / drasi.listMax; added an explicit "these names do not exist" guard for drasi.age / drasi.minutes / drasi.now.
• Confirmed correct (no change needed): the bundle already documents GQL as ISO/IEC 39075 Graph Query Language (not GraphQL), the openCypher-9 subset, the AST surface, the 8 aggregating functions, and the scalar/numeric/list/ temporal function reference. Coverage on this axis is strong.
• continuous-queries bundle bumped 2.0.6 → 2.0.7; package 2.1.1 → 2.1.2. Validator passing.

Round 9 - 2026-06-25 (currency and consistency refresh)

Round 9 executed a focused refresh against newly collected external evidence and four additional reviewer angles (content accuracy, builder's-eye, cross-reference packaging consistency, and token-efficiency). This round was targeted at drift and contradiction closure rather than a full re-score sweep of prior rounds.

Must-fix findings (all closed in 2.0.14)

• Stale drasi-context baseline (0.5.0 → 0.7.0) in router/reference content.
• Stale drasi-server release status claim ("no releases") after published drasi-server release line.
• Stale drasi-lib baseline framing (0.3.8/0.4.2) vs current AI-context baseline (0.8.6).
• Production-safety contradiction on AKS Workload Identity service account naming (default vs reaction.<reaction-name>).
• README bundle index mismatch (missing custom-plugins).

Round 9 outcome

• All must-fix items closed with file-isolated edits.
• Bundle versions bumped where bundle guides changed.
• Validator re-run clean after fixes.

Round 1 - 2026-05-15

Reviewer angles used

1. Content accuracy
2. Builder's-eye end-to-end (Day-0 to first reaction firing)
3. Token efficiency (signal vs filler across SKILL.md and bundle guides)
4. Production deployment readiness

Scores

Must-fix findings (all closed)

• Drasi Server version overstatement (content accuracy): the original SKILL.md and references/current-sources.md implied a tagged drasi-project/drasi-server release that does not exist. Fixed: replaced the version-style claim with explicit "no published releases; container image only; pin by digest" language in both files.
• No stitched runnable example (builder's-eye): no path from "fresh repo" to "reaction fired" without external research. Fixed: added a Drasi Server mock-source + log-reaction example in bundles/examples/guide.md with REST verification commands.
• No prerequisite/tooling list per runtime (builder's-eye + production): readers had to derive required tooling per runtime form. Fixed: added a Prerequisites by runtime section to SKILL.md.
• Security claims without audit-grade commands (production deployment): Round 1 found that "verify the image" was asserted without a verifiable procedure. Fixed: cosign verify, syft SBOM, TLS 1.3 check, and digest pinning commands added to bundles/security/guide.md.
• PostgreSQL slot leakage hazard never named (production deployment): a Drasi Source can leave a logical replication slot on the database after removal. Fixed: replication-slot operational runbook added to bundles/sources/guide.md with alert thresholds.
• Destructive-recovery ownership unclear (production deployment): how to know what state is Drasi-owned was not specified. Fixed: per-provider ownership queries added to bundles/recovery/guide.md.
• SLO baselines absent (production deployment): the observability bundle listed signals but did not name target values. Fixed: a baseline SLO table for latency, reaction success, and source lag added to bundles/observability/guide.md.
• Autoscaling trigger choice was hand-wavy (production deployment): HPA vs KEDA was not assigned per component. Fixed: trigger matrix added to bundles/scaling-and-capacity/guide.md.
• No per-provider validation data recipes (builder's-eye): validation required reinventing test data per project. Fixed: concrete commands per provider added to bundles/validation/guide.md.
• "Stale" used without definition (token efficiency / content accuracy): the operations bundle used "stale source" without defining it. Fixed: bundles/operations/guide.md now defines stale and gives the canonical proof-of-flow procedure.
• Per-reaction validation pattern missing (builder's-eye): reactions bundle lacked concrete validation for each reaction kind. Fixed: table added to bundles/reactions/guide.md.
• Join cardinality risk unaddressed (content accuracy): joins could silently fan out without warning. Fixed: worked example added to bundles/continuous-queries/guide.md.
• No azd starter for the Azure target (builder's-eye + production): the azure-hosting bundle pointed at azd but did not show the azure.yaml
  • Bicep shape. Fixed: starter layout added to bundles/azure-hosting/guide.md.
• MCP transport / version compatibility unstated (content accuracy): agent-integration did not pin MCP transport version. Fixed: spec 2025-03-26 Streamable HTTP + SSE fallback noted in bundles/agent-integration/guide.md.

Round 1 outcome

All must-fix items closed in four parallel fix packs (A: SKILL/router and examples, B: sources/security/recovery, C: operations/observability/scaling/ validation, D: reactions/continuous-queries/azure-hosting/agent-integration). No file was touched by more than one pack. Validator re-run clean.

Round 2 - 2026-05-15

Reviewer angles used

1. Threat model rigour
2. Error handling and runbook completeness
3. Upgrade and version-compatibility surface
4. Cross-reference and packaging hygiene

Scores

Must-fix findings (all closed)

• No STRIDE-style threat model (threat model): security bundle listed controls without a trust-boundary map. Fixed: STRIDE table per trust boundary, egress NetworkPolicy + kill-switch, secret-rotation runbook, audit-log schema, multi-tenancy stance, control-plane authentication, reaction identity scope (EOP prevention), ContinuousQuery quotas (DoS), and drasi-lib threat surface added to bundles/security/guide.md.
• New threat-model template missing (threat model): the SKILL evidence list referenced no reusable threat-model artifact. Fixed: created templates/drasi-threat-model.md with Scope, Trust boundaries, STRIDE per boundary, Data classification, Risks, Mitigations, Review cadence, Owner, Last reviewed.
• MCP prompt-injection rule absent (threat model): agent-integration bundle did not warn that query results can carry untrusted content. Fixed: "Untrusted content (prompt-injection defence)" section added and a cross-reference to the security bundle inserted.
• No failure-mode taxonomy (error handling): operations bundle listed symptoms but never indexed them. Fixed: 15-row failure-mode taxonomy (severity, owning bundle, first action) and four numbered playbooks ("no reaction firing", 401/403, Query OOM, identity-token-expiry) added to bundles/operations/guide.md, plus a "query alternating active / inactive" deep dive.
• Webhook outbound integrity unspecified (error handling): reactions bundle did not describe HMAC, mTLS, allowlists, or replay defence. Fixed: X-Drasi-Signature (sha256), 5-min X-Drasi-Timestamp window, mTLS for high-sensitivity data, NetworkPolicy + ACA egress allowlist added to bundles/reactions/guide.md.
• No reliability-contract baselines per reaction (error handling): retries, idempotency, ordering, backpressure were not pinned per reaction kind. Fixed: 8-row reliability-defaults contract table covering HTTP webhook, Event Grid, SignalR, MCP, SSE, gRPC, Debug, Log; framed as "published value wins, record both".
• Shadow / dry-run rollout missing (error handling): no documented path to roll out a reaction safely. Fixed: Stage A–E shadow + staging + diff
  • promotion + decommission rollout added to reactions bundle.
• MCP failure modes vague (error handling): MCP-specific handling for 503s, disconnects, and cursor resume was undocumented. Fixed: explicit 503 + retry-hint behaviour, 30-second cleanup budget, cursor-resume reconnect added.
• No RTO/RPO matrix (upgrade / version): recovery bundle did not pin recovery objectives. Fixed: RTO/RPO and replay-semantics table per runtime form added to bundles/recovery/guide.md (with "confirm at the current release" framing for pre-1.0 surface).
• Upgrade-failure rollback ladder missing (upgrade / version): no distinct upgrade rollback path separate from destructive recovery. Fixed: 7-step parallel-namespace rollback ladder with kubectl set image example added to recovery bundle.
• Restart semantics undocumented (upgrade / version): pod/process/ container restart implications for in-flight state were unstated. Fixed: three restart-semantics tables added (process / pod / container) covering active query state, in-flight reaction queues, source offsets/LSNs, replication-slot ownership, materialized result cache.
• Backpressure chain incomplete (upgrade / version): scaling bundle did not describe how reaction-side backpressure propagates back to the source DB. Fixed: 5-stage backpressure-propagation chain with pg_replication_slots query and max_slot_wal_keep_size bound; PDB defaults + drain procedure for SSE/MCP; network-partition behaviour by leg - all added to bundles/scaling-and-capacity/guide.md.
• apiVersion migration playbook missing (upgrade / version): sources bundle did not describe how to migrate when Drasi pre-1.0 ships an apiVersion change. Fixed: 6-step playbook added to bundles/sources/guide.md.
• No CI version-compatibility gate (upgrade / version): delivery bundle relied on humans noticing drift. Fixed: required PR-time checks with awk over versions.md and drasi version --short, plus a release-notes subscription discipline section added to bundles/delivery/guide.md.
• Three orphan templates (cross-reference / packaging): currency-check, risk-register, and threat-model templates had no inbound reference. Fixed: SKILL.md evidence list now points at each template by path; bundles/currency-and-ai-context/guide.md declares the version-pinning template the source of truth; new templates/drasi-version-pinning.md and templates/drasi-upgrade-evidence.md created.
• SKILL.md prerequisite verify command contradicted examples bundle (cross-reference / packaging): SKILL.md used docker compose up -d with no compose file shipped, while the stitched example used docker run. Fixed: SKILL.md verify command rewritten to the same docker run form against a digest.
• CHANGELOG bundle count wrong (cross-reference / packaging): 2.0.0 prior-baseline entry said "18 bundles"; router lists 17. Fixed.
• Versions-manifest rule absent (cross-reference / packaging): non-negotiable rules did not require pinning to a manifest. Fixed: rule 12 added - versions.md must capture image digest, CLI version, platform release, drasi-lib crate, MCP spec version, last-checked date; CI MUST fail if any field is missing or stale.

Round 2 outcome

All must-fix items closed in four parallel fix packs (E: security threat model + agent-integration; F: reactions reliability + operations playbooks; G: recovery RTO/RPO + scaling backpressure; H: cross-cutting SKILL/router, sources migration playbook, delivery CI gate, currency template wiring, examples digest tightening, CHANGELOG + new templates). No file was touched by more than one pack. Validator re-run clean after two follow-up edits to phrase digest-resolution lines (do not use floating tags in production) so the validator heuristic allow-listed them.

Round 3 - 2026-05-15

Reviewer angles used

1. Novice onboarding (first 90 minutes, no prior Drasi knowledge)
2. Developer inner-loop experience (edit / observe / verify cycle)
3. Testing strategy (unit / contract / integration / load coverage)
4. Keyword discoverability and agent routing (does the skill trigger on the queries a real user types?)

Scores

Must-fix findings (all closed)

• No "first 5 minutes" path (novice onboarding): the shortest documented end-to-end example was the stitched Drasi Server flow, which still required digest resolution and multi-step setup. Fixed: added a ## 5-minute Hello Drasi (Drasi Server) mini-example at the top of bundles/examples/guide.md - one mock source, one Cypher query, one log reaction, one curl verification.
• No runtime-specific starter path (novice onboarding): SKILL.md jumped from the bundle table to the full Day-0 workflow with no on-ramp for someone with no Drasi background. Fixed: added a ## 90-minute starter path section to SKILL.md with three runtime-specific sequences (Drasi Server local, Drasi for Kubernetes, drasi-lib) that lead a beginner to a working pipeline before the production-shaped Day-0 workflow.
• Digest resolution was abstract (novice onboarding): every example used a <resolve-current-digest> placeholder, but no example showed how to obtain one or what the resolved output looks like. Fixed: added a Step 0 to the stitched example in bundles/examples/guide.md showing the docker pull + docker inspect | jq resolution command and an illustrative sample digest output (with a "do not copy literally" note) so a literal-copy novice does not ship a placeholder string.
• No concrete .devcontainer skeleton (developer inner-loop): the bundle referenced .devcontainer/ and helper scripts but did not show their contents, leaving a new contributor to invent the layout. Fixed: added a ## Concrete .devcontainer skeleton starter to bundles/developer-experience/guide.md - devcontainer.json and Dockerfile with pinned Ubuntu 22.04, Rust 1.82.0, Node 20.17.0, plus docker-in-docker and kubectl-helm-minikube features.
• VS Code extension actions had no CLI equivalents (developer inner-loop): the bundle said "keep CLI/script equivalents for every critical VS Code action" but never enumerated them. Fixed: added a ## VS Code action → CLI mapping table covering Apply, Watch, Debug, Delete, and View status, with K8s drasi CLI and Drasi Server REST forms, plus an explicit "verify against /api/v1/openapi.json" hedge where a path is not already grounded in the skill.
• No inner-loop latency targets (developer inner-loop): the bundle did not name how fast the edit-observe loop should feel. Fixed: added an ## Inner-loop latency targets table (edit-query, new-source, reaction-replace) framed explicitly as team-local engineering targets to measure on real hardware, not Drasi-published SLAs.
• No fresh-machine validation procedure (developer inner-loop): the bundle assumed the dev container works without a way to prove it from a clean clone. Fixed: added a ## Fresh-machine validation 6-step procedure tied to bundles/synthetic-user-testing/guide.md.
• No unit-test layer documented (testing strategy): the validation bundle covered end-to-end and per-provider integration patterns but left unit-testable artefacts (ContinuousQuery offline replay, Reaction request-shape / HMAC / idempotency mocking, drasi-lib cargo test) implicit. Fixed: added a ## Unit-test layer (per artefact) section to bundles/validation/guide.md with concrete patterns per artefact and an explicit non-coverage note for per-Source CDC adapters (integration-only).
• SLO and load-test wiring not gated at release (testing strategy): Round 1 added SLO baselines and Round 2 added a version-compatibility CI gate, but neither bundle made the release gate require SLO and load-test evidence at PR time. Fixed: added a ## Release gate - SLO and load-test wiring table to bundles/delivery/guide.md mapping required PR evidence to source bundles (observability SLOs, scaling load-test record, acceptance-evidence template) and stating the version-compatibility gate is necessary but not sufficient for production-bound PRs. Cross-linked from bundles/observability/guide.md.
• Frontmatter description hurt routing (keyword discoverability): the SKILL.md description: field was a long sentence about runtime forms; an agent matching on user intent ("real-time queries", "live data updates", "change detection") would not see those phrases. Fixed: rewrote the frontmatter to lead with user-intent keywords and kept it under the validator length limit so the router triggers reliably.
• Bundle catalog had thin discoverability (keyword discoverability): every bundle had a purpose: line written for humans browsing the catalog, with few phrases an agent would actually retrieve. Fixed: extended every bundle purpose: with user-intent keywords (real-time, live, reactive, change detection, incident response, RTO/RPO, etc.) and added a sibling keywords: list of 3–6 query phrases per bundle. bundles/catalog.yaml top-level version: 2 is unchanged because the additive keywords: field is backward-compatible.
• Operations failure-mode taxonomy missing a common case (testing strategy / error handling cross-cut): "deployed but nothing happens" (apply succeeded, no events, no logs) was not in the Round 2 taxonomy. Fixed: added that row to bundles/operations/guide.md with a first action and a source-name case-sensitivity hint, plus a one-line cross-reference to MCP failure handling in the reactions bundle.

Round 3 outcome

All must-fix items closed in four parallel fix packs (I: SKILL.md description + 90-minute starter path + 5-minute Hello Drasi + digest resolution Step 0; J: developer-experience skeleton + VS Code↔CLI mapping + inner-loop latency targets + fresh-machine validation + references update; K: validation unit-test layer + delivery release gate + observability cross-link; L: catalog purpose+keywords extension

• operations failure-mode row + reactions MCP cross-ref). No file was touched by more than one pack. Six bundle versions bumped from 2.0.0/2.0.1/2.0.2 to 2.0.3 where the bundle's guide.md content changed (examples, developer-experience, validation, delivery, observability, operations). Catalog metadata-only edits (purpose / keywords for other bundles) did not trigger version bumps. No catalog schema change. Validator re-run clean after four follow-up edits to keep the SKILL.md frontmatter description under the length limit and phrase three floating-tag-adjacent lines (do not use floating tags in production) inside the validator's allow-list window.

Earlier rounds

Rounds 4 and earlier have been moved to QUALITY-REVIEW-archive.md to keep this file focused on recent review activity.

Four parallel reviewers re-scored the Round 3 angles against the patched skill. Three angles cleared the 8/10 threshold; one returned with concrete must-fixes.

Round 4 must-fix findings (novice onboarding, all closed in 2.0.4)

• 5-minute Hello forced a forward jump to Step 0 of the stitched example to obtain a digest. Fixed: bundles/examples/guide.md Hello block now resolves the digest inline with docker pull + docker inspect --format, pinning $DIGEST and reusing it in the same shell.
• jq dependency assumed but never called out in the Hello block. Fixed: prerequisites paragraph now states jq is not required and points at the equivalent jq-based form in the stitched example as functionally equivalent.
• 90-minute starter path placed AFTER the Day-0 workflow in SKILL.md, so a top-to-bottom novice hit twelve non-negotiable rules and twelve Day-0 steps before seeing the starter on-ramp. Fixed: section relocated to immediately after ## Prerequisites by runtime (still before ## Non-negotiable rules), and a "New to Drasi?" pointer line added near the top of SKILL.md. Drasi-lib starter also gained an explicit "if you just want to see it run, pin 0.3.8 and proceed" escape hatch.

Round 4 outcome

Only bundles/examples/guide.md content changed; examples bundle bumped from 2.0.3 to 2.0.4 in bundles/catalog.yaml. SKILL.md edits are router-level and do not bump bundle versions. No catalog schema change. Validator clean after one follow-up edit to inline the "do not use the floating latest tag in production" allow-list phrase near the digest-resolution commands.

Round 5 - 2026-05-15 (final novice re-score)

Single targeted re-score on novice onboarding only. Score returned 8/10, threshold met, zero must-fix items. One nice-to-have surfaced a real sequencing bug (Hello Step 1 referenced hello.yaml before Step 2 created it); fixed in place along with a one-line cleanup hint (Ctrl-C + --rm semantics) and a $PWD-into-/c cwd reminder.

Stopping condition

All 12 reviewer angles have been re-scored ≥ 8/10 after fixes; zero open must-fix items; ≥ 8 unique angles used (12); validator passing. The internal review-and-fix loop is complete. External-validation passes (GitHub source, docs.drasi.io, Community) follow in Round 6 before the final zip is emitted.

Round 6 - 2026-05-15 (external validation)

Three independent external validators ran in parallel, each grounded in a different source-of-truth, against the patched 2.0.4 skill.

Validator results

Must-fix findings (all closed in 2.0.5)

• Validator R: /api/v1/health does not exist (bundles/developer-experience/guide.md, line 42). The drasi-server upstream README and every other skill file use root-level /health. Fixed: changed GET /api/v1/health → GET /health (Pack M).
• Validator R: drasi-lib repo discovery gap. The crate is published from github.com/drasi-project/drasi-core; there is no standalone drasi-lib GitHub repo. Fixed: one-line clarifications added to references/current-sources.md (Pack M) and a new ## Source repository subsection in bundles/drasi-lib/guide.md (Pack N).
• Validators R + S: "no tagged GitHub releases" wording vs :0.1.0 Docker tag. Releases page is still empty (Validator R confirmed 2026-05-15) but the upstream README's docker-compose default now references ghcr.io/drasi-project/drasi-server:0.1.0. Fixed: reconciled wording in SKILL.md Current verification snapshot and references/current-sources.md - a 0.1.0 Docker tag may exist on GHCR without an accompanying GitHub Release artifact; production must still resolve to an immutable @sha256: digest before pinning (Pack M).
• Validators S + T: drasi-lib version-conflict framing too weak. The official Drasi AI context lists 0.3.8; crates.io/docs.rs now publishes 0.4.2 (built 2026-04-20); drasi-core is mid-rewrite for "Replayable Sources / Resumable Reactions" (issues #345, #369, #392, #393, #394). Fixed: reframed as a deliberate three-way pinning decision in SKILL.md, references/current-sources.md, bundles/drasi-lib/guide.md (named trait-surface changes), and bundles/currency-and-ai-context/guide.md (Packs M and N).
• Validator T: Drasi MCP Reaction is two MCP spec revisions behind upstream. Drasi pins 2025-03-26; upstream MCP has shipped 2025-06-18 (Stable; OAuth Resource Servers + RFC 8707 Resource Indicators) and 2025-11-25 (Latest Stable; OIDC Discovery, icons, incremental consent, elicitation). Fixed: explicit MCP spec lag callout added to bundles/agent-integration/guide.md and a snapshot bullet added to SKILL.md (Pack M).
• Validator T: three real-world failure modes missing from operations taxonomy. (1) Persistent-index inconsistency after crash on pre-#290 platforms (drasi-core #290 closed 2026-02 but pre-fix deployments still exposed); (2) aggregation phantom rows after groups empty out (drasi-core #384, still open); (3) plugin directory listing pagination cap at ~50 plugins (drasi-core #414, still open). Fixed: three rows added to the failure-mode taxonomy in bundles/operations/guide.md with playbook stubs (Pack O).
• Validator T: PostgreSQL 19 dynamic-effective wal_level cluster- wide side effect not named. On PG19+ a Drasi PostgreSQL Source creating a logical slot on wal_level = replica auto-promotes the effective level to logical cluster-wide, increasing WAL volume system-wide. Fixed: callout added to the replication-slot runbook in bundles/sources/guide.md (Pack O).
• Validator T: WAL accumulation as outage class without an alert query. The runbook named slot leakage but did not name "WAL accumulation" (slot retains WAL → disk fills → DB write outage) as the resulting outage class. Fixed: explicit alert on pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn) per slot added to bundles/sources/guide.md (warn > 1 GiB, page > 10 GiB) (Pack O).

Intentionally not applied

• Cross-cdylib allocator warning (drasi-core #378): niche to custom-plugin authors; deferred until a custom-plugin authoring bundle exists.
• EKS storage-class fragility (HN 2024): already implicitly covered by existing non-AKS hedging; no current community post confirms a specific pre-1.0 EKS failure mode worth a dedicated runbook.
• Dapr bridge GSoC 2026 (drasi-platform #383): proposed for summer 2026 but unshipped; revisit when a release lands.

Round 6 outcome

Three file-isolated fix packs (M: SKILL.md + references + agent-integration + developer-experience; N: drasi-lib + currency-and- ai-context; O: operations + sources) plus Pack P (catalog metadata + CHANGELOG + this section). No file was touched by more than one pack.

Six bundles bumped in bundles/catalog.yaml: agent-integration 2.0.2→2.0.3, developer-experience 2.0.3→2.0.4, drasi-lib 2.0.0→2.0.1, currency-and-ai-context 2.0.2→2.0.3, operations 2.0.3→2.0.4, sources 2.0.2→2.0.3. SKILL.md and references/current-sources.md are router/reference-level - their edits did not bump a bundle version. Skill package version 2.0.4 → 2.0.5. No catalog schema change. Validator clean after each pack.

Round 6 final stopping condition (superseded by Round 7)

All 12 internal reviewer angles ≥ 8/10; all three external validators returned pass-with-notes; every external must-fix is closed; zero open must-fix items; validator passing. Ready for final packaging.

Round 7 - 2026-05-15 (mixed-runtime coherence)

A continuation pass extended Round 6's external pass-with-notes posture with a fourth currency re-check and three additional internal reviewer angles focused on the gap most likely to break a real-world deployment that spans more than one Drasi runtime form. Round 6 already cleared single-runtime coverage, but mixed-runtime projects (drasi-lib edge → Drasi Server hub, or Drasi for Kubernetes core + drasi-lib worker) had implicit guidance scattered across bundles without an explicit set of cross-runtime constraints.

Reviewer angles used

Scores

Must-fix findings (all closed in 2.0.6)

• Reviewer W: no router-level statement of mixed-runtime patterns. SKILL.md named "Runtime decision gate" but treated runtime selection as exclusive. Real projects sometimes span runtimes deliberately. Fixed: added ## Mixed-runtime projects section to SKILL.md with three supported composition patterns and four non-negotiable cross-runtime constraints.
• Reviewer W: templates/drasi-version-pinning.md had no place to record cross-runtime version coherence. A mixed-runtime project needs a single versions.md, but the template did not name the coherence anchors (drasi-core version line, apiVersion, MCP spec, shared file location). Fixed: added ## Cross-runtime compatibility section with anchors table and CI gate.
• Reviewer W: ContinuousQuery portability across runtimes unaddressed. A query authored on drasi-lib might or might not run unchanged on Drasi Server or Drasi for Kubernetes; the skill did not name the runtime-neutral feature subset. Fixed: added ## Cross-runtime portability section to bundles/continuous-queries/guide.md with portability matrix and anti-patterns table.
• Reviewer W: per-runtime SLOs do not compose. The observability bundle's SLO baselines were per-runtime; mixed-runtime projects cannot derive an end-to-end budget by adding rows. Fixed: added ## Cross-runtime end-to-end signals to bundles/observability/guide.md with traceparent propagation rule and end-to-end freshness budget decomposition.
• Reviewer W: recovery and upgrade order undefined across a runtime boundary. The recovery bundle covered per-runtime recovery but not sequencing when both legs of a mixed project need attention. Fixed: added ## Cross-runtime recovery and upgrade sequencing to bundles/recovery/guide.md with five-step sequence rule and end-to-end RTO/RPO table.
• Reviewer V: telemetry-cost ceilings not named. Drasi cost optimisation surfaces (logs, metrics, traces, canary, probes) had no explicit cap or sampling guidance. Fixed: added ## Telemetry cost controls to bundles/observability/guide.md with cost-driver table and four enforcement steps.
• Reviewer V: ACA plan choice for Drasi Server not named. Azure hosting bundle did not say which ACA plan to pick for production. Fixed: added ## Container Apps plan choice for Drasi Server to bundles/azure-hosting/guide.md (Dedicated/Workload Profiles for production; Consumption only for dev/test with minReplicas: 1).
• Reviewer V: per-component sizing inputs and node-pool placement not explicit. Scaling bundle had an autoscaling-trigger matrix but no concrete sizing-inputs template or spot-eligibility rules. Fixed: added ## Per-component sizing template, ## Storage budgeting, and ## Node pool placement to bundles/scaling-and-capacity/guide.md.
• Reviewer U: SKILL.md and references/current-sources.md showed Checked: 2026-05-11 four days after Round 6's 2026-05-15 external re-verification. No factual deltas - just the date string. Fixed: bumped both to Checked: 2026-05-15.

Round 7 outcome

Seven file-isolated fix packs (one per touched file) applied the must-fixes. No file was touched by more than one pack.

Five bundles bumped in bundles/catalog.yaml: azure-hosting 2.0.1 → 2.0.2, continuous-queries 2.0.1 → 2.0.2, observability 2.0.3 → 2.0.4, recovery 2.0.2 → 2.0.3, scaling-and-capacity 2.0.2 → 2.0.3. SKILL.md, references/current-sources.md, and templates/drasi-version-pinning.md are router/reference/template- level - their edits did not bump any bundle version. Skill package version 2.0.5 → 2.0.6. No catalog schema change. Validator clean after each pack.

Final stopping condition (superseded by Round 8)

All 15 internal reviewer angles ≥ 8/10; Round 7 mixed-runtime coherence pass closed every cross-runtime gap identified by Reviewer W; currency re-checked and dated 2026-05-15 across SKILL .md and references/current-sources.md; zero open must-fix items; validator passing. Package ready for final 2.0.6 zip.

Round 8 - 2026-05-15 (repo-grounded custom-plugin and Cypher-subset coverage)

Round 8 was driven by an explicit user request to "use resources from https://github.com/orgs/drasi-project/repositories so search and scan the repos, and also add knowledge for creating custom sources and reactions as well as expanding the query language". Prior rounds had covered operations, security, scaling, and recovery thoroughly, but two coverage gaps remained: (1) authoring custom Sources, Reactions, and query-language extensions against the actual drasi-plugin-sdk trait surface in drasi-core, and (2) a parseable Cypher/GQL feature matrix rather than the hand-wavy "Cypher subset" framing in prior rounds.

Reviewer angles used

This is the 16th unique internal reviewer angle (cumulative across rounds 1–8); the 15 angles from rounds 1–7 are not re-scored here.

Scores

Must-fix findings (all closed in 2.0.7)

• Custom Source / Reaction authoring entirely absent (Angle Y). No bundle, template, or evidence list covered how to write a custom Source or Reaction against the drasi-plugin-sdk trait surface. The Round 6 "intentionally not applied" list explicitly deferred this. Fixed: created new bundle bundles/custom-plugins/guide.md (594 lines) grounded in the drasi-plugin-sdk 0.4.2 + drasi-lib 0.4.2 + drasi-query-ast crates, drasi-server/docs/plugin-architecture.md, the drasi-core source tree, and DeepWiki. The bundle covers: the two trait surfaces per plugin (drasi_lib::sources::Source runtime trait + SourcePluginDescriptor factory trait; same split for Reactions via Reaction + ReactionPluginDescriptor); the SourceChange / Element / ElementMetadata / ElementReference types from drasi-query-ast with a worked SourceChange::Insert example; the full SourcePluginDescriptor impl and ConfigValue<T> DTO contract with DtoMapper; the symmetric Reaction runtime trait; a 6-row delivery-semantics table (at-least-once, per-query ordering, cross-query ordering, backpressure modes, DLQ, replay); packaging per runtime (drasi-lib static + Drasi Server cdylib via export_plugin!
  • Drasi for Kubernetes); the 6-step cdylib loader lifecycle (dlopen → metadata → SDK+triple validation → init → vtable proxy); the cross-cdylib Box ownership pitfall (drasi-core issue #378) with five mitigations; the async runtime contract; identity wiring via the application-identity provider (drasi-core PR #423); a 10-row built-in plugin templates index; the query-language extension points (add a scalar function, the full AggregatingFunction trait + lifecycle for adding aggregating functions, add an operator, add a clause) plus an 8-row decision matrix for fork-versus-extension-point; the Replayable Sources / Resumable Reactions status (issues #345,

  #369, #392, #393, #394; WAL plugin #346/#362); a 4-row test- pattern table; ten non-negotiable rules for plugin authors; and a ## Acceptance evidence for a new plugin heading satisfying the validator.

• No parseable Cypher / GQL feature matrix (Angle Y). Round 1 added a join-cardinality worked example to bundles/continuous-queries/guide.md, but the bundle still framed "Cypher subset" as prose without enumerating the AST, clauses, predicates, aggregators, or the drasi.* namespace against the actual drasi-query-ast/src/ast.rs and the functions-cypher / functions-gql registries. Third-party "unsupported features" lists contradict the repo on collect(). Fixed: added ## Cypher and GQL feature support to bundles/continuous-queries/guide.md with the parseable AST surface (11 nodes from drasi-query-ast/src/ast.rs), the clauses table (MATCH/WHERE/RETURN/WITH supported; OPTIONAL MATCH, UNWIND, CALL, UNION hedged; CREATE/MERGE/SET/DELETE not applicable), the predicates and operators table, the pattern- syntax notes including elementId (Cypher) vs element_id (GQL), the 8-row aggregating-functions table (sum, avg, count, min, max, collect, linearGradient, AggregatingLast) with accumulator types, the drasi.* namespace table (8 temporal functions), GQL mode differences (separate functions-gql registry - issue #217), and a known-unsupported table that flags DISTINCT / ORDER BY / LIMIT as hedged unsupported and notes third-party "unsupported features" lists contradict the repo on collect().
• Router and currency-reference wiring missing for new content (Angle Y). New material is invisible if SKILL.md, bundles/catalog.yaml, references/current-sources.md, and CHANGELOG.md do not reflect it. Fixed:
  • SKILL.md: added a custom-plugins row to the bundle routing table after the drasi-lib row.
  • bundles/catalog.yaml: new custom-plugins entry at version: 2.0.0 with capabilities [custom-source, custom-reaction, plugin-sdk, query-extension, drasi-core] and seven user-intent keywords; continuous-queries bumped 2.0.2 → 2.0.3 and its purpose: line extended to mention the parseable Cypher/GQL subset, the eight built-in aggregators, and the drasi.* temporal namespace.
  • references/current-sources.md: new ## Custom plugins and engine internals section naming the drasi-plugin-sdk 0.4.2 surface, the drasi-core layout, the drasi-server/docs/plugin-architecture.md canonical loader description, the drasi.* namespace docs path, and the seven active-rewrite issue numbers (#298, #349, #346/#362, #345/#369/#392/#393/#394, #378, #217, #423); new ## Custom plugin URLs section with the 8 source URLs.
  • CHANGELOG.md: prepended 2.0.7 entry documenting Added / Changed / Versions / Reviewer-angles.

Round 8 outcome

Two file-isolated authoring packs landed the content (bundles/custom-plugins/guide.md + bundle.yaml; bundles/continuous-queries/guide.md feature-matrix section) plus four metadata packs (SKILL.md router row, bundles/catalog.yaml new entry + version bump, references/current-sources.md two new sections, CHANGELOG.md 2.0.7 entry). No file was touched by more than one pack.

Bundle changes in bundles/catalog.yaml:

• custom-plugins - new entry at 2.0.0.
• continuous-queries 2.0.2 → 2.0.3 (Cypher/GQL feature matrix added; purpose line extended).

Skill package version 2.0.6 → 2.0.7. No catalog schema change. Validator clean.

Final stopping condition

All 16 internal reviewer angles ≥ 8/10 (the 15 from rounds 1–7 are unchanged and not re-scored; Round 8 added the 16th angle and closed it at 9/10); all three external validators from Round 6 still hold (no factual deltas vs Round 6 / 7; re-check date still 2026-05-15); Round 8 closed both gaps called out in the user's explicit request (custom Source / Reaction authoring + query-language extension; parseable Cypher / GQL feature matrix); zero open must-fix items; validator passing. Package ready for final 2.0.7 zip.