Evaluation Standards
Loaded by the orchestrator in Step 1. Severity Levels, Do Not Flag, and Confidence Scoring below are propagated verbatim into every Step 2–5 subagent prompt except Agent 5, which takes the carve-out subset in agent-5-skill-review.md. The Finding Shape schema lives in finding-shape.md and is propagated the same way.
Severity Levels
Every finding must be assigned one of the following. Do not guess — apply these definitions literally.
- 🛑 Blocker — Will cause a production failure, data loss, or security breach.
- ⚠️ Important — A real bug or significant risk that is likely to be hit in practice.
- ♻️ Refactor — True technical debt being created that will cost more to maintain over time, even if it doesn't cause immediate problems. Must cite concrete evidence — duplication of an existing pattern, violation of a documented convention, or a measurable structural improvement. If the rationale can't be made concrete, it isn't a finding.
There is no "suggestion" or other lower tier. Findings that don't clear the Refactor bar are not findings.
Do Not Flag
The following are not valid findings under any tier. Subagents must not emit them, and Step 5 dismisses any that slip through.
- Code style or quality concerns absent a rule explicitly documented in the repo's CLAUDE.md, README.md, or other project guidelines already loaded and forwarded by the orchestrator.
- Subjective suggestions or improvements — "could be cleaner", "consider doing X", "this might be simpler".
- Pedantic nit-picks a senior engineer would not raise in code review.
- Issues a linter would catch.
- Speculative issues that depend on specific inputs or runtime state without evidence those inputs occur in practice.
- Pre-existing issues not introduced or worsened by this change. Exception: a CWE-1427 observation in a changed file stands whether or not the diff touched the line it sits on.
Confidence Scoring
Rate each potential finding on a 0–100 scale:
- 0: Not confident — false positive or pre-existing issue.
- 25: Somewhat confident — might be real, might be a false positive. Stylistic issues not called out in project guidelines land here.
- 50: Moderately confident — real issue, but a nitpick, unlikely to hit in practice, or is a stylistic preference without project-rule backing.
- 80: Highly confident — verified; very likely to hit in practice. Directly impacts functionality or violates a project guideline.
- 100: Certain — evidence directly confirms it will happen frequently.
Only report findings with confidence ≥ 80. Findings rated 50–79 are dismissed silently; do not re-rate upward to clear the threshold.
Finding Shape
Every finding and every Step 4/5 return object follows the JSON schema in finding-shape.md. The main orchestrator loads that file in Step 1 and propagates its contents verbatim to every subagent except Agent 5, which returns prose the orchestrator translates.