persistent-review specification
Dependencies: scode-harness-shellout
This file records the behavioral contract for the installed skill. Launch and transcript-access mechanics belong to the declared dependency; review scope, evidence judgment, persistence, and completion belong here.
Activation and inputs
- Explicit invocation and a request for persistent adversarial review discover this protocol. Discussion, assessment, or editing of the skill does not launch a reviewer. Loading creates no unrelated session-wide policy, and ordinary review requests do not silently become persistent.
- Reviewers follow the caller's charter, precise scope, model, effort, result destination, and relevant constraints. Session context may settle inputs; only unresolved choices that affect the task require clarification. The skill supplies no review lenses, panel, finding taxonomy, automatic-fix policy, or VCS range.
- Reviewed inputs stay read-only unless edits are separately authorized; requested results and private run artifacts may be written. Findings do not authorize fixes.
- Reviewers do not load this skill. Each receives a self-contained brief file with the charter, group scope and
interfaces, constraints, the caller's finding format, absolute
evidence read/recordcommands, the finding-event format and reviewer-facing read rules, its own log and progress paths, and the persistence and investigation requirements. Same-session continuations carry only that turn's new material, the immediate-recording instruction, and the brief's path with an instruction to reread it after a compaction of the reviewer's context; the brief is not pasted again. Caller-required reading besides source is followed, and its per-reviewer cost is stated in the launch plan because it scales with group count. - Every initial and continuation prompt directs reviewers to validate findings through source and contract analysis. Reviewers and the orchestrator do not reproduce bugs or write reproductions unless the caller explicitly requests that work. Existing tests and builds remain allowed; execution is not required to substantiate a finding. Reports distinguish source-supported conclusions from behavior observed during execution.
- Codex and Claude reviewers are supported. Other reviewer backends are reported unsupported without substitution. Installation remains shared across all seven caller harnesses.
Stable scope and evidence
All helper commands use the bundled POSIX
evidenceandschedulelaunchers. Each prefers a matching compiled executable and permits Deno source fallback only when that executable is absent. Stale, missing-source, broken, or nonexecutable compiled deployments fail without running the command through source as a fallback. Compiled helpers embed their source hash and check the neighboringevidence.tsorschedule.tsat startup. This is a freshness check, not a lock against later edits.Before launch the orchestrator establishes a mandatory inventory of stable text and ranges. Full files are required only when agreed. Discretionary exploration does not expand the inventory. Diff scope preserves its agreed old/new content. Changed inputs require retained originals or an explicit inventory reconciliation and affected-range audit.
Run artifacts have unique session-private paths. Each reviewer owns a manifest, append-only findings log, progress file, session identity, transcript paths and coverage reports. Stable snapshots and original revision/range mappings remain available behind compact references.
The bundled Deno helper verifies complete read packets in actual delivered tool results against pinned UTF-8 source text, including exact range content and source version. It validates reviewer session/cwd and call/result pairing. It emits compact coverage and evidence references without echoing source or reasoning. Codex's persisted delivered results and Claude's verbose stream are supported; raw Codex command stdout is never substituted.
The audit must not break when a harness adds a command, tool, record type, or content shape. It aborts only when continuing could misattribute identity or certify text that does not match pinned source; everything else degrades to no credit plus a report entry. No allowlist of record, tool, family, or block types may decide whether it runs.
Only tool results can earn coverage: Codex
response_itempayloads with a string call ID (any family; results end in_output) and Claudetool_use/tool_resultblocks. A result needs an earlier call with its ID in the supplied logs; Codex calls may have several results. Result text comes from a string or the text of array blocks. Codex results over 40,000 bytes earn nothing, because Codex delivers about 12,000 tokens without marking the cut.The report keeps unusable material visible: packet-bearing results that earned nothing in
rejectedwith a reason, calls without results inunfinished_calls, skipped kinds known to carry no delivery inignored, and any other skipped kind inunrecognized. Skipped records never earn coverage, even when they quote packets, and compaction neither erases original delivery nor proves missing text. The orchestrator judges whetherunrecognizedentries may hide reads and, if so, treats them as a format failure.Skipping does not weaken identity. Each Codex log has exactly one matching
session_metabefore other records; native subagent rollouts fail with a diagnostic that names them as unsupported. Every Claude record that names a session names the expected one, and each log has a matchingsystem/initbefore its first message; repeated inits must also match.Structurally valid read packets for paths outside the audited manifest are reported explicitly with claimed ranges and raw references. They are unverified context and earn no credit, do not load additional sources, and never expand mandatory scope. Malformed required fields still fail; missing required ranges remain missing.
Audit comparison uses packet meaning: path, digest, inclusive range and exact decoded text. JSON key order, additive metadata and hexadecimal digest case do not change delivery evidence. Blank JSONL separators are ignored while physical line references are preserved. Whitespace-only required logs still fail as empty.
The read producer retains its 200-line/24,000-byte budgets. Those defaults do not invalidate a larger complete delivered packet whose required source identity, bounds and text can be verified. Audit memory is still affected by the largest JSONL record and retained snapshots, independent of producer defaults.
Malformed log lines, mixed sessions or working directories, missing initialization, changed sources and malformed or mismatched packets fail visibly with a nonzero exit and no successful partial report. Valid clipped packets earn no coverage. Missing packets or ranges remain incomplete. Known failed reads are retried with smaller packets; unfinished logs may be retried after their writer reaches a boundary. Other format failures require a user decision before direct inspection, with an explicit warning about loading duplicate code and spending more tokens. No silent fallback.
Attempted reads, process success, file lists, search hits, tool counts, checksums, reviewer assertions, or raw output larger than what the model received cannot establish delivery of required ranges. Relevant truncation metadata and omission markers are interpreted using verified harness semantics. Compact references retain the route to raw evidence. Matched packet text, not packet metadata alone, establishes delivery. The helper is not a general transcript framework and establishes neither review quality nor attention.
Functional groups and findings
The protocol governs review groups, which deliver a mandatory inventory of reviewed text. A focused task judging named claims from the review's own output (verifying a finding, adjudicating a proposed exclusion) with only the excerpts those claims need is not a group and inherits no persistence, delivery audit, synthesis or discovery requirement unless the caller explicitly asks for one. Work asked to search for new defects or assigned a range to cover is a group whatever it is called, and a group launched as a review group stays one. Focused-task verdicts are recorded durably on return; a new defect it reports goes to the owning group's reviewer as cross-group input.
The orchestrator divides required scope by coherent behavior, subsystem, feature path, or lifecycle. Required contracts, callers and tests stay with the relevant behavior. Group notes name purposes and cross-group interfaces; arbitrary file counts or directory order are not partitioning rules. Every mandatory range is assigned; context may overlap. Defaults cap groups at 2,000 source lines and 64 KiB including required context; larger behaviors split at meaningful sub-boundaries, with caller approval required to exceed the ceilings. Groups are not padded to the ceilings, and one behavior is not split below them.
Each group costs at least three turns, setup reading, and an audit and parent decision per turn, independent of size. Adjacent phases of one behavior stay together; many small self-contained changes may share a group when the notes list each with its purpose. Before launch, an interim message reports the plan: group count against the minimum the ceilings imply (total mandatory lines / 2,000), mandatory-line distribution, minimum turns, concurrency and per-reviewer reading besides source, with a reason when the count is well above that minimum. Launch does not wait for a reply unless the caller asked to approve the plan.
Independent ready groups run concurrently by default, up to actual harness/resource capacity, with explicit queued states for the remainder. Caller concurrency restrictions prevail. Each group has its own resumable reviewer and private writable artifacts; shared input snapshots stay read-only. Reviewers cannot consume other groups on their own.
Every reviewer turn runs through the bundled
schedulehelper, a separate tool fromevidence. It owns only the queue of ready turns, the concurrency limit (adjustable while running), and one live turn per identity. It runs an orchestrator-written launch script per turn, records launch, exit status and the findings-log line counts at start and end, and parks the identity awaiting a parent decision. It never builds launch commands, reads transcripts or findings content, audits delivery, or queues a next turn; only the orchestrator'sreadyafter judging a turn does, andcanceltakes back a turn that has not started. Its state is one atomically replaced file guarded by an exclusive lock, and a launch is committed to it before the process exists, so a crash never leaves a started turn recorded as queued. Each turn runs in its own process group under a wrapper that records its pid and, last, the script's exit status; a turn ends when that exit file exists and islostonly when its group is gone with no exit status after a second look, or it never started within a grace period. A restarted daemon recovers turns its predecessor launched, and neither stopping the daemon nor signalling its process group kills turns. A nonblocking lifetime lock refuses a second daemon even before the owner publishes its heartbeat or if that heartbeat becomes stale. Heartbeat updates replace the file atomically so readers never mistake a partial write for a dead daemon. The journal thatwaitfollows is repaired from state after a crash, and readers ignore a partial last line.waitblocks until a turn ends and exits instead of hanging when no daemon is alive; daemon liveness needs a fresh heartbeat and a live pid, never command-line matching.statusgives the journal position, counts by state, free slots, awaiting identities oldest first with exit, a compact audit summary and findings offsets, and running turns with pids and output files. Unknown CLI options are refused. Terminal states arecompleteandstopped-incomplete, set by the orchestrator between turns.Groups advance independently: synthesis follows a group's own audit, and discovery follows its synthesis without a run-wide phase barrier. A group's discovery waits for synthesis, or an incomplete stop, in the interface groups recorded in its group notes, and for nothing else. Each group is tracked as queued, running, awaiting a parent decision, complete, or stopped incomplete.
Every initial and continuation prompt instructs immediate recording: once a substantive finding, a decision not to report something considered, or an unsettled lead is ready, append it as a
finding,candidateorleadbefore further analysis or reads, never waiting for a checkpoint, batch end, turn end, final report, or compaction. Findings and progress artifacts are initialized before source reads. No finding quota is imposed. Recording a finding does not end the assigned pass; reviewers continue that pass instead of stopping at a count.Reviewer prompts tell reviewers to read source themselves, not through subagents, whose reads reach only the subagent and earn no coverage. Subagents remain allowed for non-reading work such as running tests or builds.
Versioned reviewer events carry a stable local ID, kind, open/withdrawn state, summary, original location and complete detail in the caller's format. Kinds are
finding,candidate(considered and not reported, with the reason and cited text), andlead(unresolved), sharing one ID space; an ID may change kind, and the latest event wins. A lead closes by changing kind tofindingorcandidate;withdrawnretracts a record itself. Revisions/retractions append under the same ID and preserve history. The log, not reply prose, is the reviewer's complete record; the discovery prompt asks for records and their IDs rather than prose lists.First replies follow the caller's format; later replies report only that turn's changes (checks and results, IDs appended or revised, IDs of leads still open, and for discovery an explicit statement of whether anything was new) and reference unchanged records by ID. The brief carries this reply contract. A caller format requiring cumulative replies is honored, with its cost in the launch plan.
The orchestrator polls finding logs and progress without reading transcripts. The helper counts latest states per reviewer log, keeping reported findings (counts and list) separate from open candidates and leads, preserving source ownership and distinguishing empty logs from missing/malformed ones. Polling during an unfinished append may be retried; counts are unavailable until parsing succeeds. Raw counts remain distinct from semantic deduplication by underlying defect/location. Original reviewer IDs and evidence survive deduplication.
After each turn the orchestrator reads the events appended since that reviewer's previous offset plus the delta reply, not whole earlier replies. An offset past the end of a log is refused. A reply mentioning a new or revised record, or a closed lead, without a matching event in that range requires a continuation asking the reviewer to record it. Reviewer offsets and the disposition log path live in the current-state file.
The orchestrator creates an append-only disposition log at run start, registers every reviewer log in it, and records each disposition there, naming the reviewer log, ID, judged physical line, disposition (
verified,unverified-after-investigation,under-investigation,duplicate,rejected,excluded), deduplication group and reason. Recording refuses an unregistered log or a judged line holding no event for that ID. Consolidation requires exactly the registered logs, refuses decisions naming unknown records, derives each group's disposition from its non-duplicate members (reportingstale,conflictandduplicate-onlyinstead of resolving them), and lists undecided open records, stale decisions on later-revised records, open leads, and withdrawn records. The consolidated draft is kept current as decisions are made and stays private until the caller's recording checks.Group completion does not prove cross-group correctness. The orchestrator deduplicates as results arrive, reconciles conflicts, and sends concrete interface questions back to the relevant reviewers for synthesis, batched into each reviewer's next continuation. An incomplete group keeps the overall run incomplete. New evidence or findings go to every group whose behavior or interfaces they bear on, and count as pending cross-group input until sent. Input reaching a group after its discovery reopens it; one continuation may synthesize the input and search again. An earlier empty pass does not close later questions. The run completes only when quiescent: every group complete, no cross-group input pending, and each group's last discovery after the last cross-group input sent to it.
Persistent execution
- Each group's completion requires an initial turn, later-turn synthesis after delivery, and a subsequent discovery search on the same session identity: at least three actual turns. Between turns the orchestrator receives a result, audits relevant evidence, and sends a concrete continuation. Multiple sweeps inside one turn do not satisfy this.
- Findings never excuse missing scope, and an empty first result still requires continuation. Missing or uncertain ranges receive named, targeted same-session repairs independent of finding count. Large scopes create more groups.
- Synthesis occurs later than the initial review and after required content delivery, revisiting interactions and unresolved assumptions under the charter. Final repair and synthesis may share a turn. The orchestrator audits that turn afterwards; remaining gaps require repair and updated synthesis subject to the stopping rules below.
- Native delegation is usable only when the parent can audit supported delivered-content transcript evidence and resume the same reviewer session. Otherwise an observable shellout path is considered even if native supports the requested model. Explicit mechanism choices are honored; unavailable evidence or continuation is reported rather than silently changing the mechanism, model, backend, effort, or promised standard.
- Before choosing external evidence, the dependency and relevant Codex or Claude harness file are read through the canonical loading contract. Harness files own launch and evidence-access mechanics. Local deviations require verification of the relevant delivery semantics, not assumptions based on a harness name or JSON availability.
Productive discovery
- Verified coverage and synthesis always lead to a separate same-reviewer search turn, even when no finding or concrete unresolved lead exists. The reviewer reuses context and durable artifacts, receives its own findings and those of its named interface neighbours rather than a run-wide catalog, searches beyond earlier issues under the original charter, records findings immediately, and may legitimately return empty.
- A new item duplicating a finding the reviewer was never shown is a duplicate, not progress; when that finding came from a group outside the recorded interfaces, the interface is added to both groups' notes. A pass whose only new items are confirmed such duplicates counts as reporting no new findings, and its work summary and leads must still meet the stopping condition; a reply that only restates findings it was shown does not.
- The orchestrator verifies new records against pinned source and applicable contracts and checks novelty across the whole run. It accounts for verified, unverified-after-investigation, duplicate, rejected, excluded, and still-open records without erasing source records. New IDs, event counts, restatements, speculation without a concrete source basis, and scope drift do not establish discovery progress. No severity or lens-specific gate is imported; actionability follows the caller's charter.
- Missing premises or open substantive leads receive focused verification and same-reviewer clarification as needed. These dispositions apply to possible findings from every review phase, including initial review and cross-group synthesis. Concrete next checks are pursued while they can establish, refute, or narrow the suspected defect. Rejecting a suspicion does not require proof of its absence. Uninvestigated leads cannot count as completed findings.
- Available evidence includes readable context and permitted tests under the charter, even outside mandatory scope; exploration does not expand mandatory inventory. Relevant accessible evidence and cross-group clarification are pursued before declaring a premise unavailable. The orchestrator assesses investigation records against source and observable check results. Claimed checks without useful results or readily checkable missing premises require a targeted continuation.
- A concrete concern that available checks cannot establish or refute is retained as unverified after investigation once further work would repeat checks or require unavailable evidence. Its finding record and caller-visible result name the source basis, suspected consequence, checks actually performed and results, exact missing premise, why available checks cannot resolve it, and what evidence would settle it. A private progress note or an unverified label without actual investigation is insufficient. Existing claims retain their IDs when revised. Certainty is recorded in the summary/detail using the existing event schema, separately from open/withdrawn lifecycle state.
- The orchestrator decides dispositions and requests any finding/revision from its owning reviewer under the original ID, confirming the append before crediting it as recorded. It never writes the reviewer's log. Cross-group duplicates retain original owners and IDs; conflicting verification claims receive synthesis and a reconciled disposition in the disposition log.
- Retaining an investigated uncertainty settles its review disposition without proving or disproving the suspected bug. It does not make a group or review incomplete or require failure status under this protocol. A generic caller requirement for a complete review does not add a requirement to settle these concerns before advancing a checkpoint; explicitly stricter caller acceptance conditions still apply. It cannot excuse missing mandatory delivery, unusable continuation, unperformed passes, or uninvestigated leads.
- A new verified finding or distinct concrete concern retained as unverified after investigation requires another search continuation on that same session. There is no default pass cap or elapsed-time limit. Searches across independent groups remain concurrent up to actual capacity. Prompts explicitly allow no findings and do not require a quota or automatic scope expansion.
- Known uncertainties do not count as new progress and are reopened only when new evidence offers a useful check. Discovery completes after a properly executed search explicitly reports no new findings of either kind, with a credible short work summary and every substantive lead verified, rejected, deduplicated, or recorded as unverified after investigation. Known findings of either kind may remain. Bare empty replies, refusals, unreadable required input, blocked review passes, outputs that only restate findings the reviewer was shown, and drift cannot establish that condition. Reviewers unable to perform their assigned pass stop incomplete with retained findings and the reason; an investigated uncertainty alone is not such a failure. Reviewers that only repeat findings, produce unsupported claims without useful progress, or leave their charter instead of performing the requested search also stop incomplete. Neither finding quotas nor renewed proof of known uncertainties may be demanded.
- Caller-imposed limits are honored; stopping before the final empty pass or required investigation is disclosed as incomplete discovery. Group progress retains search-pass counts, new finding IDs separated by verification status and empty-pass/failure evidence. An empty pass establishes the operational stopping condition, not proof of exhaustive review.
Continuity and visibility
- Questions about progress, cost or problems, and side requests during a run, are not stop requests. The coordinating turn does not end while work can advance and the harness will not wake the orchestrator on reviewer completion; answers and protocol-required questions go in interim messages where the harness allows, while unaffected groups keep running. The turn ends at completion, an incomplete stop, an explicit caller pause or cancel, or when nothing can advance until the caller answers.
- Progress updates report groups by state, idle capacity, the oldest pending parent decision, completed versus planned minimum turns, and token totals when exposed. Per-turn usage that already includes a resumed session's earlier turns is taken at its latest value, never summed. Cost reporting never licenses ending a progressing review.
- One current-state file in the run directory, rewritten in place before each decision is acted on, holds pending
decisions, pending cross-group input, and run-record paths, pointing at the schedule directory for group states rather
than copying them. Recovery after compaction or resume starts from it and
schedule statusrather than from chronological notes, replies or transcripts.
Stopping and reporting
- The protocol imposes no elapsed-time limit or required time-budget question. Overall and per-turn deadlines must not be invented, including to satisfy delegation or shellout defaults requiring a hard deadline. Monitoring and termination of failed or stuck runs still apply. Honor a time limit only when explicitly supplied by the caller.
- Stop incomplete when transcript access or same-session continuation becomes unusable, or two consecutive repair turns make no verified coverage progress. Progress means delivery of previously missing mandatory content, not new findings. A successful repair resets the counter. Do not replace the reviewer identity to bypass a failed run. Termination uses the selected mechanism's facilities. These conditions apply per group; other progressing groups continue.
- Coverage repair and discovery have separate progress criteria: newly delivered required text and new verified or unverified-after-investigation findings, respectively. Neither criterion substitutes for the other. Empty searches or caller search limits never establish delivery of unread scope.
- Preserve findings on every stop. The result includes completion status and reason, stable scope, backend/model/effort, per-group session identities and actual turn counts, verified delivery references, exact missing or uncertain material, later-turn/cross-group synthesis status and quiescence, raw and deduplicated finding counts, and original finding-log references. Verified and unverified-after-investigation findings and counts are reported separately, with the latter's investigation records. Per-kind counts come from the per-group dispositions consolidation derives after semantic deduplication, never label absence or subtraction from raw active counts. Undecided, stale or conflicting records and open leads mean decisions are still owed. The helper counts every reviewer-open finding, including duplicates, rejected claims and possible findings still under investigation, and does not certify defects. Discovery-pass counts record new findings of each kind per pass and final empty-pass or incomplete stop evidence. Use an accompanying note when the caller's format requires it. A review meeting the protocol may complete with investigated uncertainties; completion and confidence in individual findings remain separate.
- Delivery proves neither attention nor understanding, correctness, or saturation. Hidden chain of thought is neither inspected nor required. Findings stay grounded in reviewed artifacts. No findings plus incomplete evidence is an incomplete review, never a clean completion.