Index of every page on this site: llms.txt
All skills
bitwarden avatar

/verifying-doc-parity

@a11c77b official
by bitwardenbitwarden/ai-plugins165 stars
27
Run it yourself

This skill should be used when the user mentions doc drift, documentation verification, README or docs/ updates that follow a code change, or a doc-parity Stop hook block — even if the request does not name a skill or documentation explicitly. Also runs as the documentation pass of a pull request review. Triggered by phrasings such as "verify doc parity", "are my docs up to date", "did I miss any doc updates", "what documentation should this change touch", "do I need to update any docs for my current changes", or "check if the docs still match the code".

Use this Skill: https://skilld.dev/gh/bitwarden/ai-plugins/verifying-doc-parity

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

evalsREADME.md

≈960 tokens on demand. Your agent reads this file only when SKILL.md points to it.

verifying-doc-parity evals

Behavior test cases for the verifying-doc-parity skill, in the skill-creator schema.

behavior-eval.json holds cases exercising what happens when the skill fires. Covers both primary success paths (session and pull-request review), every outcome the skill claims to produce (Update, Attest, Dismiss), and each recently-added instruction (below-component source-embedded surfaces, placement follows what the doc describes, root-level placement for a repo-wide capability, the out-of-ancestor-chain restatement search, and the condensed Step 4 report shapes).

behavior-baseline.json is the recorded benchmark that intentional skill changes must not regress silently.

Do not add trigger evals for this skill

It was found that the run_eval.py's methodology incorrectly failed to detect skill triggers due to requiring immediate skill execution rather than context exploration, which models almost always do regardless of description changes.

This skill's load-bearing invocation paths are the doc-parity Stop hook and the ai-review workflow's documentation pass. Natural-language triggering is a nice-to-have fallback, not the critical path.

Eval cases

Behavior cases run against the Seeder subsystem of bitwarden/server (util/Seeder/, util/SeederApi/, util/SeederUtility/), chosen because its code-to-docs mapping is known.

These evals are executed against a pinned server ref, d6c84a7562cc6b464de910dbf829690885500137. To increase stability in testing, evaluate against that reference point.

Each case is constructed so a run that skips a documented scope during the walk, token-edits its way past the gate, or emits a report shape that mishandles the outcome type (Update list, one-line Attest summary, or one-line Dismiss) fails a named expectation. Behavior cases assume the run happens inside a bitwarden/server checkout so the ground-truth docs are readable.

Case 10 exercises the review context (out-of-repo discovery against the contributing-docs corpus); all other cases are session cases. Cases involve live edits to a scratch checkout, so run them against a disposable clone or a git worktree, never a working tree you care about.

Each case's expectations array is the pass criterion — every expectation is graded independently. Denominators differ per case because they count expectations, not runs.

Running

Run with /skill-creator:skill-creator in benchmark mode (with-skill vs. without-skill) with a config-blind grader. Install the runner from the claude-plugins-official marketplace if it is not already present:

/plugin install skill-creator@claude-plugins-official

The AI Review Guidelines want at least three iterations per case per configuration so pass rate and variance are both captured.

Regression check against the current baseline compares the aggregate summary and per-run pass rates; timestamps, models, and per-run evidence prose vary on every execution and are not signal:

proj='{run_summary, runs: [.runs[] | {eval_id, configuration, run_number, pass_rate: .result.pass_rate}]}'
diff <(jq -S "$proj" behavior-baseline.json) <(jq -S "$proj" result.json)

An empty diff on that projection means no regression on the graded data. When a change is intentional and the new numbers are the new desired state, replace the baseline with the new results in the same PR as the skill change.

Updating the test surface

Update behavior-eval.json when the test surface needs to evolve: a new load-bearing rule in the skill, a Seeder doc restructure that invalidates a case's ground truth, or an expectation that turned out to be ambiguous. The eval set encodes intent, not implementation — when a run fails, fix the skill, not the case, unless the case itself is wrong.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warning28d3 checks · Risk SAFE
  • Gen Agent Trust Hub28d

    The skill 'verifying-doc-parity' is designed to maintain documentation accuracy by checking for drift against code changes. It operates by analyzing git diffs and searching for relevant external documentation. The skill's primary risk is an indirect prompt injection surface, as it ingests untrusted data from pull requests and external websites while possessing file-editing capabilities.

  • Socket28d

    No alerts

  • Snyk28d

    Risk: MEDIUM · 1 issue

Signed by skilld at a11c77b. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 days ago
What it can do
Runs commands Reads files Edits files Network
agent
general-purpose
context
fork
background
false
All 8 allowed tools
Bash(git diff:*)Bash(git ls-files:*)ReadGrepGlobEditWebSearchWebFetch(domain:contributing.bitwarden.com)

README badge

README badge for bitwarden/ai-plugins/verifying-doc-parity