Index of every page on this site: llms.txt
All skills
bitwarden avatar

/consulting-adrs

@a11c77b official
by bitwardenbitwarden/ai-plugins165 stars
27
Run it yourself

Check a design, change, pull request, plan, or threat model against Bitwarden's Architecture Decision Records (ADRs), or locate and summarize the ADR catalog. Use when assessing whether an approach conflicts with, is governed by, or lacks an accepted ADR, including reviewing a PR or diff for alignment with recorded architecture decisions, or when someone needs to find or summarize ADRs. Trigger phrases include "does this conflict with any of our ADRs", "check my PR against our architecture decisions", "is there an accepted ADR governing this", and "summarize our ADRs". Produces structured findings (conflict, gap, stale-reference, aligned, proposed) with cited ADRs, or an ADR summary when that is the ask.

Use this Skill: https://skilld.dev/gh/bitwarden/ai-plugins/consulting-adrs

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

evalsREADME.md

≈749 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Evals: consulting-adrs

Eval set for the consulting-adrs skill, covering the Structure and Behavior assertion categories from Bitwarden's AI Review Guidelines. Baselines were recorded on claude-opus-4-8.

Files

  • evals.json — structure + behavior cases with assertions.
  • benchmark.json — recorded structure/behavior result (with-skill vs baseline).
  • fixtures/adr/ — synthetic ADRs, structurally matching the real catalog's frontmatter (adr/status/date/tags), heading, and Deprecated-admonition format (contributing.bitwarden.com/architecture/adr/): accepted (9001, 9004, 9005), a superseded->replacement pair (9002 -> 9005), and a deprecated one (9003). Superseded is a real, documented ADR status; the live catalog just hasn't used it yet. Fabricated so offline behavior cases grade deterministically without depending on the live catalog. They are NOT real Bitwarden decisions.

How to run

Structure + behavior: run each evals.json case with the skill vs without (baseline), then grade blind; actual run counts per arm are recorded in benchmark.json. Grading must be an LLM grader, not regex: the type names (CONFLICT/GAP/STALE-REFERENCE) appear both as finding labels and inside the roll-up count line, and label rendering is not fixed, so pattern matching both over-counts and misses. Assertions grade the finding a run reached, not the characters it used. Give the grader room to reason: one sentence of justification before a VERDICT: line. A grader constrained to a bare one-word answer returns verdicts that track the model rather than the output, strong models failing outputs they pass once allowed to explain. Blind all three tiers (subject, observer, grader).

The skill runs as a forked subagent, so the with-skill artifact to grade is the result the skill returns, not the calling session's summary of it. The caller paraphrases, and the paraphrase is not the skill's output.

Behavior cases point the skill at fixtures/adr/ via its local-clone path so grading is deterministic and offline. The source-call-live-contributing-site case provides no local checkout, so the skill must reach contributing.bitwarden.com; the recorded assertions check that the output cites a real ADR path and does not fabricate, not that a fetch actually occurred.

Known issues / boundaries

  • The fixtures path leaks the skill name into every offline prompt. Baseline runs read skills/consulting-adrs/evals/fixtures/adr and go looking for a skill that is absent from that environment, spending turns on a call that cannot succeed. The baseline stays valid, since nothing loads, but it is not blind to the skill's existence.
  • Four cases do not discriminate. stale-reference-superseded-adr, no-adr-found-do-not-invent, not-governed-suboptimal-is-not-a-conflict, and summarize-adr-catalog pass in both arms, so they guard against regression rather than measure the skill's contribution.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warning1mo3 checks · Risk SAFE
  • Gen Agent Trust Hub1mo

    The skill is a specialized utility for reviewing architectural decisions against Bitwarden's official ADRs. It is securely designed with a restricted toolset that prevents any shell command execution or file modifications. The skill also incorporates proactive instructions to ignore any potential directives found within the external documentation it analyzes, effectively mitigating the risk of indirect prompt injection.

  • Socket1mo

    No alerts

  • Snyk1mo

    Risk: MEDIUM · 1 issue

Signed by skilld at a11c77b. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 months ago
What it can do
Network Reads files
disallowed-tools
Write, Edit, Agent, Bash, WebSearch, Skill
context
fork
agent
general-purpose
All 4 allowed tools
WebFetch(domain:contributing.bitwarden.com)ReadGrepGlob
Other metadata
background
false

README badge

README badge for bitwarden/ai-plugins/consulting-adrs