Index of every page on this site: llms.txt
All skills
bitwarden avatar

/setting-up-regression-runs

@a11c77b official
by bitwardenbitwarden/ai-plugins165 stars
27
Run it yourself

Use when setting up the QA manual regression runs for a Bitwarden release in Testmo — e.g. "set up the 2026.8.0 regression runs", "create the partial release regression suite", "create the full regression runs for this milestone". Picks the release profile (partial or full), applies any per-release exclusions, and hands the profile's shipped specs to the bitwarden-testmo-tools creating-test-runs skill. Do NOT use it for a one-off Testmo run outside a release (use creating-test-runs directly) or to write test cases.

Use this Skill: https://skilld.dev/gh/bitwarden/ai-plugins/setting-up-regression-runs

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈138 tokens for metadata: the name and description. ≈789 when used: this file. ≈4.4k more on demand in 17 files.

Description uses 6.9% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈549 characters, or 6.9%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Setting Up Regression Runs

Bitwarden's manual regression suite for a release is a fixed set of Testmo runs, one per spec in specs/, all linked to the release's milestone. This skill decides which specs a release needs; Skill(bitwarden-testmo-tools:creating-test-runs) creates them. If that skill is not available, stop and ask the user to install bitwarden-testmo-tools.

Release profiles

release-profiles.json lists every spec in each profile:

  • partial — Web (Password Manager, Admin Console, Admin Portal / SM / Providers), Old Client / New Server, Mobile (iOS, Android), and Directory Connector (BWDC).
  • full — the same Web, Old Client / New Server, and Mobile runs, plus CLI, 3 Desktop and 4 Extension configuration variants. No BWDC.

full is not a superset of partial. Use exactly the list in the file.

Workflow

  1. Collect the inputs from the user: the profile (partial or full), the milestone's exact name (e.g. 2026.8.0 Manual Regression), and any spec to skip this release. Ask for whichever is missing.
  2. Confirm the milestone exists. The user creates it in the Testmo UI — the API cannot. Releases use a parent YYYY.M.0 Release milestone with … Manual Regression and … Automated Regression children; manual regression runs link to the … Manual Regression child.
  3. Build the spec list. Read the profile from release-profiles.json and drop any skipped specs. A name the user asks to skip that is not in the profile is a typo — stop and ask rather than continuing. Turn each remaining name into ${CLAUDE_PLUGIN_ROOT}/skills/setting-up-regression-runs/specs/<name>.json, written out as a literal absolute path. Never pass ${CLAUDE_PLUGIN_ROOT} itself to creating-test-runs — in that skill it means the bitwarden-testmo-tools root, not this plugin's.
  4. Hand off. Invoke Skill(bitwarden-testmo-tools:creating-test-runs) and follow it, passing every spec path as its own --spec and the milestone as --milestone-name. That skill owns the dry-run, the user's review, and --create — never run the dry-run and --create in the same turn.
  5. Finish the multi-configuration runs. Every Mobile, Desktop, and Extension run needs a manual Testmo UI pass after creation (full has all three; partial only Mobile). Tell the user which runs still need it, using references/regression-configurations.md and each spec's _comment.

To add one run that is missing from an already-populated milestone, pass only that spec — the script's per-run duplicate check leaves the existing runs alone.

Changing the suite

The specs and profiles here are shipped in the plugin; edits in the installed copy are overwritten by /plugin update. If a spec is skipped every cycle, or a new run joins the suite, change specs/ and release-profiles.json in a PR to bitwarden/ai-plugins.

Case counts drift as the repository changes. Compare each dry-run against the previous cycle; a large swing usually means a renamed folder or tag.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts21d3 checks · Risk SAFE
  • Gen Agent Trust Hub21d

    This skill automates the setup of manual regression test runs in Testmo for Bitwarden software releases. It safely manages test specifications and coordinates with established internal tools without exhibiting any malicious patterns or security vulnerabilities.

  • Socket21d

    No alerts

  • Snyk21d

    Risk: LOW · No issues

Signed by skilld at a11c77b. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 3 weeks ago
What it can do
Reads files
argument-hint
[partial | full] [milestone name]
All 2 allowed tools
ReadSkill(bitwarden-testmo-tools:creating-test-runs)

README badge

README badge for bitwarden/ai-plugins/setting-up-regression-runs