Index of every page on this site: llms.txt
All skills
erkamyaman avatar

/black-box-review

@7fcc555
Run it yourself

Use when reviewing code an AI agent wrote, especially in a language or area you don't know well. Judges the result through tests and behaviour instead of reading every line.

  • 2 files
  • 1.9 KB
  • Updated 2 months ago
  • GitHub
Use this Skill: https://skilld.dev/gh/erkamyaman/dhh-p-bloom-agent-skills/black-box-review

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈48 tokens for metadata: the name and description. ≈313 when used: this file. ≈124 more on demand in 1 file.

Description uses 2.4% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈192 characters, or 2.4%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Black-box review

Inspired by DHH's Rails World 2026 keynote: https://www.youtube.com/watch?v=vDjW_dRyKXY

From the talk

DHH says he doesn't know Rust and evaluates what the agents produce as a black box from the outside, the way a business owner judges work from commissioned programmers. The testing steps, red flags and security caveat below are our additions, not from the talk.

Judge agent output from the outside, the way a business owner judges work from a dev team: does it do what was asked, reliably, and fast enough?

When to use

  • Code in a language or framework you don't read fluently.
  • Large agent-generated changes where line-by-line review isn't practical.

Steps

  1. Run the existing tests. Add tests for the new behaviour before trusting it.
  2. Exercise the real behaviour: run the app, call the endpoint, use the CLI.
  3. Measure what matters: speed, memory, bundle size, startup time.
  4. Check the edges: empty input, large input, failures, offline.
  5. Scan the diff for red flags only (see references/red-flags.md), not for style.

Verification

Black-box review is weaker than reading the code. For anything touching security, money or user data, also have a person who knows the language review those parts.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 7fcc555. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last month.

Activeupdated 2 months ago

README badge

README badge for erkamyaman/dhh-p-bloom-agent-skills/black-box-review