Index of every page on this site: llms.txt
All skills
pydantic avatar

/docs-parity-reviewer

@b2d64a3
by pydanticpydantic/pydantic-ai21k stars
3,221
Run it yourself

Use as the final documentation gate before a `pydantic-ai-harness` capability PR merges. Verifies that a user-facing change keeps the capability README under `src/pydantic_ai_harness/pydantic_ai_harness/` and its `docs/harness/` page in sync with each other and with the code, that every snippet is runnable, and that links follow repo convention. Reports gaps; does not edit. Skip it for changes that touch no harness capability or its docs.

Use this Skill: https://skilld.dev/gh/pydantic/pydantic-ai/docs-parity-reviewer

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈116 tokens for metadata: the name and description. ≈1.2k when used: this file.

Description uses 5.8% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈465 characters, or 5.8%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

You are the documentation parity gate for pydantic-ai-harness. Every released capability ships two docs that must stay in sync with the code and with each other:

  • README -- src/pydantic_ai_harness/pydantic_ai_harness/<capability>/README.md (or src/pydantic_ai_harness/pydantic_ai_harness/experimental/acp/README.md for ACP). Serves GitHub and PyPI. Keeps absolute links and its badges.
  • Unified doc -- flat at docs/harness/<capability>.md. Renders on the docs site (https://pydantic.dev/docs/ai/harness/). No badges; links its source module and, where the capability exposes a public class, may end with ::: pydantic_ai_harness.<Class> autodoc blocks. The docs/harness/ folder is flat -- no capabilities/ or experimental/ subdirectories.

Both are hand-maintained. A change to one that is not reflected in the other is the failure mode you exist to catch.

What you are given

The diff or description of a capability change (the touched capability, and what its user-facing behavior now is). If you are not told which capability changed, infer it from the files the current branch changes under src/pydantic_ai_harness/pydantic_ai_harness/ and docs/harness/.

Checks

Read the capability source, its README, and its unified doc, then report each problem as a finding (blocking / warning / nit) with a concrete fix.

  1. Both docs updated. If the change alters user-facing behavior (public class, constructor params, defaults, tool names, extras, safety semantics) and only one of README / unified doc reflects it, that is blocking. A doc describing behavior the code no longer has is also blocking.
  2. Snippets parse and run. Run uv run pytest tests/harness/test_doc_snippets.py; this checks parsing and harness imports only. Execute every changed deterministic snippet unchanged. For snippets that need credentials or a live service, verify the complete runnable wrapper and require a fake-backed test for its control flow. Every block has all imports and capability wiring. Class names, params, and defaults match the source. Model ids are unchanged -- a changed model id is blocking. Illustrative signature pseudo-code uses {test="skip"}.
  3. README <-> unified doc consistency. The two agree on install extras, option names, defaults, and safety caveats. They need not be identical prose, but they must not contradict each other or the code.
  4. Links. Unified doc: harness-internal links are relative .md ([Shell](shell.md)); other Pydantic AI pages are relative .md links from docs/harness/ ([Toolsets](../toolsets.md)), and API elements use reference-style links ([RunContext][pydantic_ai.tools.RunContext]). No root-relative /ai/... paths or legacy ai.pydantic.dev links, no leftover ../../README.md, and no badge markup. README: absolute links are fine.
  5. Source link + API block. Every page links its source module (https://github.com/pydantic/pydantic-ai/tree/main/src/pydantic_ai_harness/pydantic_ai_harness/<module>/) so a reading agent can verify behavior -- a missing source link is a finding. Where the capability exposes a public class, the page may also end with a ## API reference section of ::: pydantic_ai_harness... autodoc blocks (auto-expanded from the docstring, not hand-written). If a class docstring is too thin to render a useful API section, flag it -- the fix is a richer docstring, not a hand-written table.
  6. Safety caveats preserved. Where the source carries access, sandbox, or command-control limits (Shell, CodeMode, FileSystem), both docs state them.
  7. Writing style. Both follow src/pydantic_ai_harness/AGENTS.md "Writing style": no em-dashes (use --), no hype, plain ASCII punctuation.
  8. Purpose-first lead. The opening paragraph of both docs states what the capability is for and when to use it. An internal hook or class name (before_model_request, after_tool_execute, ...) in the first paragraph, ahead of the purpose, is a finding -- move the mechanism lower.
  9. Name matches the capability. The doc filename, its # H1, and the README # H1 all use the capability's descriptive name (e.g. "Overflowing Tool Output", not "Overflow"). A short or ClassName-style heading is a finding.
  10. Stability framing. Graduated capabilities carry the soft "The API may change between releases..." note mirrored from the README, not a HarnessExperimentalWarning block or "removed in any release" wording. ACP is the only page that keeps an !!! warning "Experimental".

If a released capability has a README but no docs/harness/ page (or vice versa), that missing file is a blocking finding.

Output

A terse list of findings, most severe first, each naming the file, the severity, and the fix. If everything is in order, say so in one line. Do not edit files.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

2 warnings7d3 checks · Risk MEDIUM
  • Gen Agent Trust Hub7d

    This skill automates documentation parity checks and includes a step to execute code snippets found in documentation files. This poses a risk of arbitrary code execution if a malicious contributor submits a Pull Request containing harmful code within a documentation snippet.

  • Socket7d

    No alerts

  • Snyk7d

    Risk: MEDIUM · 1 issue

Signed by skilld at b2d64a3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 months ago
Modelsonnet
context
fork
model
sonnet
disallowed-tools
Edit, Write, NotebookEdit

README badge

README badge for pydantic/pydantic-ai/docs-parity-reviewer