---
name: "review-library-rail"
description: "Review a pull request that adds or modifies a rail in nemoguardrails/library/ (built-in or vendor guardrail integration). Runs the mechanical gates first, then judgment dimensions tests cannot check (privacy honesty, retry semantics, fail-closed behavior, dialect coverage, docs accuracy), then a security pass. Use when reviewing rail-integration PRs, vendor integration contributions, or changes under nemoguardrails/library/. Trigger keywords - review rail PR, rail integration review, vendor integration PR, library rail review, guardrail contribution."
license: "Apache-2.0"
title: review-library-rail
canonical_url: https://skilld.dev/gh/nvidia-nemo/guardrails/review-library-rail
last_updated: 2026-11-18T15:00:49.000Z
---

> Index of every page on this site: [llms.txt](https://skilld.dev/llms.txt)

> **Skill from skilld.dev.** Follow the user's requested action.
> If the user asked to fork this Skill, follow the fork workflow below. Do not execute the Skill instructions while copying it.
> Otherwise, follow the instructions below for this session. You do not need to install anything.
>
> If the user asked to install this Skill, run `npx skilld install nvidia-nemo/guardrails/review-library-rail`. Install writes the Skill files into the project, so every session loads them.
>
> ## Fork workflow
>
> A fork creates an editable local Skill with its original author and licence. The request authorizes copying and local installation.
> 1. Check `./skills/review-library-rail`, the project lockfile, and selected Agent targets together. If the local directory or installed Skill exists, stop. Never overwrite an existing directory or Agent target.
> 2. Read [source metadata](https://skilld.dev/api/v1/skills/nvidia-nemo/guardrails/review-library-rail) once. Use sourceUrl, sourceCommit, skillPath, sourceGone, and license. If the source is gone or its path is missing, stop. If license is null, read licence files at the source commit.
> 3. Fetch only the source commit into a temporary Git repository. Do not clone full history. Derive repository_url from sourceUrl, including repository renames. If sourceCommit is absent, resolve the sourceUrl ref once. Set source_commit to that actual commit. Run these commands in one shell call:
>
> ```sh
> git init --quiet "$temporary_dir"
> git -C "$temporary_dir" fetch --quiet --depth=1 "$repository_url" "$source_commit"
> git -C "$temporary_dir" checkout --quiet --detach FETCH_HEAD
> ```
>
> Read applicable licence declarations and notices at that commit. If copying is not permitted, report the restriction and stop.
> 4. Inspect source entries together, then copy the directory containing skillPath into `./skills/review-library-rail`. Use the user's path if selected. Keep the original SKILL.md, relative links, scripts, binary assets, and executable modes. Exclude .git metadata. Reject symlinks and paths outside the Skill directory. After checking entries, use cp -a where available. A regular source directory needs no custom copy script. Do not save this page wrapper as SKILL.md.
> Preserve author credit, notices, and applicable licence files from repository or parent directories. Add PROVENANCE.md with the Skill page, source URL, actual commit, original path, and licence. Retain any existing PROVENANCE.md and record new provenance separately. Batch source inspection, copying, and provenance work where practical.
> 5. In the project root, run `skilld install ./skills/review-library-rail --mode copy --plain`. If skilld is unavailable, use `npx skilld install ./skills/review-library-rail --mode copy --plain`. This known command needs no help lookup. Install does not support --json. Use detected Agent targets, or add --agent for the targets the user selected. Install the local path, never the upstream selector. If installation fails, preserve the local copy and report the exact failure.
> 6. Confirm the local lockfile source and installed Agent copies once. Report the local path, actual commit, and Agent targets. After edits, reinstall the same local path. Upstream updates must not replace it. Do not publish or push unless the user asks.

# Reviewing a Library Rail Contribution

This skill reviews PRs that add or change a rail under
`nemoguardrails/library/`. The contract being reviewed against is defined by
the `add-library-rail` skill; do not restate it, read it. Follow the
repository Review Mode rules (compare against the merge base with `develop`,
inspect tests as well as implementation, treat every finding as advisory
until verified against the real code path).

The guiding fact, learned from evaluation: contributors and reviewers both
comply with what is named and miss what is omitted. Work through every
numbered check; do not stop at the first few findings.

## Step 0: Scope and duplicates

- Confirm the PR is a rail contribution: files under
  `nemoguardrails/library/<name>/`, usually with tests and a docs page.
- Check for duplicate or in-flight integrations for the same vendor:
  `gh pr list --state open --search "<vendor>"` and the same for issues. If
  another PR covers the vendor, surface that before reviewing further.
- Is this vendor integration plausibly wanted? If there is no linked triaged
  issue or maintainer signal, flag for maintainer judgment; do not review it
  into shape first.

## Step 1: Run the mechanical gates (do not re-review what they check)

Run pre-commit FIRST, per the repository Review Mode rule: it reformats, so
running it later moves the lines your findings cite.

```bash
uv run --locked pre-commit run --files <the PR's changed files>
make test TEST=tests/test_rail_skill_references.py
make test TEST="tests/rails/llm/test_builtin_rail_manifests.py tests/rails/llm/test_builtin_rail_conformance.py tests/rails/llm/test_library_flow_files.py"
make test TEST="tests/rails/llm/test_config.py tests/test_rails_config.py tests/test_runtime_flow_gate_equivalence.py"
make test TEST="tests/guardrails/test_compiled_rail.py tests/guardrails/test_guardrails.py tests/guardrails/test_rails_manager.py"
make test TEST="tests/llm/test_call_import_graph.py tests/telemetry/test_usage_reporting.py"
make test TEST=tests/http/test_library_boundary.py
make test TEST="<space-separated PR test files>"
make test TEST=tests/recorded/rails/library ARGS="--block-network -q"
make test TEST=tests/recorded/test_cassette_provenance.py
git diff <merge-base> -- pyproject.toml uv.lock   # usually empty; see below
```

A red gate is a finding by itself; report it and keep going. A green gate
proves only that its collected cases passed. Inspect the summary for relevant
skips, especially optional vendor dependencies and IORails cases; a skipped
PR path is not a completed dimension. Spend judgment attention on what the
gates do not encode.

Check packaging diffs against the current optional extras in `pyproject.toml`
and `nemoguardrails/manifests/manifest.py::RailRequirements`. A justified new
extra is valid; adding an optional vendor package to the default install path
is not. The authoring skill's dependency section owns the install checklist.

## Step 1b: New-rail completeness (presence, not quality)

The manifest gate detects action packages without an ancestor `rail.py`, but a
config-only or no-action package can evade that scan. For every new rail,
confirm the applicable items are present before judging quality:

- `rail.py` with a `RAIL` manifest. For an ordinary action package the
  manifest gate enforces this; inspect config-only/no-action contributions
  explicitly.
- For a flow-bearing rail, both-dialect unit tests: block path under Colang 1
  AND under `colang_version: "2.x"`. A v2 flow file with no v2 test is
  unexecuted code; inbound PRs have shipped v2 files that parse but cannot run.
- For tool-only rails, IORails integration tests instead of Colang files and
  dialect tests: allow/block, argument scoping when supported, and errors.
  `cel` and `tool_safety_check` declare tool surfaces with empty flow-file lists.
- For an executable provider-backed rail, an engine-level test of the action
  raising (vendor down, fail-closed), through Colang where applicable and
  through IORails for tool-only rails.
- A recorded entry under `tests/recorded/rails/library/`, mandatory per
  `nemoguardrails/library/README.md`. Provider-backed rails cover allow,
  block, and one recordable provider error. Pure-Python rails cover applicable
  allow/block or transform behavior without a provider-error case. A
  config-only manifest covers its public configuration contract.
- Docs page and example config.

Each missing item is a finding; request it rather than inferring it is
covered elsewhere.

## Step 2: Judgment dimensions (what tests cannot check)

Each item names the check, where to look, and what "wrong" looks like.

1. **Privacy honesty.** Compare `RailPrivacy` in `rail.py` against what the
   request layer actually transmits and what the vendor's own documentation
   says about retention. `sends_user_text`/`sends_bot_text` must match the
   surfaces' bindings, including `sends_retrieved_chunks` for a retrieval
   surface and `sends_tool_call_text`/`sends_tool_result_text` for tool data
   sent to a remote judge or vendor; `remote_services` must be non-empty for
   any HTTP rail; `data_retention` must be stated if the vendor states one. A
   defaults-only `RailPrivacy()` on a vendor rail is almost always wrong.
2. **Requirements honesty.** Every env var the code reads
   (`os.environ`/`os.getenv`) appears in `RailRequirements.env_vars`; every
   lazily imported package appears in `optional_dependencies` (and in
   `extras` when the package already ships in a nemoguardrails extra); the
   docs `pip install` line matches the declaration and carries the version
   bound, which `RailRequirements` cannot express. Named model selections use
   `Binding.model_param(...)` or `Binding.model(...)`, not ordinary surface or
   literal bindings: the model resource drives IORails compilation checks and
   `RailsConfig` validation for global and per-tool model references.
3. **Retry semantics.** For HTTP rails: a rail-owned `RetryPolicy` in one
   named module-level place, either a constant or a single config-derived
   builder, never inline at the call site; POST retries only via explicit
   `retryable_methods={"POST"}` carrying a comment that names one of the
   three permitted resend-safety mechanisms (documented idempotency, an
   idempotency key header, or a stateless scan endpoint), and no comment is
   a finding; no hand-rolled retry
   loop around `http_call`. Apply it through
   `ensure_retrying_http_client`, not by directly constructing
   `RetryingHTTPClient(injected_client, policy)`: existing retry policy takes
   precedence, retry must sit inside instrumentation, and duplicate retry or
   instrumentation layers are rejected. Then verify any retry/backoff claims
   in docs or comments against `nemoguardrails/http/retry.py`: backoff is full-jitter
   exponential with a lower bound of zero, `Retry-After` is honored only
   within `max_retry_after` and is otherwise discarded rather than capped
   unless `clamp_retry_after=True`, and vendor override headers are ignored
   unless `honor_retry_override_header` is set. Contributors document this
   from guesswork.
4. **Fail-closed behavior, tested not asserted.** Vendor errors, timeouts,
   and malformed payloads must surface as typed errors or blocks, never a
   silent allow. For an executable provider-backed rail, there must be a
   test of the action raising through each supported engine; a summary or
   comment claiming fail-closed without a test is a finding. IORails maps
   ordinary action exceptions to `RailOutcome.failure` but propagates
   status-bearing `LLMCallException` and `ModelEngineError`. Verify the
   relevant public behavior rather than expecting every error to raise or
   every error to become a refusal.
5. **Dialect coverage.** For flow-bearing rails, the flow-files gate checks
   structure; behavior needs tests. Both dialects' block path must be exercised
   end to end in the UNIT suite (`colang_version: "2.x"` variant present?), and a per-rail
   duplicate Colang 2 RECORDED test is itself a finding when it produces the
   same provider traffic. Public-API v2 recordings remain valid when the
   runtime changes the calls, and IORails parity may reuse a sibling cassette
   or deliberately run a deterministic no-cassette case. If any surface declares
   a `transform_target` and exposes Colang flows, the transform path also needs
   a Colang 2 behavior test; the flow-gate harness is Colang 1 only, so a v2
   transform can be entirely unexecuted while the block-path tests pass. Every
   `enable_rails_exceptions` branch must be either tested or absent. Check
   `flows.co` and `flows.v1.co` against the four equivalence items in
   add-library-rail Step 3 (same flow names, same action and bindings per
   flow, same branch structure, same asserted output); divergent semantics
   between dialects is a finding even when both parse.
6. **Outcome discipline.** Actions return `RailOutcome` with neutral decision
   evidence. A supported `metadata["refusal_message"]` override may carry
   localized or configured refusal text, as in #2470; verify the consumer and
   rendering/fallback tests for each supported engine. Unsupported presentation
   keys or embedded exception types are findings. Flows must only read metadata
   keys the action actually sets (grep the action for each `metadata[` subscript
   used in a flow). Then cross-check the config model: every field in `rail_config.py`
   must have a test at a non-default value, and decision-changing fields
   need both sides of the boundary. Read the comparison operator against the
   documented semantics; an inverted or off-by-one threshold passes every
   gate.
7. **Telemetry content-freeness.** No logging of checked text, request or
   response bodies, URLs with query strings, API keys, or tokens; exception
   messages must not embed payloads. Check `log.` calls and every f-string
   in raised errors.
8. **Test placement and hygiene.** Deterministic LLMRails tests in `tests/`
   may inject `RecordingHTTPClient` (from `nemoguardrails.testing`) through
   `register_action_param`. IORails does not support arbitrary action-param
   registration; its compiled rail receives the engine-managed HTTP client.
   Compare the action's signature with
   `nemoguardrails/guardrails/compiled_rail.py::CompiledRail::_request_dependencies`
   and its manifest bindings instead of assuming LLMRails injection is portable.
   Recorded suite entries follow the
   `recorded-tests` skill (vcr-structured, or non-vcr with a defensible
   reason). Nothing enforces that split, so read every suite test in the
   diff that has no `vcr` mark and challenge it against the skill's
   criteria. Secrets via `monkeypatch`; any real-looking key in a test or
   cassette is a blocking finding.
9. **Docs accuracy.** Every statement in the catalog page must match the
   code: config options, defaults, env vars, install line, the vendor's data
   retention period (matching `RailPrivacy.data_retention`), and
   limitations. Verify each "known limitation" is real; a false limitation
   is as bad as a missing one, and an uncited or unreproducible one is a
   finding. `docs_url` in the manifest is the repo-relative `.mdx`
   path of the page the PR adds; the manifest gate asserts that path
   resolves.
10. **Fail-open semantics.** If the rail supports fail-open at all: it must
    default to fail-closed; the switch must live in the rail's config model
    where a config reviewer sees it, not in an env var (env vars are for
    secrets, and fail-open flips the rail's security posture); a fail-open
    pass must be distinguishable from a genuine vendor clear, carrying an
    explicit marker in the outcome metadata (as `_fail_open_outcome` in
    `nemoguardrails/library/f5/actions.py` does) so it is never mistaken for
    a real clear in logs and traces, and it must be logged. A
    fail-open path also needs its own test alongside the fail-closed one.
    Cross-check the docs against the code: if the page claims the switch
    covers availability only, confirm the fail-open branch actually excludes
    4xx. Failing open on 401 or 403 while documenting otherwise leaves a
    revoked key silently unguarded, and is a finding.
11. **Cassette provenance.** A cassette is a claim by its submitter, and an
    agent or contributor without service access can fabricate one that
    looks recorded. The sanitizer fixed-point gate
    (`tests/recorded/test_cassette_provenance.py`) catches naive
    fabrication (realistic ids, timestamps, cookies that the recorder
    would have scrubbed), but a sentinel-perfect forgery passes it, so for
    new cassettes from external contributors also ask HOW they were
    recorded. If the answer is unverifiable and the org holds no key to
    re-record, the honest options are maintainer re-recording before
    merge or demoting the tests to the explicit `fake_cassette` regime;
    do not merge unverifiable cassettes as recorded truth.
12. **Conformance drift (a green suite can hide this).** The conformance
    gates pass if a contributor made them pass the WRONG way, so diff the
    generic infrastructure, not just the rail. Reject weakened checks in
    `test_builtin_rail_manifests.py`, `test_builtin_rail_conformance.py`, or
    `test_library_flow_files.py` unless the PR's purpose is the gate itself.
    Extending an applicable pinned inventory, such as the set of model-bound
    surfaces, is required; changing the rule to admit an invalid rail is not.
    Reject a new entry in `LEGACY_UNMANIFESTED_PACKAGES` or
    `NON_PORTABLE_DECLARED_FLOWS` unless it carries an explicit, reviewed
    design rationale (a new rail should satisfy the contract, not join the
    exceptions). For any change to `schemas/rails_config.snapshot.json`,
    confirm it was regenerated for a deliberate config-schema change and
    read the diff as a public-schema change, not a mechanical refresh.
13. **Error-path coverage (unit, exhaustive).** Dimension 4 checks that
    errors fail closed; this checks that EVERY error branch is actually
    exercised. Enumerate the action's failure branches (timeout, connection
    error, each handled status class, 429 retried and retry-exhausted,
    malformed payload, missing credential, missing optional dependency, each
    crossed with fail-open/closed where supported) and require a unit test
    per branch with a mocked transport. A branch with handling code but no
    test is a finding. These belong in the unit layer, not recorded: a
    synthetic error forced into `tests/recorded/` violates the placement
    rule, so do not accept "it's covered by the recorded suite" for anything
    but the recordable real error.
14. **Client composition, ownership, and error typing.** These are not fully
    gate-checked; the HTTP
    boundary test is an import-only AST check, so read the code. The action
    must never call `close` or `aclose` on an injected `http_client`; only
    the `factory=` path passed to `http_call` may own a client, and that
    factory must return the fully composed closable client, since wrapping
    an injected client must not transfer ownership
    (`nemoguardrails/http/request.py`). Verify composition through
    `ensure_retrying_http_client`: the canonical topology is instrumentation
    outside retry outside transport, and an existing retry/instrumentation
    layer is preserved rather than compounded. Vendor failures must surface as
    rail-specific error types (the `errs.py` pattern, exemplar
    `nemoguardrails/library/clavata/errs.py`), not as raw transport or
    vendor SDK exceptions escaping the action.

## Step 3: Security pass

- **Credential handling**: key read from env only, never from config values
  that end up in logs or snapshots; JWT/signature construction does not log
  the secret; no credentials committed in tests, cassettes, or examples.
- **Outbound URL control**: if the config exposes `base_url` or an endpoint
  override, note that guardrails configs are sometimes attacker-adjacent;
  the URL must only ever be used for the vendor call (no template
  injection into paths), and redirects stay disabled (transport default).
- **Input handling**: user/bot text goes into the request body as data,
  never interpolated into URLs, headers, or shell commands; response
  parsing treats the payload as untrusted (no eval, no attribute access on
  unvalidated structures without try/except).
- **Deserialization**: only `response.json()`; anything like pickle or yaml
  load on vendor bytes is blocking.

## Step 4: Process conformance

- DCO sign-off on every commit; conventional commit title; no changelog
  edits; AI_POLICY disclosure if applicable.
- Docs build if docs changed: `make docs-fern`.
- Every reviewer finding verified against the actual code path before it is
  reported (Review Mode rule); drop anything you cannot reproduce or point
  to a line for.

## Verdict format

Report findings ranked by severity, each with file:line evidence and the
dimension number it came from. State explicitly which gates ran and their
results, and which dimensions came up clean; a review that only lists
problems hides its own coverage. End with one of: approve, approve with
nits, request changes (list the blocking subset).
