/switchyard-testing-ci

Switchyard Testing And CI

Read .github/workflows/ci.yml before claiming local CI equivalence. Select tests from the current diff and symbol references rather than a static subsystem inventory.

Validation Matrix

Standard Gates

Python pre-PR gate:

uv run ruff check .
uv run mypy switchyard
env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY \
  -u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration"

Rust workspace gate:

cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace

Use -o addopts= when the repository's pytest defaults would stop failure collection too early. Run focused tests first, then broaden only when the changed contract or shared ownership warrants it.

Live Tests

Integration tests can spend money, require credentials, and contact real services. Never run them as an automatic pre-PR step. Provision NVIDIA_API_KEY through the environment or a secret manager before running the user-authorized path, and state which provider was called:

uv run pytest tests/e2e/<focused_test>.py \
  -v -m integration -o addopts=

Skill Validation

Each retained skill must have YAML frontmatter with name and description:

uv run python - <<'PY'
from pathlib import Path

import yaml

for path in Path(".agents/skills").glob("*/SKILL.md"):
    parts = path.read_text().split("---", 2)
    data = yaml.safe_load(parts[1]) if len(parts) == 3 else None
    if not isinstance(data, dict) or not data.get("name") or not data.get("description"):
        raise SystemExit(f"{path}: missing name or description")
print("skill frontmatter is valid")
PY
git diff --check

Reporting

Report exact commands, pass/fail status, and whether any live network call occurred. Do not call a focused test run "full CI", and do not treat mypy or clippy warnings as unrelated without checking whether the diff introduced them.