Index of every page on this site: llms.txt
All skills
starslingdev avatar

/skills-registry-security

@9a84115
by StarSlingstarslingdev/skills134 stars
3
Run it yourself

Checks the third-party security audit status of a skill published to a skills registry (skills.sh) across every cached surface, decides whether a failing finding is real or points at content already removed, and drives a stale badge to green unattended. Runs a real install to capture what users see, reads the per-provider finding codes the JSON API omits, and greps each flagged literal against the repository and shipped tree; emits a next-action verdict, checks whether an install can even fire the re-index beacon in this environment, and watches on a durable schedule until the registry's own re-audit clears a stale finding. Use when a published skill shows a FAIL or CRITICAL audit, when an audit looks stale or predates a fix, when a registry badge disagrees with a merged change, when deciding whether to request a re-index, before or after publishing a flagged skill, or when a Snyk, Socket, or Gen Agent Trust Hub verdict needs to be explained or disputed — so no one has to babysit a rescan by hand.

  • 9 files
  • 132.4 KB
  • Updated 3 months ago
  • GitHub
Use this Skill: https://skilld.dev/gh/starslingdev/skills/skills-registry-security

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

README.md

≈1k tokens on demand. Your agent reads this file only when SKILL.md points to it.

skills-registry-security

Maintainer-only. Deliberately outside skills/, so it never ships to end users.

Answers one question quickly: a published skill shows a failing security audit — is that real, or is the registry auditing stale content?

The security analysis itself belongs to the registry's providers (Gen Agent Trust Hub, Socket, Snyk). This tool triages their verdict: it decides whether a finding describes the skill as it is today, and if not, produces the evidence that says so.

Running it

python3 maintainers/skills-registry-security/scripts/registry_audit.py \
    starslingdev/skills/ci-secure

About 6 seconds with a live install, ~1.4s with --no-install. Everything is read-only: HTTP GETs, a throwaway install into a temp directory, and greps over a local checkout. Nothing is written to the repository and nothing is POSTed to the registry.

What one run does, in order

  1. Fans out concurrently across every cached surface: the JSON audit API, the install-path cache the CLI actually reads, the rendered page badges, and the three per-provider detail pages.
  2. Scrapes the finding codes (E005, W011, …) from those detail pages. The JSON API returns only prose like "CRITICAL · 2 issues", so the pages are the only place the actual findings exist.
  3. Runs a real npx skills add in a temp directory and captures the "Security Risk Assessments" table verbatim — what a user actually sees.
  4. Fetches the stored snapshot (/api/download/...), the content the scanners read, along with its hash.
  5. Searches every literal a finding quotes against three corpora: the git ref, the freshly installed tree, and the snapshot.
  6. Classifies each literal and prints a verdict:
    • REAL — still in current content. The finding stands; fix it or accept it with a written rationale.
    • STALE_INPUT — gone from the repository but still in the snapshot. Nothing to patch. Prints the snapshot hash and offending paths, which is what makes an escalation verifiable without access to your repository.
    • LAGGING — gone from both, and the audit predates the current snapshot. The ordinary post-fix state: the fix worked and the badge has not caught up. Wait ~a day; do not escalate.
    • PHANTOM — gone from both, and the audit has already read this snapshot. Only then is the scanner serving its own cached result, so a re-index alone may not clear it.
    • PHANTOM_OR_LAGGING — gone from both, with no --snapshot-changed-at to tell those two apart. It declines to guess rather than accuse the scanner.
    • UNVERIFIED — no corpus to compare against, so it declines to guess.

It also reports disagreement between surfaces, which is common and usually resolves on its own as caches catch up.

What ships with it

Path What it is
SKILL.md The agent-facing contract, plus the gotchas that cost the most time
scripts/registry_audit.py The engine. stdlib only
references/audit-pipeline.md How the registry actually works: the twenty-second diagram, an annotated real timeline, the precondition check, observed timings, tested dead ends, an evidence log, and how to gate CI so this cannot recur
references/registry-surfaces.md Exact URL shapes, response schemas, the E/W finding-code split
evals/evals.json Four scenarios
tests/ Offline tests, wired into the repo suite

Why it exists

The investigation this came from took about six hours and produced four wrong conclusions along the way — including "installs never trigger indexing," which came from an experiment invalidated by a silently suppressed telemetry beacon.

The script reproduces the correct answer in seconds. The references record both the working model and the mistakes, so the next person does not repeat them. The most important single line in there is the precondition check: an install whose beacon never fired looks exactly like a registry that ignores installs, and confusing the two sends you hunting a bug that does not exist.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 9a84115. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 16 hours ago.

Activeupdated 3 months ago

README badge

README badge for starslingdev/skills/skills-registry-security