Index of every page on this site: llms.txt
All skills
hagaiw avatar

/author-blind-evals

@33ac0e9
Run it yourself

Author a small preregistered SkillRoll case pack from a target skill and neutral repository context without seeing prior run outcomes.

  • 4 files
  • 9.1 KB
  • Updated 2 weeks ago
  • GitHub
Use this Skill: https://skilld.dev/gh/hagaiw/skillroll/author-blind-evals

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencescontext.md

≈509 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Blind authoring context

The author may inspect the selected SKILL.md, directly linked references, and repository instructions needed to understand the skill. That is not outcome leakage. Keep these materials out of the authoring context:

  • earlier SkillRoll verdicts, transcripts, and report filenames;
  • suspected bugs or desired pass/fail outcomes;
  • baseline or no-skill responses; and
  • private success criteria disguised as user instructions.

Use a fresh worker or context per target when the platform supports it. Give the worker only the pinned preparation manifest and permitted source paths.

Every case is Markdown at <skill>/evals/<scenario>.eval.md:

# Scenario title

```skillroll
schema_version: 1
```

## Input

<realistic request>

## World

<private external facts and simulated action results>

## Success criteria

- <observable behavior allowing equivalent wording or actions>

Freeze a small pack

Prefer one case per distinct behavior and no more than two initial cases per skill. Input must resemble a real request and must not reveal the answer. World prose is evaluator-only; the evaluated skill learns external facts through actions. Criteria should be supported by final-output or transcript evidence.

For a boundary case, distinguish facts supplied in Input or the skill, facts discoverable through World actions, and facts or capabilities that remain unavailable. Define material absence, failure, or conflict semantics without turning a structural prompt concern into a predetermined behavioral verdict.

Run offline validation, then record each .eval.md path and content hash in a case manifest. Unchanged starter text, missing sections, invalid metadata, or an external scenario fact needed by the case but absent from realistic Input or World blocks the live run. A skill-owned expectation may come from the selected SKILL.md. Artifact rendering, real repository commands, and real service changes need their own evidence and should not be claimed by a semantic case.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 33ac0e9. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 6 days ago.

Activeupdated 2 weeks ago

README badge

README badge for hagaiw/skillroll/author-blind-evals