Index of every page on this site: llms.txt
All skills
hagaiw avatar

/run-blind-evals

@33ac0e9
Run it yourself

Execute a frozen SkillRoll case pack with live inference, preserve its evidence, and classify technical, case-authoring, and target-skill outcomes.

  • 4 files
  • 7.9 KB
  • Updated 2 weeks ago
  • GitHub
Use this Skill: https://skilld.dev/gh/hagaiw/skillroll/run-blind-evals

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencescontext.md

≈392 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Live-run context

Live inference is the primary signal. Require explicit authorization, a named model and endpoint, and a key supplied only through the configured environment variable. Do not put credentials in commands, configuration, artifacts, or candidate-visible environments. Keep calls, samples, turns, tokens, and retries within the authorized bounds.

Run frozen cases in stable order and do not drop one after observing another's result. A no-skill comparison or extra samples are diagnostics, not silent additional gates.

Classify from evidence

  • PASS: a complete live run satisfies the authored behavior.
  • TARGET_BEHAVIOR: a valid, realistic case completed and the target skill missed one or more criteria.
  • CASE_AUTHORING: Input, World, criteria, path, or metadata did not express a fair self-contained experiment.
  • PRODUCT: SkillRoll itself failed and the evidence isolates that failure, preferably with a known-good control.
  • PROVIDER_ENVIRONMENT: transport, compatibility, credential, rate, or model failure prevented a behavioral verdict.
  • UNRESOLVED: available evidence cannot distinguish the cause.

An ERROR is not a behavioral FAIL. A timeout stays technical unless a complete result proves otherwise.

Preserve every artifact. One unchanged retry is reasonable only for a plausibly transient technical error and only within authorization. Correcting an authoring defect or changing a case, model, limit, endpoint, or revision creates a distinct non-comparable run. Report both the original and the new result.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 33ac0e9. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 6 days ago.

Activeupdated 2 weeks ago

README badge

README badge for hagaiw/skillroll/run-blind-evals