Index of every page on this site: llms.txt
All skills
masihmoafi avatar

/experiment-workflow

@c0a5d13
by Masih Moafimasihmoafi/skills10 stars
Run it yourself

Use when designing, running, or comparing a repeatable experiment where a decision question, controlled variables, baselines, measurements, and raw evidence matter. Not for ordinary unit tests or one-off checks.

Use this Skill: https://skilld.dev/gh/masihmoafi/skills/experiment-workflow

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈58 tokens for metadata: the name and description. ≈353 when used: this file.

Description uses 2.9% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈233 characters, or 2.9%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Experiment workflow

One stated question at a time, with enough evidence for someone else to rerun it and judge the conclusion.

  1. Decision. State the decision the experiment informs and the result that would change it. One precise question; say what it cannot establish.
  2. Variables. Name the independent variable, outcomes, fixed conditions, baseline and known confounders. Change one variable at a time when you claim a cause; if several must move together, call it a comparison.
  3. Setup. Record exact inputs, versions, configuration, hardware, seeds and preprocessing; keep them fixed unless one is the variable.
  4. Measures, set before running. Units, method, pass threshold, repetitions, aggregation, and how failures and outliers count. Use the smallest sample that answers honestly; claim no certainty the design cannot carry.
  5. Run and keep evidence. Commands, raw outputs, failures and deviations, with baseline and variant side by side.
  6. Judge. Separate observation from interpretation, compare against the threshold, state uncertainty and the decision the evidence supports. This run is the next one's baseline.

For retrieval or RAG, keep the query set, retrieved passages, ranks, latency, chunking settings and corpus identity. A semantic/keyword/hybrid query suite is one useful stress test, sized to the corpus (about 10 of each type for a medium-to-large one).

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts3mo3 checks · Risk SAFE
  • Gen Agent Trust Hub3mo

    The skill provides a structured methodology for system testing and experimentation. It is a documentation-only skill with no code or active components, posing no inherent security risk.

  • Socket3mo

    No alerts

  • Snyk3mo

    Risk: LOW · No issues

Signed by skilld at c0a5d13. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last week.

Activeupdated 3 weeks ago

README badge

README badge for masihmoafi/skills/experiment-workflow