skilld
All skills
shipshitdev avatar

/evaluation

@e20ffde
by Ship Shitshipshitdev/library37 stars
3

Build evaluation frameworks for agent systems. Use when testing agent performance, validating context engineering choices, or measuring improvements over time.

Ask your Agent

Use this Skill: https://skilld.dev/gh/shipshitdev/library/evaluation

This session only. Nothing lands on disk.

README.md

≈557 tokens on demand. Your agent reads this file only when SKILL.md points to it.

evaluation

Build evaluation frameworks for agent systems — deterministic validation plus model-judged quality, with attention to token/tool/model performance drivers.

Upstream

Derived from muratcankoylan/Agent-Skills-for-Context-Engineering (MIT).

Field Value
Source skills/evaluation/SKILL.md
Upstream ref main
Synced at commit 25e1fa79a33f
Last synced 2026-06-13
License MIT

Local modifications: Imported 2026-01-20 (this repo's commit ef42a98) from muratcankoylan/Agent-Skills-for-Context-Engineering at v1.0.0-era content. Ported forward 2026-06-13 to upstream HEAD (commit 25e1fa79a33f); local body now tracks upstream v1.2.0 — carried the deterministic-validation concept, Examples 3-4 (deterministic gate + quality-gate dimensions), 8-entry Gotchas, the claim-evaluation-browsecomp-variance ID, and the softened Performance Drivers table (concrete 80%/~10%/~5% -> qualitative Primary/Secondary labels). Reference to a sibling not vendored here (harness-engineering) was stripped. Local divergence: scripts/evaluator.py adopts the upstream citation-detection fix (naive bracket-matching -> academic-citation regex). references/metrics.md is byte-identical to upstream. A 2026-06-13 review-hardening pass (CodeRabbit on PR #21) further diverges scripts/evaluator.py: evaluation_history and samples are now bounded deques (10k/50k) to cap memory growth, and two no-op f-string prefixes were removed (Ruff F541) — candidates to push upstream. To diff: compare the upstream path on main since commit 25e1fa79a33f.

Checking for upstream changes: when upstream has moved ahead of the synced marker above, diff skills/evaluation/SKILL.md on main since commit 25e1fa79a33f, port anything worth bringing home, then bump metadata.upstream_commit (or metadata.upstream_version) and metadata.last_synced in SKILL.md and this table.

Source: SKILL.md on GitHub

Third-party checks

1 warning15d4 checks · Risk SAFE
  • Gen Agent Trust Hub15d

    The skill provides a framework for evaluating agent performance and quality using multi-dimensional rubrics and evaluation scripts. The code is well-structured and follows security best practices without any dangerous capabilities.

  • Socket15d

    No alerts

  • Snyk15d

    Risk: LOW · No issues

  • Runlayer7mo

    4/4 files flagged

Provenance

Signed by skilld at e20ffde. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 2 days ago.

Activeupdated 3 months ago

Capability

Other metadata
metadata
{
  "version": "1.2.0",
  "source": "https://github.com/muratcankoylan/Agent-Skills-for-Context-Engineering/blob/main/skills/evaluation/SKILL.md",
  "upstream_repo": "muratcankoylan/Agent-Skills-for-Context-Engineering",
  "upstream_ref": "main",
  "upstream_commit": "25e1fa79a33f",
  "last_synced": "2026-06-13",
  "license": "MIT",
  "tags": "evaluation, agents, testing"
}

README badge

README badge for shipshitdev/library/evaluation