Index of every page on this site: llms.txt
All skills
langchain-ai avatar

/langchain-eval-engineering

@fb22e06 official
Run it yourself

Inspect an agent repository and optional traces, interview the user, write reviewed Task Specs, build and audit Harbor tasks, and bootstrap reusable project World Knowledge Skills. Use for agent evals, benchmark design, Task generation, controlled Environments, synthetic data, Verifiers, Harbor runs, calibration, or continuous benchmark maintenance.

Use this Skill: https://skilld.dev/gh/langchain-ai/langchain-skills/langchain-eval-engineering

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesexamplesservice-desk.md

≈750 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Service-Desk World Knowledge Example

This example shows how reusable project knowledge grows across Tasks. It is not a template for one fixed service-desk scenario.

After Task 1

Task 1 asks an agent to resolve a support request with account, requester, entitlement, owner, and history data. Its collocated Task.md keeps the exact request, focal records, accepted result, and Verifier criteria.

The audit proves reusable project facts:

  • the Harness uses search_cases, get_case, and update_case;
  • case visibility depends on team membership;
  • entitlement and account state can change the valid action;
  • updates write an event and change the case row in one transaction;
  • reset rebuilds SQLite from a fixed structured fixture; and
  • the Verifier can compare raw case and event state without exposing it.

Create .agents/skills/service-desk-world/SKILL.md. Add only these proven facts, their source paths, and routes to any reusable script.

After Task 2

Task 2 uses the same product but tests a different capability: it requires the agent to connect a stale knowledge article, a recent incident, and a customer message before making an update.

The second audit finds more reusable knowledge:

  • documents need stable structured facts before prose generation;
  • article freshness is decided by effective_at, not insertion order;
  • incident-product and case-product links must stay complete;
  • generated messages need fact-ID review for unsupported claims; and
  • known-good, valid-alternative, wrong, shortcut, collateral, and corrupt- evidence fixtures expose different Verifier faults.

Add a short reference for the data relationships and a reusable fixture validator only because two distinct Tasks now use them. Update SKILL.md to route agents to those items.

Resulting project skill

.agents/skills/service-desk-world/
├── SKILL.md
├── references/
│   └── data-and-verification.md
└── scripts/
    └── validate_fixture.py

Its root skill states when the knowledge applies and gives exact project paths, commands, state rules, and known limits. It can contain project-specific Task, Environment, data, Verifier, Harbor, and calibration guidance. It does not say only “follow langchain-eval-engineering,” and it does not copy broad guidance without a project-specific adaptation.

The skill does not include either Task's request, selected records, expected answer, or exact rubric. Those stay with each Task. It also does not add empty folders, a catalog, a manifest, or a file per concept. New content must reduce rediscovery or prevent a demonstrated defect in another Task.

Evidence of value

Task 3 tests whether the World Skill works. A new author should be able to find the correct tool contract, create relationship-complete data, reset state, and reuse the raw-state Verifier boundary without studying Tasks 1 and 2 in full. If a rule fails on Task 3, narrow or correct it and cite the new evidence.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warningtoday3 checks · Risk SAFE
  • Gen Agent Trust Hubtoday

    This skill provides a robust framework for engineering agent evaluations, including tools for discovery, environment building, and verifier design. It includes scripts for database snapshotting and tool schema comparison. The skill demonstrates high security awareness by implementing read-only SQL constraints and providing detailed guidance on isolating untrusted agent data. No significant security risks were detected.

  • Socket5d

    No alerts

  • Snyktoday

    Risk: MEDIUM · 1 issue

Signed by skilld at fb22e06. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated yesterday

README badge

README badge for langchain-ai/langchain-skills/langchain-eval-engineering