Index of every page on this site: llms.txt
All skills
google avatar

/agent-platform-eval-flywheel

@5fb62c2
by googlegoogle/skills24k stars
2,026
Run it yourself

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when generating synthetic user scenarios, evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.

Use this Skill: https://skilld.dev/gh/google/skills/agent-platform-eval-flywheel

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesattribution.md

≈490 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Usage Attribution

MUST READ: Before executing any gcloud, REST API, or MCP command with this skill, follow these usage attribution instructions. Tag every command you execute directly in the terminal so usage can be attributed. The tag identifies only the skill and its version; it carries no user data.

  • gcloud CLI Commands: Prefix every gcloud invocation, whatever the subcommand, with the metrics environment variable. Set it inline on each command because shell state may not persist between commands:

    CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:agent-platform-eval-flywheel)" \
    gcloud <command> [flags]

    Do not use gcloud config set for this: it would persist beyond the current task and mislabel unrelated usage.

  • Direct HTTP, REST, and MCP Requests: On direct HTTP calls to Google Cloud APIs (for example with curl) or HTTP requests to a remote MCP server, include the agent-skills/1.0 and skill:agent-platform-eval-flywheel tokens in the User-Agent header. If a User-Agent header is already specified, append the attribution suffix to the end of it rather than overwriting existing User-Agent data — the collection pipeline parses the agent-skills/<version> and skill:<name> tokens, so any rewording breaks attribution:

    User-Agent: <existing_user_agent> agent-skills/1.0 (skill:agent-platform-eval-flywheel)
  • Constraint for Generated Code: The tracking prefix is strictly for commands you execute directly in the terminal. Do NOT include this tracking prefix in generated scripts, Terraform templates, or code provided to the user for their own repositories.

  • User Opt-Out: If the user asks to disable or remove tracking or attribution tags, omit the CLOUDSDK_METRICS_REQUEST_ATTRIBUTION variable and custom User-Agent suffix from all commands.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts18d3 checks · Risk SAFE
  • Gen Agent Trust Hub18d

    This skill facilitates AI model and agent evaluation using the Google GenAI Evaluation SDK. It includes standard developer operations, such as command-line execution for authentication and dependency installation, all of which are managed through a robust safety tier system that requires explicit user confirmation for impactful actions.

  • Socket18d

    No alerts

  • Snyk18d

    Risk: LOW · No issues

Signed by skilld at 5fb62c2. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 3 weeks ago
metadata
{
  "version": "1.0.4",
  "category": "AiAndMachineLearning"
}

README badge

README badge for google/skills/agent-platform-eval-flywheel