skilld
All skills
github avatar

/phoenix-evals

@4214189 official
by githubgithub/awesome-copilot40k stars
5,040

Build and run evaluators for AI/LLM applications using Phoenix.

Ask your Agent

Use this Skill: https://skilld.dev/gh/github/awesome-copilot/phoenix-evals

This session only. Nothing lands on disk.

referencessetup-typescript.md

≈246 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Setup: TypeScript

Packages required for Phoenix evals and experiments.

Installation

# Using npm
npm install @arizeai/phoenix-client @arizeai/phoenix-evals @arizeai/phoenix-otel

# Using pnpm
pnpm add @arizeai/phoenix-client @arizeai/phoenix-evals @arizeai/phoenix-otel

LLM Providers

For LLM-as-judge evaluators, install Vercel AI SDK providers:

npm install ai @ai-sdk/openai      # Vercel AI SDK + OpenAI
npm install @ai-sdk/anthropic      # Anthropic
npm install @ai-sdk/google         # Google

Or use direct provider SDKs:

npm install openai                 # OpenAI direct
npm install @anthropic-ai/sdk      # Anthropic direct

Quick Verify

import { createClient } from "@arizeai/phoenix-client";
import { createClassificationEvaluator } from "@arizeai/phoenix-evals";
import { registerPhoenix } from "@arizeai/phoenix-otel";

// All imports should work
console.log("Phoenix TypeScript setup complete");

Source: SKILL.md on GitHub

Third-party checks

No alerts9d4 checks · Risk SAFE
  • Gen Agent Trust Hub9d

    This skill provides comprehensive documentation and examples for building AI evaluators using Arize Phoenix. No malicious patterns were detected. It follows security best practices by recommending XML delimiters for LLM prompts and uses standard package management for its dependencies.

  • Socket9d

    No alerts

  • Snyk9d

    Risk: LOW · No issues

  • ZeroLeaks5mo

    Score: 93/100 · 2 sections analyzed

Provenance

Signed by skilld at 4214189. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 19 hours ago.

Activeupdated 2 months ago

Capability

Other metadata
compatibility
Requires Phoenix server. Python skills need phoenix and openai packages; TypeScript skills need @arizeai/phoenix-client.
metadata
{
  "author": "oss@arize.com",
  "version": "1.0.0",
  "languages": "Python, TypeScript"
}

Topics

README badge

README badge for github/awesome-copilot/phoenix-evals

What it does

Builds and runs evaluators for LLM applications using Phoenix, supporting code-based checks, LLM-as-judge approaches, and human validation workflows. Includes pre-built evaluators for RAG systems, error analysis, experiment tracking, and production monitoring across Python and TypeScript.

Generated from the current SKILL.md.

Frequently asked

Does this skill work with Python and TypeScript?
Yes. Python skills require the phoenix and openai packages; TypeScript skills require @arizeai/phoenix-client. Both require a running Phoenix server.
Can I build custom evaluators or only use pre-built ones?
You can build both code-based evaluators (deterministic logic) and LLM-based evaluators (using an LLM as a judge), with templates and validation support for both.
Does this cover RAG system evaluation?
Yes. The skill includes a dedicated RAG evaluators workflow covering retrieval quality and answer faithfulness.
Can I validate that my evaluators are accurate?
Yes. The skill provides validation references to test your evaluators' accuracy (TPR/TNR targets) against human labels.
What should I do before building evaluators?
The skill recommends starting with error analysis and tracing to observe actual failures, then categorizing them before automating evaluation.

Generated from the current SKILL.md. These answers refresh after source changes.