skilld
All skills
github avatar

/arize-annotation

@4e136f3 official
by githubgithub/awesome-copilot40k stars
5,040

Creates and manages annotation configs (categorical, continuous, freeform label schemas) and annotation queues (human review workflows) on Arize. Applies human annotations to project spans via the Python SDK. Use when the user mentions annotation config, annotation queue, label schema, human feedback, bulk annotate spans, update_annotations, labeling queue, annotate record, or human review.

Ask your Agent

Use this Skill: https://skilld.dev/gh/github/awesome-copilot/arize-annotation

This session only. Nothing lands on disk.

referencesax-setup.md

≈383 tokens on demand. Your agent reads this file only when SKILL.md points to it.

ax CLI — Troubleshooting

Consult this only when an ax command fails. Do NOT run these checks proactively.

Check version first

If ax is installed (not command not found), always run ax --version before investigating further. The version must be 0.14.0 or higher — many errors are caused by an outdated install. If the version is too old, see Version too old below.

ax: command not found

macOS/Linux:

  1. Check common locations: ~/.local/bin/ax, ~/Library/Python/*/bin/ax
  2. Install: uv tool install arize-ax-cli (preferred), pipx install arize-ax-cli, or pip install arize-ax-cli
  3. Add to PATH if needed: export PATH="$HOME/.local/bin:$PATH"

Windows (PowerShell):

  1. Check: Get-Command ax or where.exe ax
  2. Common locations: %APPDATA%\Python\Scripts\ax.exe, %LOCALAPPDATA%\Programs\Python\Python*\Scripts\ax.exe
  3. Install: pip install arize-ax-cli
  4. Add to PATH: $env:PATH = "$env:APPDATA\Python\Scripts;$env:PATH"

Version too old (below 0.14.0)

Upgrade: uv tool install --force --reinstall arize-ax-cli, pipx upgrade arize-ax-cli, or pip install --upgrade arize-ax-cli

SSL/certificate error

  • macOS: export SSL_CERT_FILE=/etc/ssl/cert.pem
  • Linux: export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
  • Fallback: export SSL_CERT_FILE=$(python -c "import certifi; print(certifi.where())")

Subcommand not recognized

Upgrade ax (see above) or use the closest available alternative.

Still failing

Stop and ask the user for help.

Source: SKILL.md on GitHub

Third-party checks

1 warning16d4 checks · Risk SAFE
  • Gen Agent Trust Hub16d

    The skill is safe and implements proper security guidelines. It provides standard configuration and usage instructions for the Arize CLI (ax) and Python SDK to manage annotation configurations and review queues without exposing sensitive credentials.

  • Socket16d

    No alerts

  • Snyk16d

    Risk: MEDIUM · 1 issue

  • ZeroLeaks5mo

    Score: 93/100 · 2 sections analyzed

Provenance

Signed by skilld at 4e136f3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 20 hours ago.

Activeupdated 5 months ago

Capability

metadata
{
  "author": "arize",
  "version": "1.0"
}
compatibility
Requires the ax CLI and a configured Arize profile.

Topics

  • Python
  • CLI
  • arize
  • annotation
  • labeling
  • human-feedback
  • span
  • dataset
  • queue

README badge

README badge for github/awesome-copilot/arize-annotation

What it does

Defines and manages annotation configs (label schemas) and queues (human review workflows) on Arize, then applies human annotations to project spans via the Python SDK. Use when creating categorical, continuous, or freeform label types; routing spans to reviewers; or bulk-updating span annotations with external labels.

Generated from the current SKILL.md.

Frequently asked

Does this skill work with dataset examples and experiment outputs, or only spans?
The skill covers annotation configs and queues that apply to spans, dataset examples, experiment-related records, and queue items. The Python SDK (`spans.update_annotations`) applies annotations specifically to project spans; dataset and experiment annotations are managed via the Arize UI or linked skills like arize-dataset and arize-experiment.
What are the three types of annotation configs I can create?
Categorical (fixed set of labels), continuous (numeric score within a range), and freeform (open-ended text feedback).
How do I bulk-apply annotations to spans from an external labeling tool?
Use the Python SDK `ArizeClient.spans.update_annotations()` with a DataFrame containing span IDs and annotation columns (label, score, or both). Annotations apply only to spans within 31 days prior to submission.
Do I need to create an annotation config before setting up a queue or applying labels?
Yes. An annotation config must exist in the space before you can attach it to a queue or expect labels to persist on spans, datasets, or experiments.
What happens if I delete an annotation config?
Deletion is irreversible. Any annotation queue associations to that config are also removed in the product, though the queues themselves remain.

Generated from the current SKILL.md. These answers refresh after source changes.