Index of every page on this site: llms.txt
Avatar for Comet

Comet

Comet builds the infrastructure teams use to develop, evaluate, and improve modern AI applications.

19 skills 23k

comet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

19 skills 23k
npx skilld add comet-ml/opik --all
  • Recipe for wiring a new linter into Opik's unified ๐Ÿ™ Code Quality pipeline (pre-commit + CI). Use when adding a pre-commit-driven linter/formatter to the repo โ€” enumerates every file that must change (`.pre-commit-config.yaml`, `scripts/precommit-hook-descriptions.tsv`, `scripts/precommit-detect-hooks.py`, `CONTRIBUTING.md`), the non-obvious gotchas (mandatory explicit `files:`, the blank-description trap, `TOOLCHAIN_BY_ID`/`TYPED_IDS`), the retroactive fix-vs-suppress policy, and the pass/fail verification loop. Worked examples: actionlint (live) and hadolint (OPIK-6673).

  • Add product analytics (BI) events to Opik features. Use when wiring events on the frontend, the backend, or the Python SDK - all three report through Segment to PostHog.

  • Use when an Opik E2E test has failed and a developer wants it investigated โ€” e.g. "why did this e2e test fail?", "investigate the failing run on my PR", "is dataset-crud-smoke flaky?", "the nightly e2e suite went red". Takes a failure from a CI check, a TestOps launch, a test name, or a local run; gathers the trace and history, classifies regression vs. flake, and proposes a fix. Read-only โ€” it diagnoses and proposes, it does not edit tests.

  • Generate self-contained HTML architecture diagrams. Use when creating visual diagrams for PRs, task plans, or architectural explanations.

  • Feature documentation and release notes patterns. Use when documenting changes, writing PR descriptions, or preparing releases.

  • Use when a developer wants an e2e test covering a change they just made โ€” e.g. "explore this feature", "add a test for my PR", "cover the feature in PR

  • Local development environment setup and commands. Use when helping with dev server, Docker, or local testing.

  • Specification for instrumenting an opik-backend workflow with operational OpenTelemetry metrics โ€” per-stage throughput/latency/error counters and native histograms, dimensioned per-customer (workspace). Use when a pipeline (scoring, ingestion, experiments, jobs) needs per-stage visibility. Covers metric emission only; building the Grafana dashboard from these metrics is a separate skill. Distinct from analytics-instrumentation (PostHog product events).

  • Java backend patterns for Opik. Use when working in apps/opik-backend, designing APIs, database operations, or services.

  • Build or update an Opik integration that lives OUTSIDE this repo โ€” a standalone opik-* package (e.g. opik-openclaw, opik-claude-code-plugin) or Opik support contributed into a third-party project (e.g. LiteLLM, Dify). Use ONLY when the user names an external repository or external package as the target; for integrations under sdks/ use the opik-integrations skill instead.

  • React frontend patterns for Opik. Use when working in apps/opik-frontend, on components, state, or data fetching.

  • Build, update, test, and document Opik SDK integrations (Python & TypeScript). Use when adding a new framework/provider integration under sdks/python/src/opik/integrations or sdks/typescript/src/opik/integrations, updating an existing one, or verifying that an integration logs traces correctly.

  • Use when building or extending a Page Object Model (POM) for the Opik E2E suite (under `tests_end_to_end/e2e/pom/`) and you need to choose stable selectors against the live UI. Walks through seeding required state, exploring the running page with the Playwright MCP (accessibility snapshot + data-testid enumeration), picking the most stable locator for each element, and verifying it before committing. Used as the discovery sub-step by the `writing-e2e-tests` skill.

  • Python SDK patterns for Opik. Use when working in sdks/python, on SDK APIs, integrations, or message processing.

  • Validate what a ClickHouse query actually costs before merging it โ€” at production scale through read-only environment access, or on a revived Testcontainers dataset extrapolated to 20k/500k/1M entities. Use when a DAO query changes, when an endpoint is slow, or when a reviewer asks "what does this cost at scale".

  • TypeScript SDK patterns for Opik. Use when working in sdks/typescript.

  • Authoring Fern MDX documentation pages for the Opik docs site, plus release-note and changelog routing. Use when writing or updating pages under apps/opik-documentation/documentation/fern/, drafting PR descriptions, or picking the right changelog surface.

  • Use when a developer wants to add, write, or create an end-to-end test for an Opik feature, page, or branch โ€” e.g. "add an e2e test for the experiments comparison page", "write a test for the feature I just built", "e2e test for this branch", "cover the dataset items flow with a test". Runs the full loop in tests_end_to_end/e2e/ โ€” analyze the feature and frontend code, explore the live UI with the Playwright MCP, write the Page Object Model + spec, and run it locally until green.

  • Use when a developer wants to add a visual regression (screenshot) test for an Opik UI page or panel โ€” e.g. "add a visual test for the trace sidebar", "screenshot each tab of the dataset panel", "visual regression test for the new empty state". Covers the page-object pattern, seeding via the test-helper-service, unique screenshot naming, per-test masking, baseline (re)generation, and the local-run-until-stable loop in tests_end_to_end/visual-tests/.

Skills published by Comet. Checked GitHub just now