Index of every page on this site: llms.txt
All skills
datadog-labs avatar

/dd-instrument-llmo

@618fae3 official
Run it yourself

Instrument the current project with Datadog LLM Observability for Python or Node.js/Next.js backends that call LLMs or run AI agents. Detects the runtime and LLM framework, provisions credentials, adds SDK init (ddtrace/dd-trace) with the correct kwargs, persists the dependency into the deploy manifest, and audits session-ID plumbing for gaps — fixing them when found. Use when the user says "instrument this project with LLM Observability", "add LLM Observability", "monitor my AI app in Datadog", "add LLM spans", "add agent session tracking", or "verify/repair my LLM Observability setup".

Use this Skill: https://skilld.dev/gh/datadog-labs/agent-skills/dd-instrument-llmo

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencescommon-verify-report.md

≈630 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Verification and reporting

Verify

Confirm the dd-trace/ddtrace package you added is declared in the dependency manifest the deploy installs from (the one identified in SKILL.md Phase 1e) — not merely present in your local environment. Grep that manifest for the package you added, or do a clean install into a throwaway env. This matters because a local pip install ddtrace followed by a successful uvicorn run passes the startup check below while the manifest is still missing the dependency — so the deploy would crash with ModuleNotFoundError even though local verification looked green.

For every backend runtime you instrumented, actually execute the instrumented entrypoint — do not rely on "the project's start/dev script." In a split frontend/backend repo there is often no single script that runs both halves, so "run the start script" can pass while never importing the backend at all. Load the module that contains your LLMObs.enable(...) / .init(...) so that init line runs:

  • Python: from the same working directory the deploy uses (e.g. cd backend when the module lives under backend/, so PYTHONPATH/cwd match the deploy), run python -c "import app.main" (substitute the real module path), or boot the real start command (uvicorn app.main:app) and confirm it binds with no traceback. If you used the ddtrace-run / import ddtrace.auto alternative (no in-code LLMObs.enable(...) to import), boot with that wrapper instead — e.g. ddtrace-run uvicorn app.main:app.
  • Node.js: node -e "require('./<entry>')" (or an import for an ESM entry), or boot the server entrypoint. If instrumentation is via NODE_OPTIONS='--import dd-trace/initialize.mjs', boot with that env set so the loader actually runs. For Next.js, run next build and confirm it completes without errors.

This is the step that catches a wrong import path or bad enable()/init() arguments — e.g. from ddtrace.llm_observability import LLMObs raising ModuleNotFoundError — which a passing manifest check will not surface.

If the app builds/runs successfully, tell the user where to look:

  • LLM Observability: https://app.datadoghq.com/llm/traces?query=@ml_app:<mlAppName>

Report

{
  "success": boolean,
  "mlApp": "llm-app-name",
  "message": "Human-friendly summary of what happened"
}

Alongside this shape, summarize goal status in the human-readable message — see "Reporting goal status" in llmobs-python.md / llmobs-nodejs.md.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts13d3 checks · Risk SAFE
  • Gen Agent Trust Hub13d

    This skill instruments projects with Datadog LLM Observability by installing official SDKs and configuring environment variables. It follows standard development practices for observability setup and credential management.

  • Socket14d

    No alerts

  • Snyk13d

    Risk: LOW · No issues

Signed by skilld at 618fae3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 days ago.

Activeupdated 2 months ago
Other metadata
metadata
{
  "version": "0.1.0",
  "author": "datadog-labs",
  "repository": "https://github.com/datadog-labs/agent-skills",
  "tags": "datadog,llm-observability,llmobs,instrumentation,python,nodejs,nextjs,ddtrace,dd-trace,agents",
  "alwaysApply": "false"
}

README badge

README badge for datadog-labs/agent-skills/dd-instrument-llmo