All skills
datadog-labs avatar

/agent-observability-experiment-bootstrap

@47d0cf5 official

Bootstrap a reproducible LLM Observability experiment through the Python ddtrace SDK or the Node dd-trace SDK. Use for experiment, dataset, evaluator, benchmark, regression, or LLM-as-a-judge scaffolding. The legacy Python invocation remains supported.

Use this Skill: https://skilld.dev/gh/datadog-labs/agent-skills/agent-observability-experiment-bootstrap

This session only. Nothing lands on disk.

referencespythonprovidersanthropic.md

≈398 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Provider: Anthropic

Triggered by introspection (Workflow step 2.5) when the call-site function imports anthropic and calls one of:

  • anthropic.messages.create
  • client.messages.create (where client = anthropic.Anthropic(...))
  • Anthropic(...).messages.create

{{PROVIDER_ASSERTS}} substitution

assert os.getenv("ANTHROPIC_API_KEY"), "ANTHROPIC_API_KEY is required for the wired task_fn."

Optional env vars (do NOT assert; document in # TODO comments)

  • ANTHROPIC_BASE_URL — only needed when pointing at a proxy or self-hosted relay.

Adapter notes

  • anthropic.Anthropic().messages.create(model=..., max_tokens=..., messages=[...]) returns a Message object. Extract text via .content[0].text (note: content is a list of blocks, not a single string).
  • If the user's function returns the raw Message, wrap with a .content[0].text extractor in task_fn.
  • max_tokens is required — unlike OpenAI, Anthropic raises if it's missing. If the user's function omits it, leave their signature alone; the call will fail at runtime and the user can fix.
  • Async (AsyncAnthropic): wrap with asyncio.run(...) inside a sync task_fn.

Common gotchas

  • Tool use response format differs from OpenAI — tool_use blocks are interleaved with text blocks in .content. Don't assume .content[0] is text; loop and concatenate text blocks if needed.
  • Anthropic models require messages to alternate user/assistant (no two consecutive same-role messages). If the user's function builds messages, trust it; don't second-guess.

Source: SKILL.md on GitHub

No alerts1mo3 checks · Risk SAFE
  • Gen Agent Trust Hub1mo

    This skill provides scaffolding for LLM observability experiments using Datadog's official SDKs. It adheres to security best practices by explicitly forbidding the hardcoding of credentials, implementing PII scrubbing for datasets, and utilizing official vendor libraries and documentation.

  • Socket1mo

    No alerts

  • Snyk1mo

    Risk: LOW · No issues

Signed by skilld at 47d0cf5. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated last month

README badge

README badge for datadog-labs/agent-skills/agent-observability-experiment-bootstrap