All skills
datadog-labs avatar

/dd-apm

@9bcb3ce official

APM - install, onboard, instrument, enable, set up, configure, traces, services, dependencies, performance analysis. Use for any request involving Datadog APM setup, instrumentation (SSI, ddtrace, agent install), or analysis.

Use this Skill: https://skilld.dev/gh/datadog-labs/agent-skills/dd-apm

This session only. Nothing lands on disk.

linux-ssiverify-ssiSKILL.md

≈1.6k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Verify APM SSI on Linux

Before doing anything else: Fully resolve all variables in ## Context to resolve before acting. Do not begin Step 1 until every variable has a concrete value.

Triggers

Invoke this skill when the user expresses intent to:

  • Confirm SSI is working after installing the Datadog Agent on Linux
  • Check whether a Linux process is being instrumented
  • Verify the tracer is running and reporting telemetry

Do NOT invoke this skill if:

  • SSI has not been enabled yet — run agent-install first
  • Services have not been restarted since the agent was installed — restart them first, then verify

Prerequisites

  • agent-install is complete
  • Application services have been restarted since the agent was installed

pup-cli: check, install, and authenticate

Claude runs

pup --version

If not found:

Claude runs

brew tap datadog-labs/pack
brew install pup

Check auth:

pup auth status --site <DD_SITE>

If not authenticated:

Claude runs

pup auth login --site <DD_SITE>

This opens a browser tab for OAuth. Complete the login there — Claude will continue once the command exits.

If valid token — proceed. ERROR: No browser available: export DD_APP_KEY=<your-app-key>


Context to resolve before acting

Variable How to resolve
DD_HOSTNAME Hostname as Datadog sees it — from sudo datadog-agent status output
SERVICE_NAME Expected service name in APM — ask the user
ENV Environment tag — ask the user
DD_SITE grep "^site:" /etc/datadog-agent/datadog.yaml via SSH, or ask the user
SSH_KEY Path to SSH private key
SSH_USER SSH username
SSH_HOST Hostname or IP of the target host

Step 1: Confirm the Process is Injected

Use /proc/<pid>/maps — this is the authoritative check. It shows the actual shared libraries loaded into the running process, which is the only way to confirm the launcher and tracer .so files were actually loaded.

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "pgrep -a -f '<SERVICE_NAME>' | head -5"

Use the PID from above:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo cat /proc/<PID>/maps | grep -E 'launcher|apm-library|datadog'"

If the output includes both the launcher (e.g. launcher.preload.so) and a language library (e.g. apm-library-python) — injection succeeded for this process.

ERROR: Launcher present but no language library — launcher ran but couldn't inject. Check for injection errors:

Claude runs

pup apm troubleshooting list --hostname <DD_HOSTNAME> --timeframe 1h

ERROR: Neither present — process was not injected. Check /etc/ld.so.preload:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> "cat /etc/ld.so.preload"

If empty — install did not set up the launcher. Re-run the install script with DD_APM_INSTRUMENTATION_ENABLED=host. If non-empty but the process still isn't injected — the process was started before the launcher was installed. Restart the service and recheck.


Step 2: Confirm the Agent is Receiving Traces

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo datadog-agent status 2>&1 | grep -A 15 'APM Agent'"

Healthy output shows:

  • feature_auto_instrumentation_enabled: true
  • Receiver (previous minute) with > 0 traces

ERROR: feature_auto_instrumentation_enabled: false — SSI not active on the agent. Check apm_config in /etc/datadog-agent/datadog.yaml.

ERROR: Receiver (previous minute): 0 — agent running but no traces yet. Generate traffic first (see Step 3), then recheck.


Step 3: Confirm the Service is Visible in Datadog

Claude runs

DD_SITE=<DD_SITE> pup apm services list --env <ENV> --from 1h

If <SERVICE_NAME> appears with isTraced: true — traces are reaching the Datadog backend.

Flask / ddtrace v3 naming note: With ddtrace >=3.x, Flask spans are emitted as service:flask rather than service:<DD_SERVICE>. The DD_SERVICE value appears as base_service on the spans. If you set DD_SERVICE=my-app, search for service:flask in the APM UI — the service list will show flask, not my-app. Check the base_service tag to confirm it matches your DD_SERVICE.

ERROR: Service missing — generate traffic to trigger trace creation:

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo ss -tlnp 2>/dev/null | grep <PID> || sudo netstat -tlnp 2>/dev/null | grep <PID>"

Use the port from above:

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "for i in \$(seq 1 10); do curl -s -o /dev/null http://localhost:<PORT>/; done"

Wait 30 seconds, then retry:

DD_SITE=<DD_SITE> pup apm services list --env <ENV> --from 10m
DD_SITE=<DD_SITE> pup traces search --query "service:<SERVICE_NAME>" --from 10m --limit 5

ERROR: Still missing — check for injection errors and go to troubleshoot-ssi:

pup apm troubleshooting list --hostname <DD_HOSTNAME> --timeframe 1h

Done

Exit when ALL of the following are true:

  • Step 1: launcher + language library both visible in /proc/<PID>/maps
  • Step 2: agent APM receiver shows > 0 traces/min
  • Step 3: service appears in pup apm services list

If any check fails, go to troubleshoot-ssi.

When all steps pass, automatically proceed to onboarding-summary now — do not ask the user for permission.


Security constraints

  • Never write a raw API key into any file or chat message
  • Always confirm before restarting production services

Source: SKILL.md on GitHub

1 warning2d5 checks · Risk SAFE
  • Gen Agent Trust Hub2d

    The skill facilitates the installation, configuration, and troubleshooting of Datadog APM for Kubernetes and Linux environments. It uses official Datadog installation scripts and a CLI tool ('pup') from the 'datadog-labs' GitHub organization. It requires administrative privileges (sudo) and SSH access to perform system-level instrumentation and configuration. All behaviors align with its documented purpose as a Datadog Labs utility.

  • Socket2d

    2 alerts: gptAnomaly

  • Snyk2d

    Risk: LOW · No issues

  • Runlayer6mo

    1/1 file flagged

  • ZeroLeaks5mo

    Score: 93/100 · 2 sections analyzed

Signed by skilld at 9bcb3ce. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 4 months ago
alwaysApply
true
Other metadata
metadata
{
  "version": "1.1.0",
  "author": "datadog-labs",
  "repository": "https://github.com/datadog-labs/agent-skills",
  "tags": "datadog,apm,tracing,performance,distributed-tracing,dd-apm,install,onboarding,instrumentation,ssi,agent",
  "globs": "**/ddtrace*,**/datadog*.yaml,**/*trace*"
}
  • Performance
  • datadog
  • apm
  • tracing
  • distributed-tracing
  • instrumentation
  • ddtrace
  • kubernetes
  • linux
  • service-remapping

README badge

README badge for datadog-labs/agent-skills/dd-apm

Installs the Datadog agent, enables single-step instrumentation (SSI) for automatic tracing, and provides commands to search traces and view service maps. Use this skill for any Datadog APM setup, onboarding, or performance analysis task on Kubernetes, Linux, or to rename services in APM.

Generated from the current SKILL.md.

Does this skill handle both Kubernetes and Linux host APM setup?
Yes. The skill routes to Kubernetes-specific sub-skills (k8s-ssi) when a cluster orchestrator is mentioned, and Linux-specific sub-skills (linux-ssi) for single hosts or VMs with no orchestrator.
Can I use this skill to rename services in Datadog?
Yes. The skill includes a service-remapping sub-skill that rewrites service names at ingestion time without requiring a deployment rollout.
Does this skill set up Single Step Instrumentation (SSI)?
Yes. SSI auto-instrumentation is covered in both the k8s-ssi and linux-ssi sub-skills; SSI requires no code changes and is enabled via agent installation flags or init container injection.
What if my request doesn't match Kubernetes, Linux, or service remapping?
The skill supports trace searching, service analysis, and metrics queries via pup commands. If your request still doesn't fit, the skill asks you to clarify rather than guessing a workflow.
Do I need to install Datadog Pup separately?
Yes. Datadog Labs Pup must be installed before using this skill; setup instructions are in the main agent-skills repository.

Generated from the current SKILL.md. These answers refresh after source changes.