Index of every page on this site: llms.txt
All skills
datadog-labs avatar

/dd-logs

@618fae3 official
Run it yourself

Log management - search, archives, metrics, and cost control.

Use this Skill: https://skilld.dev/gh/datadog-labs/agent-skills/dd-logs

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈18 tokens for metadata: the name and description. ≈1.2k when used: this file.

How this Skill uses context

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈71 characters, or 0.9%.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Datadog Logs

Search, process, and archive logs with cost awareness.

Prerequisites

Datadog Pup should already be installed. See Setup Pup if not.

Command Execution Order (Token-Efficient)

For scoped commands, use this order:

  1. Check context first (prior outputs, conversation, saved values).
  2. If a required value is missing, run a discovery command first.
  3. If still ambiguous, ask the user to confirm.
  4. Then run the target command.
  5. Avoid speculative commands likely to fail.

Quick Start

pup auth login

Search Logs

# Basic search
pup logs search --query="status:error" --from="1h"

# With filters
pup logs search --query="service:api status:error" --from="1h" --limit 100

# JSON output
pup logs search --query="@http.status_code:>=500" --from="1h"

Search Syntax

Query Meaning
error Full-text search
status:error Tag equals
@http.status_code:500 Attribute equals
@http.status_code:>=400 Numeric range
service:api AND env:prod Boolean
@message:*timeout* Wildcard

Commands

pup logs search --query="status:error" --from="1h"
pup logs list
pup logs aggregate --query="*" --from="1h" --compute="count" --group-by="service"
pup logs patterns --query="status:error" --pattern-field="message" --from="1h"
pup logs query --query="service:api" --from="1h"
pup logs saved-views list

pup logs archives list
pup logs archives get <archive-id>
pup logs custom-destinations list
pup logs metrics list
pup logs metrics get <metric-id>

# Restriction queries are a top-level command, not a logs subcommand.
pup logs-restriction list

--compute and --group-by on aggregate, and the pattern flags, follow the installed pup logs <command> --help if it disagrees with the examples above.

Common Processors

{
  "name": "API Logs",
  "filter": {"query": "service:api"},
  "processors": [
    {
      "type": "grok-parser",
      "name": "Parse nginx",
      "source": "message",
      "grok": {"match_rules": "%{IPORHOST:client_ip} %{DATA:method} %{DATA:path} %{NUMBER:status}"}
    },
    {
      "type": "status-remapper",
      "name": "Set severity",
      "sources": ["level", "severity"]
    },
    {
      "type": "attribute-remapper",
      "name": "Remap user_id",
      "sources": ["user_id"],
      "target": "usr.id"
    }
  ]
}

Exclusion Filters (Cost Control)

Index only what matters:

{
  "name": "Drop debug logs",
  "filter": {"query": "status:debug"},
  "is_enabled": true
}

High-Volume Exclusions

# Find noisiest log sources
pup logs search --query="*" --from="1h" | jq 'group_by(.service) | map({service: .[0].service, count: length}) | sort_by(-.count)[:10]'
Exclude Query
Health checks @http.url:"/health" OR @http.url:"/ready"
Debug logs status:debug
Static assets @http.url:*.css OR @http.url:*.js
Heartbeats @message:*heartbeat*

Archives

Store logs cheaply for compliance:

# List archives
pup logs archives list

# Archive config (S3 example)
{
  "name": "compliance-archive",
  "query": "*",
  "destination": {
    "type": "s3",
    "bucket": "my-logs-archive",
    "path": "/datadog"
  },
  "rehydration_tags": ["team:platform"]
}

Rehydrate (Restore)

# Pup has no logs rehydrate command.
# Rehydrate from the Datadog UI or API.

Log-Based Metrics

Create metrics from logs (cheaper than indexing):

# List log-based metrics
pup logs metrics list

# Get one metric by ID
pup logs metrics get api.errors.count

Cardinality warning: Group by bounded values only.

Sensitive Data

Scrubbing Rules

{
  "type": "hash-remapper",
  "name": "Hash emails",
  "sources": ["email", "@user.email"]
}

Never Log

# In your app - sanitize before sending
import re

def sanitize_log(message: str) -> str:
    # Remove credit cards
    message = re.sub(r'\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b', '[REDACTED]', message)
    # Remove SSNs
    message = re.sub(r'\b\d{3}-\d{2}-\d{4}\b', '[REDACTED]', message)
    return message

Troubleshooting

Problem Fix
Logs not appearing Check agent, pipeline filters
High costs Add exclusion filters
Search slow Narrow time range, use indexes
Missing attributes Check grok parser

References/Documentation

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warning15d5 checks · Risk SAFE
  • Gen Agent Trust Hub16d

    This skill provides commands and documentation for managing Datadog logs via the 'pup' CLI tool. All operations are legitimate and follow Datadog-labs' standard practices for log management, search, and cost control.

  • Socket15d

    No alerts

  • Snyk16d

    Risk: MEDIUM · 1 issue

  • Runlayer8mo

    1/1 file flagged

  • ZeroLeaks7mo

    Score: 93/100 · 2 sections analyzed

Signed by skilld at 618fae3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 days ago.

Activeupdated 3 weeks ago
Other metadata
metadata
{
  "version": "1.0.2",
  "author": "datadog-labs",
  "repository": "https://github.com/datadog-labs/agent-skills",
  "tags": "datadog,logs,logging,search,dd-logs",
  "globs": "**/datadog*.yaml,**/*log*",
  "alwaysApply": "false"
}
  • datadog
  • logs
  • logging
  • search
  • archives
  • metrics
  • cost-control
  • pup

README badge

README badge for datadog-labs/agent-skills/dd-logs

Searches, processes, and archives logs in Datadog using the pup CLI tool, with built-in cost control through exclusion filters and log-based metrics. Covers log querying, pipeline configuration, grok parsing, sensitive data scrubbing, and archive setup for compliance storage.

Generated from the current SKILL.md.

What version of pup does this skill require?
The skill is tested with pup 0.42.0. Some commands like log-based metrics and archives are available in that version; rehydration workflows require the Datadog UI or API.
Can I use this skill to reduce log costs?
Yes. The skill includes exclusion filter examples (health checks, debug logs, static assets) to drop low-value logs, and archive configuration to store logs cheaply in S3 for compliance.
Does this skill handle sensitive data scrubbing?
It provides hash-remapper and example Python code to redact credit cards and SSNs before logs are sent. Scrubbing rules can be applied via Datadog pipelines.
What log sources can I search?
You can search any logs ingested into Datadog using the pup CLI. Queries support full-text search, tag filters, attribute filters, numeric ranges, boolean operators, and wildcards.

Generated from the current SKILL.md. These answers refresh after source changes.