Index of every page on this site: llms.txt
All skills
launchdarkly avatar

/alert-investigation

@f0ee110 official
Run it yourself

Investigates a firing or recently fired observability alert: finds the firing and replays the value it evaluated, decides whether the alert is chronic noise or a new problem, reuses any automatic investigation, then finds the cause (what changed, which host, version or route is over-represented) and returns a structured diagnosis with cause, scope and next steps.

Use this Skill: https://skilld.dev/gh/launchdarkly/ai-tooling/alert-investigation

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

metrics.md

≈557 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Alert investigation: metrics

Load this file for an OpenTelemetry metric alert (a threshold on metric_name values, a composite or formula alert). Load the observability/metrics guide too for the metric's type and unit.

Before you look for a cause

  • Replay the firing with the replay arguments from get-alert-firing-history. Confirm you see the value that crossed the threshold, then add compareTo: "1w".
  • Check the aggregator suits the metric. Max of a cumulative counter only ever grows, so it fires on uptime, not on a problem. Avg of system.cpu.utilization across its state and cpu attributes is meaningless; group by state. A Sum of a gauge across hosts grows with the host count. An alert built on the wrong aggregator is a finding: recommend the right one.
  • Look at the onset. Replay from two or three alert windows before the first firing with bucketCount 24 to 48: a sharp step, a slow drift, or a daily cycle that crosses the threshold every day?

Investigation shape

  1. Which series moved: group the replay by the metric's attributes (host.name, service_name, k8s.pod.name; list them with get-keys) with bucketCount: 1 and compareTo. One host or pod off its peers is the usual cause.
  2. What else that host or service reports: search-aggregations with productType: "metrics", environments and services (the metric's own values, from get-keys with productType: "metrics"), the slice filter and groupBy: "metric_name" lists its metrics, such as pool sizes, memory and queue depth.
  3. Tie it to requests: traces and error logs for the same service and host in the window (see investigate).
  4. What changed: get-changes for the service.

What goes in the diagnosis

  • What triggered: metric name, aggregator, threshold, the replayed value with its unit, and the group.
  • Verdict: new, chronic or misconfigured.
  • Likely cause: the series that moved and the records behind it (trace IDs, log templates).
  • Next steps: the fix, or the alert retune.

Common mistakes

  • Reporting the metric value without record-level evidence.
  • Reading a counter's Max as a level.
  • Quoting a value without its unit.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warningtoday3 checks · Risk SAFE
  • Gen Agent Trust Hubtoday

    The skill is designed to investigate observability alerts by querying a LaunchDarkly MCP server. It retrieves logs, traces, metrics, and session data to diagnose system failures. The primary security consideration is the ingestion of untrusted data from logs and error messages, which presents a surface for indirect prompt injection.

  • Sockettoday

    No alerts

  • Snyktoday

    Risk: MEDIUM · 2 issues

Signed by skilld at f0ee110. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 days ago
compatibility
Requires the remotely hosted LaunchDarkly MCP server
metadata
{
  "author": "launchdarkly",
  "version": "0.3.0"
}

README badge

README badge for launchdarkly/ai-tooling/alert-investigation