Index of every page on this site: llms.txt
All skills
datadog-labs avatar

/unblock-pr

@618fae3 official
Run it yourself

Load when investigating a failing PR CI pipeline or checking PR health. Attributes each CI failure as flaky, infra, or regression, proposes a targeted action, and reports code coverage and quality/security status.

Use this Skill: https://skilld.dev/gh/datadog-labs/agent-skills/unblock-pr

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesfailure-classification.md

≈1.2k tokens on demand. Your agent reads this file only when SKILL.md points to it.

PR CI Failure Classification

Use this reference during STEP 3 of the unblock-pr workflow. In pup mode, translate tool calls using the main skill’s Tool Reference.

Threshold note: The STEP 3 table in the main skill is authoritative. This reference uses the same thresholds: ≥3 branches for flaky (cross-branch intermittent pattern), ≥3 for infra (widespread pre-existing failures). The term "widespread" in the infra heuristics below refers to the same ≥3-branch criterion.


Primary Signal: @error_classification Fields

Check job-level pipeline events for these attributes before running blame guard queries. When present, they provide the most direct classification signal:

Attribute Description
@error_classification.domain High-level failure domain (e.g. test_failure, infrastructure, runner)
@error_classification.type Specific failure type within the domain
@error.domain Error category (may be present even without full classification)
@error.subdomain More specific error category

If @error_classification.domain indicates an infrastructure or runner issue → classify as infra without further queries. If @error_classification.domain indicates a test failure → proceed to flaky check and blame guard.


Classification Heuristics

flaky

Signals:

  • Test failure AND test appears in get_datadog_flaky_tests with flaky_test_state:active
  • Job failing on ≥3 branches AND NOT failing on the default branch (intermittent, cross-branch pattern)
  • @error_classification.type indicates a non-deterministic failure category

Confirm with:

Tool: get_datadog_flaky_tests
query: @test.name:"<test-name>" @test.suite:"<suite>" flaky_test_state:active @git.repository.id_v2:"<repo>"

Action: Load triage-flaky-test for agent-native fix attempt and quarantine option.

Attribution note: Failure history takes precedence — a test failing on the default branch or across multiple branches is pre-existing (quarantine is appropriate), even if this PR touches its file; only a genuinely branch-local flake is author-owned and steered to fix. See STEP 5 → flaky in the main skill for the exact hint rule, and triage-flaky-test's STEP 2.5 for the authoritative attribution definition.


infra

Signals:

  • @error_classification.domain indicates runner, infrastructure, or environment failure
  • Job is a non-test step (build, lint, deploy, docker push, etc.)
  • Same job has been failing on the default branch with count > 0 in the 24h window
  • High blast radius (≥3 branches) with no correlation to test failures

Confirm with:

Tool: aggregate_datadog_ci_pipeline_events
query: @ci.status:error @ci.job.name:"<job>" @git.branch:<default-branch> @git.repository.id_v2:"<repo>"
ci_level: job
aggregation: count
from: now-24h

Action: Follow STEP 5 in the main skill: retry only transient failures, use the available provider/backend retry path, explain the all-failed-jobs scope for GitHub, and verify the async result. Pre-existing deterministic failures need an upstream fix.


regression

Signals:

  • Job failing ONLY on this PR's branch (blast radius = 1)
  • Job was NOT failing on the default branch in the same time window
  • @error_classification absent or indicates test failure on this branch specifically
  • Failure correlates with files changed in this PR

Confirm with:

  • Default branch check returns count = 0
  • Blast radius check shows only this PR's branch

Action: Prompt user to investigate their code changes. Suggest narrowing to changed files.


unknown

Signals:

  • Single data point (only 1 pipeline run available)
  • Missing test events for a failing test-runner job
  • Contradictory evidence (e.g. failing on default branch AND only on this branch)
  • @error_classification absent AND blame guard returned insufficient data (< 3 pipeline events)

Action: Suggest checking raw job logs via Datadog CI pipeline view or the CI provider's UI.


Edge Cases

No test events for a failing test job: The test job may have failed before producing test events (e.g. compilation error, setup failure). Classify as unknown and recommend checking raw logs rather than guessing.

Default branch derivation fails: If git symbolic-ref refs/remotes/origin/HEAD is unavailable, use main as fallback. Mention this assumption in the triage brief evidence field.

Very new repository (< 24h of CI data): Insufficient blame guard data. Classify as unknown and note the limited history window.

Multiple jobs failing with different classifications: Report each job separately in the triage brief. The overall summary should list counts per classification.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warning16d3 checks · Risk SAFE
  • Gen Agent Trust Hub17d

    The skill is a CI/CD triage utility from Datadog Labs that automates the investigation of pipeline failures. It uses common developer tools (git, gh, pup) to gather data and classify failures. While it processes branch names and repository URLs that could serve as injection vectors, it includes normalization steps and follows standard operational patterns for this use case.

  • Socket16d

    No alerts

  • Snyk17d

    Risk: MEDIUM · 1 issue

Signed by skilld at 618fae3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 days ago.

Activeupdated 3 weeks ago
Other metadata
metadata
{
  "version": "1.1.0",
  "author": "datadog-labs",
  "repository": "https://github.com/datadog-labs/agent-skills",
  "tags": "datadog,ci,cicd,flaky,flaky-tests,pipeline",
  "alwaysApply": "false"
}
  • datadog
  • ci
  • cicd
  • flaky-tests
  • pipeline
  • pr
  • triage

README badge

README badge for datadog-labs/agent-skills/unblock-pr

Analyzes a failing PR CI pipeline to classify each job failure as flaky, infrastructure, or regression, then proposes targeted actions like retrying transient failures or investigating code changes. Integrates with Datadog CI Visibility and GitHub Actions, and can invoke the triage-flaky-test skill for deeper flaky test investigation.

Generated from the current SKILL.md.

What do I need to install to use this skill?
You need either the Datadog MCP server connected or the pup CLI installed and authenticated. The skill also requires the dd-pup and triage-flaky-test skills to be available.
Does this skill work with CI providers other than GitHub Actions?
Yes. The skill detects the CI provider from event data and handles GitHub Actions, GitLab, and other providers. For non-GitHub providers, it will direct you to the provider UI rather than attempt an automated retry.
What does this skill actually do with a failing PR?
It queries CI and test events from the past 24 hours, attributes each failure as flaky (intermittent), infra (pre-existing), or regression (introduced by this PR), and proposes a targeted action — either deep investigation, quarantine, retry, or root cause analysis.
Can this skill fix flaky tests automatically?
For flaky failures, it invokes the triage-flaky-test skill, which attempts an agent-native fix using flaky category and stack trace data. If a quick fix isn't possible, it proposes quarantine.
Does this skill report code quality and security metrics?
Only in MCP mode — it fetches code quality and security status from PR insights. In pup mode, these metrics show as 'No data available'.

Generated from the current SKILL.md. These answers refresh after source changes.