/pr-babysitter

Bot Patterns

Detection, severity, deduplication, and false-positive rules for PR review bots and human reviewers.

Contents

• Triage default: unlisted reviewers are findings
• Bot classification
• Shared identity: github-actions bot
• Active review bots
• Merge-gate bots
• Noise bots
• Check-only bots
• Human review patterns
• Severity normalization
• Deduplication
• False positive detection

Triage default: unlisted reviewers are findings

The tables below are a shortcut for stripping boilerplate and mapping severity. They are not an allowlist of who counts. Any author not listed, bot or human, is triaged as an active reviewer, and every one of its inline comments is read in full.

For an unrecognized author:

1. Treat each inline comment as a real finding at Major unless its own text says otherwise.
2. Take severity from whichever of these the body carries: a bold **<Word> Severity** line, a P1/P2/P3 or high/medium/low token (including inside a badge image URL), a coloured circle emoji, or the words critical, blocker, major, minor, nit.
3. Strip generically: HTML comments, <details> blocks, <sup> and <sub> footers, lines that are only a link image ("Fix in ...", "Open in ..."), Reviewed by ... for commit <sha> footers, "Comment @x review to trigger another review" invitations, "Was this helpful?" prompts. What survives is the finding.
4. Label it in the report as unlisted reviewer: <login> so the user can have it added here.

Noise requires a positive match on a detection marker below. Never classify an author as noise because it is missing from a table. A bot listed here as Noise or Check-only that posts an inline review comment is a stale row: triage the comment on its content and flag the drift in the report.

Bot classification

Classify by content first, then username. github-actions[bot] is a shared identity: some workflows are active reviewers, others pure noise.

Identity hints, never verdicts: a [bot] login suffix and GraphQL author { __typename } == "Bot" both suggest automation, but machine users such as developer-platform-actions are plain User logins, and github-actions[bot] is shared across unrelated workflows. Content decides.

Shared identity: github-actions bot

Match comment content to determine which workflow posted.

DangerJS: active reviewer

Detection:

• DangerID: danger-id- in an HTML comment at the top of the body
• HTML table with data-danger-table="true"
• footer Generated by :no_entry_sign: dangerJS

Severity: parse from the HTML comment metadata:

<!--
  0 failure:
  1 warning:  Multiple MFEs are...
  DangerID: danger-id-Danger;
-->

• failure count > 0 → major
• warning count > 0 → minor
• table cells use :warning: emoji for warnings

Body is an HTML table of actionable warnings; treat as real findings. Updates in place: the same comment ID gets new content on each push, so compare updated_at, not just the ID.

Schema compatibility checker: active reviewer

Detection:

• heading with a schema filename in <code> tags (e.g. ## <code>profileShareIntended_2-0-2.schema.json</code>)
• ### ⚠️ Warnings or ### 🔴 Errors section headers
• table columns: Field, Type, Status, Details

Severity markers in the Status column:

• 🟡 Warning → minor
• 🔴 Error → major (breaking changes)
• > 💡 **Tip**: blockquotes are informational, not findings

Event-lib RC trigger: noise

Detection: single-line comment starting with "Event-lib RC version is triggered", then a Buildkite URL.

Changeset releases: noise

Detection: comment body starts with "This PR was opened by the [Changesets release]" or contains a # Releases heading with version changelogs.

Empty-body approvals: noise (any automation identity)

Detection: a review with state APPROVED, an empty body, and no inline comments of its own, from any automation identity. Keyed on shape, not login: github-actions[bot], developer-platform-actions, and any other machine account qualify. Skip it as a finding, but record that it fired: it is often the evidence that a merge gate cleared.

An empty-body APPROVED that does have inline comments is not this pattern. Triage the inline comments.

Active review bots

For every bot here, findings live in the inline review comments, not in the review body. The body is a count or a summary. Never triage a bot on its review body alone, and never conclude a bot found nothing because its body reads like a header.

Codex (chatgpt-codex-connector[bot])

Detection:

• review body contains ### 💡 Codex Review and **Reviewed commit:** <sha>``
• inline comments open with a shields.io priority badge: **<sub><sub>![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)</sub></sub> <Title>**

Severity from the badge token in the image URL:

Structure: badge, then the bold title, then the finding in the following paragraph.

Strip: the badge markup and <sub> wrappers, "Here are some automated review suggestions for this pull request.", the <details> <summary>ℹ️ About Codex in GitHub</summary> block.

Rate limit, noise but not silence: a body that is only "You have reached your Codex usage limits", with no ### 💡 Codex Review heading and no inline comments. Skip it as a finding, and report once that Codex did not review this PR (usage limit), so the user knows a reviewer was silent rather than satisfied.

Devin (devin-ai-integration[bot])

Posts a PR review (state: COMMENTED); may inject a badge into the PR body. Findings arrive as inline comments on specific file lines, not the review-level body.

No findings (noise):

• review body starts with ## ✅ Devin Review: No Issues Found and no inline threads
• "View in Devin Review to see N additional findings" (paywalled teaser) with no inline detail → noise

Findings (active reviewer):

• review body reads **Devin Review** found N potential issue(s). → expect N inline comments. Fewer threads than N means the remainder is paywalled: report the gap rather than assuming N was covered
• each inline comment opens with metadata, then a 🔍 **<Title>** heading:

<!-- devin-review-comment {"id": "BUG_pr-review-job-<hash>_0001", "file_path": "...", "start_line": 78, "end_line": 78, "side": "RIGHT"} -->
🔍 **<Title>**

• that JSON is the authoritative anchor: file_path, start_line, end_line, and side hold even when the thread's line is null
• the id prefix classifies the kind: BUG_ → a defect, default Major; ANALYSIS_ → an observation, default Minor unless the text names a defect
• the emoji on the title line carries the severity, and takes precedence over the kind's default: 🔴 Critical, 🟠 Major, 🟡 Minor. 🔍 is a marker, not a severity
• kind and severity are independent, so both tokens can appear together and neither is redundant. A BUG_ titled 🟡 is a real defect at Minor
• downgrade to Minor when the language is advisory ("worth a follow-up", "pre-existing", "outside the PR diff")

Strip: the HTML comment, the 🔍 title prefix, trailing "Was this helpful?" reaction prompts.

Badge injection (always ignore): <!-- devin-review-badge-begin --> / <!-- devin-review-badge-end --> in the PR body, with <picture> elements linking to app.devin.ai. This is in the PR description, not a comment.

CI check: check-devin-approval gates merging on Devin's review. Not a comment; skip.

CodeRabbit (coderabbitai[bot])

If present, parse inline review comments. Severity markers (emoji in body):

Comment structure:

• multi-section with collapsible <details> blocks
• "Analysis chain": skip (internal reasoning)
• bold imperative summary: the finding
• "Proposed fix": diff block (```diff)
• "Prompt for AI Agents": extract for fix guidance

PR-level summary comments (walkthrough narratives, base64 metadata in HTML comments): skip, not findings.

Gemini Code Assist (gemini-code-assist[bot])

If present, parse inline review comments. Severity markers (SVG image refs):

Scan for gstatic.com/codereviewagent/ in image URLs.

Uses native ```suggestion blocks with the proposed fix. Summary comment starts with "Hello @author, I'm Gemini Code Assist!"; skip it.

Cursor Bugbot (cursor[bot])

Inline comments only, no summary body and no review verdict, so a PR with zero Bugbot comments means Bugbot found nothing (or has not run: the Cursor Bugbot check shows which). Severity is a bold line in the body:

Strip: the <sub> wrappers around the severity line, the "Fix in Cursor" / "Fix in Web" link line (cursor.com/agents), and the Bugbot Autofix is OFF / ON footer. A comment posted by the Autofix agent reporting a pushed fix is status, not a finding. Re-request a review by commenting bugbot run.

GitHub Copilot (copilot-pull-request-reviewer[bot])

Always a COMMENTED review, never APPROVED or CHANGES_REQUESTED, so its verdict is never a blocker. The review body is a summary of the PR; findings are inline, usually with a native ```suggestion block. Copilot marks no severity: default Major, downgrade to Minor when the text is advisory ("Consider...", "Optionally..."). Re-request with gh pr edit --add-reviewer @copilot.

Merge-gate bots

A comment that states a verdict about whether the PR may merge (auto-approval, merge freeze, release window, CODEOWNERS coverage) is a readiness-check input, not a finding. Record the verdict, never open a fix item from it, never reply to it, never resolve it.

The entries below are one organization's gates, kept as worked examples of the shape. A repo with its own gate bot gets a new entry here with the same fields: detection marker, verdict table, and what would flip the verdict.

Auto-approval assessment (linktree-stamp[bot])

Detection: an issue-level comment, not a review, containing <!-- stamp-assessment -->. Heading is ## ✅ Auto-Approval Assessment ([<sha>](...)) or ## 👀 Auto-Approval Assessment (...).

Edited in place on each commit: the comment ID never changes and only the body does, so a state comparison keyed on comment IDs sees nothing new. Compare updated_at, and compare the commit SHA in the heading against the head SHA to spot a stale assessment.

Carry these fields verbatim into the report when present: **Custom criteria:**, **Path criteria:**, **Change type:**, **Blast radius:**, **Sensitive domains:**.

### Reasoning bullets are context, not findings. Do not open fix items from them.

### <emoji> Path-specific review: <criteria> sections: 🟢 means that path review found nothing. Any other colour, or an explicit issue list, means treat the section body as findings and triage at Major.

Strip the > If you're unsure ... post in #ask-developer-platform footer.

The approving identity is separate: developer-platform-actions posts the empty-body APPROVED review once the gate clears. That review is noise as a finding and evidence as a gate.

mergefreeze

Merge gate, status only ("Ok to merge"). A freeze verdict blocks merging and is not fixable from code.

Noise bots

Always ignore, on a positive marker match only; no actionable findings:

Match the content marker, not the login: the Linear bot's login has already drifted once (from linear[bot]) while <!-- linear-linkback --> survived.

Removed from this table on purpose: chatgpt-codex-connector[bot] is an active reviewer above. Only its zero-finding and rate-limit bodies are noise.

Check-only bots

Appear in gh pr checks but have left no comments. Nothing to triage:

• Buildkite: CI pipeline checks (buildkite/{project}/{step})
• Telemetry service attributes: service metadata validation from blstrco/telemetry

Check-only means "has posted no comments on this PR", which is an observation about today, not a property of the bot. A review or an inline comment from any of these promotes it to active reviewer, triaged on content under the unlisted-reviewer procedure.

Human review patterns

An empty review body is the normal case, not an absence of content. A human COMMENTED review with an empty body means all of the content is in inline thread comments. Several empty-body reviews from the same person in a row are one review pass, not several: group them by author. Never report that a reviewer left no comments on the strength of an empty body.

Replies nested in bot threads are human comments. A thread's author is its first comment's author, so a human reply inside a devin-ai-integration[bot] thread sits under a bot's name. It carries full human weight: never deduplicated, never auto-resolved, never skipped because the thread's author is a bot. Classify per comment, not per thread.

Intent decides what handling means. Severity says how urgent; intent says what the reviewer wants back.

A question with no code change is not an ignored item. It goes in the plan's Questions section and gets a reply. Reporting "0 items to fix" while a reviewer is waiting on an answer is a triage failure, not a clean run.

Classify by review state and context for blocking status:

Human comments default to Major unless language or context signals otherwise.

Severity normalization

Unified four-level scale:

When multiple sources flag the same issue, use the highest severity.

Severity orders the fix queue. It never decides whether a comment gets read.

Deduplication

Multiple bots may flag the same issue on overlapping lines.

Grouping rule: comments on the same path within a 3-line range likely address one issue.

Within each group:

1. keep the highest-severity comment
2. prefer one with a suggestion block or diff (most actionable)
3. mark others ignore-duplicate, referencing the kept thread ID

Exceptions:

• never deduplicate human comments; each gets its own entry
• human + bot on the same issue: keep both (the human confirms the bot, raising confidence)
• a bot finding listing several LOCATIONS is one item with several sites. Do not split it, and do not treat the extra sites as duplicates
• a bot re-reviewing a new commit opens a fresh thread on the same lines while the old one goes isOutdated. Keep the newest, mark the older ignore-superseded, and quote its thread ID

False positive detection

Pre-existing code

The flagged line was not added or modified in this PR.

gh api "repos/{owner}/{repo}/pulls/{pr}/files" --paginate

Each file's patch has diff hunks. If the flagged line is not in an added (+) range, mark ignore-pre-existing.

Irrelevant file types

Bot comments on these are almost always false positives:

• .md, .json, .yaml, .yml config files (unless security-related)
• .lock files (auto-generated)
• .sql migration files (auto-generated)
• files with "auto-generated" or "DO NOT EDIT" headers

Convention contradictions

If a bot finding contradicts a rule in the project's CLAUDE.md or AGENTS.md, mark ignore-contradicts-conventions.

Outdated threads

isOutdated == true is not grounds to ignore anything. It means GitHub can no longer map the thread onto the current diff, which is also why line comes back null. Recover the anchor first via the ladder in the GitHub API reference, read the current file, and mark ignore-outdated only when the flagged construct is genuinely gone. An outdated thread carrying a human reply is never ignored.