Browser-Sniff Capture Implementation
When to read: This file is referenced by
06-browser-sniff-gateof the printing-press skill. Read it when the user approves temporary browser discovery (browser-use, agent-browser, or manual HAR capture of live site traffic).Context: This file documents what happens AFTER
06-browser-sniff-gatedecides to browser-sniff. The decision itself — approve, decline, or silent-skip — is recorded in$PRESS_RUNSTATE/runs/$RUN_ID/browser-browser-sniff-gate.jsonby that phase before this reference is loaded.08-ecosystem-absorb-gaterefuses to proceed without that marker file. See 06-browser-sniff-gate.md "Enforcement: the browser-browser-sniff-gate.json marker file" for the contract.Browser discovery is a temporary generation-time aid. It exists to learn URLs, methods, request bodies, persisted GraphQL hashes, BFF envelopes, auth/header construction, response shapes, and replayability. It is not permission to generate a printed CLI that keeps a browser open for normal commands.
Cardinal Rules
Prefer browser-use CLI mode for capture, but keep valid fallbacks. Use browser-use CLI mode when available because it gives stable open/eval/scroll control and response interception without an LLM key. If browser-use is unavailable or incompatible, the valid fallback set is:
agent-browser(when it can produce equivalent network capture artifacts), the Claude chrome-MCP (mcp__claude-in-chrome__*— drives the user's existing Chrome session via the browser extension; see Step 2e), computer-use as a visual-feedback aid for the manual-HAR flow (mcp__computer-use__*— read-only on browsers, used to take screenshots and guide the user; see Step 1d's manual-HAR expansion), or asking the user for a manual DevTools HAR. Do NOT substitute curl probing, JS bundle grepping, or agent-browser auto-connect alone for an approved browser capture. Detection of which fallbacks are actually available happens in Step 1; menu composition for failure-recovery selection is in Step 2c.5. Rule 5 (replayability) applies to all of these — fallback shape does not relax the success criterion.Do NOT skip auth discovery when the session expires. (Only applies when
AUTH_SESSION_AVAILABLE=true— the user confirmed they're logged in.) If a Chrome profile loads but the session has expired (login page visible instead of account page), offer headed login as a fallback. Never proceed without auth just because the profile session was stale. For anonymous sniffs (no auth context), this rule does not apply.Use click-based SPA navigation after installing interceptors.
browser-use opentriggers a full page reload which resets the JS context and destroys fetch/XHR interceptors. After installing interceptors, navigate by clicking links (browser-use eval "document.querySelector('a[href*=account]').click()"orbrowser-use click). Only usebrowser-use openfor the first page load or when you need to re-install interceptors.Run
cli-printing-press probe-reachabilitybefore announcing any browser escalation, and don't expose transport tiers to the user as peer choices. If research or preflight saw Cloudflare/Vercel/WAF/DataDome/PerimeterX/CAPTCHA evidence, the first action is the no-browser probe — not a Chrome-attach prompt and not a transport-tier menu like "Browser-sniff + clearance cookie / Browser-sniff Surf-only / HAR / Hold". Intent menus are fine (yes/no, browser-sniff or pivot, etc.); the wrong shape is forcing the user to pick between Surf vs cookie vs full browser, which is the classifier's job. Many passive challenges (Vercel TLS-fingerprint mitigation, lighter Cloudflare gates) clear with Surf alone, no cookie, no setup. Ifprobe-reachabilityreturnsmode: browser_http, the printed CLI will ship Chrome-compatible transport with zero clearance-cookie capture — runtime is settled silently. (Browser-sniff for endpoint discovery is a separate decision handled by Phase 1.7's normal matrix; if that matrix says to ask, the existing intent-level prompts already disclose Chrome attach as a possibility — that's the right place for that consent, not bundled into a transport-tier menu.) Only when the probe returnsbrowser_clearance_httporunknownshould you tell the user direct HTTP is blocked and proceed with a real browser capture. Do NOT replace the target with RSS/docs/official API or ask for a smaller CLI shape until after browser capture has failed by the criteria below.Replayability is the success criterion. A browser capture succeeds only when it produces a shippable surface: replayable API calls, persisted-query registry entries, browser-clearance cookies that can be imported and replayed, or structured HTML/SSR/RSS/JSON-LD extraction targets. If the only observed path requires live page-context execution, report HOLD or return to discovery for a lighter surface. Do not continue as if resident browser transport is acceptable.
If user approves browser-sniff
Browser-Sniff Pacing
When making API calls during browser-sniff (browser-use eval, fetch, or direct HTTP requests), apply adaptive pacing to avoid rate limits:
- Start conservative: Wait 1 second between API calls
- Ramp up on success: After 5 consecutive successful calls, reduce the delay by 20% (minimum 0.3 seconds)
- Back off on 429: If you get a rate-limited response (HTTP 429), immediately double the delay and log: "Rate limited — increasing delay to Xs"
- Hard stop on repeated 429s: If you hit 3 consecutive 429s, pause for 30 seconds before continuing
- Never abort: Rate limiting during browser-sniff is recoverable. Always continue after the backoff — do not abort discovery due to rate limits
Track the current delay mentally. Report the effective rate when summarizing browser-sniff results: "Sniffed N endpoints at ~X req/s effective rate."
Proxy Pattern Detection
After capturing API traffic, check if the API uses a proxy-envelope pattern:
- Same-URL signal: If all captured XHR/fetch URLs resolve to the same path (e.g., all calls go to
_api/ws/proxy), the API likely uses a proxy pattern - Envelope signal: If intercepted request bodies contain
service,method, andpathkeys (or similar routing fields), it's a proxy-envelope - Confirmation: If both signals are present, classify as
client_pattern: proxy-envelope
When a proxy pattern is detected:
- Note the proxy URL (it becomes the spec's
servers[0].url) - Extract the service routing from request bodies — build an
x-proxy-routesmap of path prefixes to service names - Write
x-proxy-routesinto the generated spec'sinfoextensions:info: x-proxy-routes: /v1/api/: publishing /search-all: search - Pass
--client-pattern proxy-envelopeto the generate command in Phase 2
Step 1: Detect capture tools
Check which browser automation tools are available:
# Prefer browser-use (CLI-driven, Performance API collection).
# Use `command -v` only. Do NOT use `uvx browser-use --help` as a fallback
# probe: when uvx exists but browser-use doesn't, that command silently
# downloads and caches the package, which is an unconsented install.
# The capture commands below invoke `browser-use` directly (not via uvx),
# so a uvx-cache-only state would lie to the detection.
if command -v browser-use >/dev/null 2>&1; then
SNIFF_BACKEND="browser-use"
# Fall back to agent-browser only if it can provide equivalent network capture artifacts.
elif command -v agent-browser >/dev/null 2>&1; then
SNIFF_BACKEND="agent-browser"
else
SNIFF_BACKEND="none"
fi
# Check if browser-use can run in autonomous agent mode (optional, not required)
BROWSER_USE_HAS_LLM=false
if [ -n "$ANTHROPIC_API_KEY" ] || [ -n "$OPENAI_API_KEY" ] || [ -n "$BROWSER_USE_API_KEY" ]; then
BROWSER_USE_HAS_LLM=true
fiTreat command -v agent-browser as sufficient only when agent-browser was already present before this step, or when this run just installed it and the user-run agent-browser install step completed. This detection step does not launch agent-browser to prove browser-cache readiness for pre-existing installs. If this run attempted the package-manager install but the post-install step was declined, failed, or unclear, set SNIFF_BACKEND="none" and fall back to manual HAR; do not let a second detection pass select the half-installed binary. If a pre-existing agent-browser later reports missing browser binaries, surface ! agent-browser install and use a fallback backend until the user confirms it completed.
If a tool is found, report: "Using <tool> for temporary traffic capture during generation (CLI-driven mode — no LLM key needed)." and proceed to Step 1c to verify compatibility.
Important: browser-use has two modes: autonomous Agent mode (requires an LLM API key like ANTHROPIC_API_KEY) and CLI mode (open/eval/scroll — no key needed). Always use CLI mode for browser-sniff. It is more reliable, version-stable, and does not require the user to provide an additional API key. Do NOT attempt to use browser-use's Python Agent class — it requires an LLM key that may not be available.
Also detect: optional MCP-driven fallback backends. Two additional capture options exist when the runtime exposes them — they enter only on failure-recovery in Step 2c.5 or as opt-in choices in Step 1b's install picker, never as defaults. Detection is agent-prose, not shell-probe — the agent inspects its own available and deferred tool lists (visible in system reminders / the deferred-tool block / ToolSearch catalog) and asserts the flags inline in its reasoning. Bash blocks cannot read the agent's own tool registry; do not write command -v mcp__claude-in-chrome__ style probes.
Inspect your tool catalog now and assert these flags in your reasoning, not as shell vars. They are read by Step 2c.5 (recovery menu composition) and Step 2e (chrome-MCP capture playbook) later in this same conversation turn.
- If
mcp__claude-in-chrome__*tools appear in your available or deferred tool list, setCHROME_MCP_AVAILABLE=true. Otherwisefalse.- If
mcp__computer-use__*tools appear in your available or deferred tool list, setCOMPUTER_USE_AVAILABLE=true. Otherwisefalse.- The probe is intentionally cheap — actual schema loading via
ToolSearchis deferred until the user picks the option from a menu.- On platforms that don't expose a deferred-tool list (non-Claude-Code targets running this skill via plugin install), the agent observes neither MCP and both flags default to
false. Downstream behavior is unchanged from today's.
If either MCP flag is true, extend the status report:
"Using <tool> for traffic capture. Fallbacks available: chrome-MCP, computer-use." (list whichever flags are true)
Step 1b: Install capture tool (fallback — preflight should have prompted first)
Preflight (references/setup-checks.md section 6) offers to install browser-use and agent-browser on every run, so most users arrive at Step 1 with one or both already installed. This step is a fallback for the case where the user declined the preflight prompt and the current run actually needs a browser backend.
If neither tool is installed, offer to install via AskUserQuestion. Do not install automatically:
"No browser automation tool found. I need one to temporarily inspect the live site during generation. Which would you like to install?"
Options:
- Install browser-use (Recommended) — "CLI-driven browser automation for generation-time capture. Requires Python."
- Install agent-browser — "Alternative capture backend when it can provide network artifacts. Requires Node.js."
- Skip — I'll provide a HAR manually — "Export a HAR yourself from browser DevTools and provide the path."
If user picks browser-use:
# Detect Python package manager. Use `uv tool install` (not `uv pip install`):
# `uv pip install` targets the active venv and won't put the binary in PATH
# outside it; `uv tool install` creates an isolated env and symlinks the
# entry-point into `~/.local/bin`.
if command -v uv >/dev/null 2>&1; then
uv tool install browser-use
elif command -v pip >/dev/null 2>&1; then
pip install browser-use
else
echo "Neither uv nor pip found. Install Python first: https://www.python.org/downloads/"
# Fall back to asking about agent-browser or manual HAR
fiAfter install, re-run detection. If browser-use is now available, set SNIFF_BACKEND="browser-use" and proceed to Step 1c. If install failed, show the error and offer agent-browser as alternative or fall back to manual HAR.
If user picks agent-browser:
# Detect Node.js package manager
if command -v brew >/dev/null 2>&1; then
brew install agent-browser
elif command -v npm >/dev/null 2>&1; then
npm install -g agent-browser
else
echo "Neither brew nor npm found. Install Node.js first: https://nodejs.org/"
# Fall back to manual HAR
fiAfter the brew or npm install succeeds, use the same post-install rule as preflight (references/setup-checks.md section 6): complete agent-browser's browser-binary setup as a user-run step:
! agent-browser installThe leading ! is intentional: surface the command for the user to run manually instead of invoking it through the agent's shell tool. Do not treat command -v agent-browser alone as a complete install after the package-manager step; the agent-browser install step must complete before browser-sniff flows rely on it. If the user declines the manual step or completion is unclear, do not run it yourself; fall back to manual HAR. If agent-browser was already installed before this Step 1b fallback, do not rerun redundant setup here.
After install, re-run detection. If agent-browser is now available and the user-run agent-browser install step completed, set SNIFF_BACKEND="agent-browser" and proceed to Step 1c. If install failed, show the error and fall back to manual HAR.
If user picks manual HAR, ask the user for a HAR file path and skip to Step 3.
Step 1c: Verify capture tool compatibility
After detection (Step 1) or installation (Step 1b), verify the installed version supports the CLI commands the browser-sniff process needs.
For browser-use — The CLI 2.0 commands (open, eval, scroll, close) all shipped in v0.12.3. Versions before that have an incomplete or experimental CLI that won't work for browser-sniff.
# browser-use has no --version flag; get version from pip metadata
BROWSER_USE_VERSION=$(pip show browser-use 2>/dev/null | grep -i '^Version:' | awk '{print $2}')
MIN_BROWSER_USE="0.12.3"
# Compare versions (lexicographic sort works for dotted semver)
if printf '%s\n' "$MIN_BROWSER_USE" "$BROWSER_USE_VERSION" | sort -V | head -1 | grep -qx "$MIN_BROWSER_USE"; then
BROWSER_USE_COMPAT=true
else
BROWSER_USE_COMPAT=false
fiFor agent-browser — check that the network subcommand exists (needed for HAR capture):
AGENT_BROWSER_VERSION=$(agent-browser --version 2>&1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)
if agent-browser network --help >/dev/null 2>&1; then
AGENT_BROWSER_COMPAT=true
else
AGENT_BROWSER_COMPAT=false
fiIf the selected tool fails the compatibility check, offer to upgrade via AskUserQuestion:
"Found <tool> v<version>, but browser-sniff requires v<min-version>+ for CLI capture commands. Would you like to upgrade?"
Options:
- Yes — upgrade <tool> — runs the appropriate upgrade command (see below)
- Try <other-tool> instead — switch to the other backend (install it if needed)
- Skip — I'll provide a HAR manually
Upgrade commands:
- browser-use:
uv pip install --upgrade browser-use(ifuvavailable) orpip install --upgrade browser-use - agent-browser:
brew upgrade agent-browser(if brew-installed) ornpm update -g agent-browser
After upgrade, re-check the version. If the upgrade resolves the issue, proceed to Step 2. If it doesn't, offer the next fallback (other tool or manual HAR).
Do NOT upgrade automatically. Always ask permission first — upgrading packages can have side effects on the user's environment.
If the tool passes the version check, proceed to Step 1d (if authenticated browser-sniff) or Step 2a/2b (if anonymous browser-sniff).
Step 1d: Session Transfer (authenticated browser-sniff only)
This step only runs when the user chose "authenticated browser-sniff" (from Phase 1.7's sniff-as-primary or sniff-as-enrichment prompts, or when AUTH_SESSION_AVAILABLE=true and the user confirmed).
Situation detection:
CHROME_RUNNING=false
if pgrep -x "Google Chrome" >/dev/null 2>&1; then
CHROME_RUNNING=true
fiWhen Chrome IS running, use agent-browser to grab cookies, then ask the user to quit Chrome so browser-use can load the profile for capture:
Present via AskUserQuestion:
"Chrome is running. I'll grab your cookies, then need you to quit Chrome so I can browser-sniff with full page access."
- Grab session, then quit Chrome (Recommended) — "I save your cookies via agent-browser, you quit Chrome, then I browser-sniff with browser-use using your profile. Full DOM access."
- Log in within a new browser window — "I'll open a visible browser. You log in, then I browser-sniff."
- I'll export a HAR file — "You browse the site in DevTools and export the HAR. I use it for discovery, then keep only surfaces that pass lightweight replayability checks."
For option 1 (save-then-restore):
IMPORTANT: --auto-connect, --state, --profile, and --headed are daemon launch options in agent-browser. They only take effect when starting a new daemon. You MUST close the daemon between save and load.
# Grab cookies from running Chrome. $SESSION_STATE_FILE lives outside
# $DISCOVERY_DIR (initialized in 02-run-initialization) so the
# Phase 5.5 `cp -r "$DISCOVERY_DIR"` cannot pick it up.
agent-browser --auto-connect state save "$SESSION_STATE_FILE" 2>&1
# Close the auto-connect daemon so --state can start a fresh one
agent-browser close 2>&1
# Start a new headless daemon with the saved auth state
agent-browser --state "$SESSION_STATE_FILE" open <url>If auto-connect fails (no debug port), explain: "Chrome doesn't have remote debugging enabled. Quit Chrome and relaunch with --remote-debugging-port=9222, or pick option 2."
For option 1 after cookies are saved and Chrome is quit:
# Start browser-use with the Chrome profile (has all saved cookies/logins)
browser-use --profile "Default" open <url>When Chrome is NOT running, prefer browser-use (loads real Chrome profile with all cookies):
Present via AskUserQuestion:
"Chrome isn't running. I can load your Chrome profile directly — all your saved logins will be available."
- Use your Chrome profile (Recommended, requires browser-use) — "Loads your real Chrome profile. Zero setup."
- Log in within a new browser window — "I'll open a visible browser. You log in, then I browser-sniff."
- I'll export a HAR file — "I use the HAR for discovery, then keep only replayable HTTP/HTML/RSS/SSR/API surfaces in the printed CLI."
For option 1 (browser-use profile reuse):
browser-use open <url> --profile "Default"If browser-use is not available, fall back to agent-browser headed login.
If Chrome profile lock error occurs (Chrome is actually running): "Chrome's profile is locked. Quit Chrome first, or switch to option 2."
Session transfer vs capture are separate concerns. Use agent-browser for session transfer only (grabbing cookies from a running Chrome). Always use browser-use for the actual capture (Steps 2a.*) because it has full DOM access via eval, scroll, click, and snapshot. Agent-browser's auto-connect mode cannot access the DOM or run eval — it can navigate and record HAR but cannot interact with pages.
Recommended flow when Chrome IS running:
- Use agent-browser
--auto-connect state saveto grab cookies - Close agent-browser daemon
- Ask user to quit Chrome
- Start browser-use
--profile "Default"for capture (loads the same cookies via the Chrome profile)
When Chrome is NOT running:
- Use browser-use
--profile "Default"directly for both session and capture
For headed login (option 2 with either tool):
# agent-browser
agent-browser --headed --session-name "<api>-auth" open <login-url>
# or browser-use
browser-use open <login-url> --headed --session "<api>-auth"Instruct the user: "A browser window is open. Please log in to <site>. Let me know when you're done."
After login, save state:
agent-browser state save "$SESSION_STATE_FILE"Close the headed browser and restart headless with the saved state.
For HAR export (option 3): Guide the user through the DevTools HAR-export flow. Make clear that a HAR is discovery input, not a promise that every captured HTML/XHR route becomes a printed CLI command. After analyzing the HAR, keep only surfaces that replay through lightweight HTTP/Chrome-compatible HTTP, browser-clearance cookie import plus replay, or structured HTML/SSR/RSS extraction. If the HAR only proves live page-context execution works, HOLD or pivot scope.
Manual HAR body capture pitfall. Chrome can export page responses from disk cache as 206 partial-content entries with empty response.content.text, even when the user chose a HAR-with-content export. If browser-sniff analysis reports many 206 entries or missing bodies, tell the user the capture did not preserve response bodies and give the fix directly: in DevTools > Network, check Disable cache, then hard-reload each page while DevTools stays open before exporting the HAR. If Chrome still omits bodies, ask for a Firefox HAR export instead; Firefox's HAR export is more reliable for preserving page bodies.
Chrome 147+ DevTools HAR export — concrete instructions. Recent Chrome versions (147+) removed "Save all as HAR with content" from the right-click menu in the Network panel. The download-arrow icon at the top of the Network panel is now the only stable export path. Walk the user through these steps in order — they are the steps a user got stuck on in a recent session, so the language is deliberately literal:
- Open DevTools.
Cmd+Option+I(macOS) orCtrl+Shift+I(Windows/Linux). If DevTools is already open but on the wrong tab, the next step covers it. - Switch to the Network panel. It is in the top tab strip alongside Elements, Console, Sources, Performance. If DevTools is narrow, the Network tab may be hidden behind a
>>overflow chevron at the right end of the tab strip — click>>and pick Network. Do not pick "Recorder" — that is a different panel. - Confirm recording is on. A red dot at the top-left of the Network panel means recording is on. If it is gray/black, click it once to enable.
- Check "Preserve log" and "Disable cache" — both are checkboxes in the Network panel toolbar. Preserve log keeps records across navigations; disable cache forces fresh requests so the HAR contains real network activity and page response bodies.
- Clear any prior requests. Click the 🚫 (clear / no-entry) icon in the toolbar to start with an empty log.
- Hard-reload each page, then reproduce the user flow on the target site — hard-reload with DevTools still open before interacting so Chrome records full
200responses instead of cached206bodies. Then navigate, click into the section the printed CLI needs, scroll, interact. Wait for network activity to settle between actions. - Export the HAR. Click the download-arrow icon at the top-left of the Network panel toolbar (between the upload-arrow icon
↑and the record/clear icons — it looks like a↓arrow with a horizontal bar underneath). A macOS/Windows save dialog opens. Save as<api>-capture.harsomewhere accessible like~/Downloads/. - Tell the agent the path. The agent runs
cli-printing-press browser-sniff --har <path>next.
Computer-use visual-feedback-loop (only when COMPUTER_USE_AVAILABLE=true). Computer-use cannot click or type into Chrome (browsers are tier-"read" — visible in screenshots, but input is blocked). Its value here is closing the loop with the user when text instructions get them stuck. Pattern:
- Before screenshotting, instruct the user to collapse the Network panel detail pane so only the request list is visible. The detail pane shows full request and response headers including
AuthorizationandCookie; collapsing it before each screenshot keeps credentials out of the captured PNG. (Click the×on any open request detail, or click in the request list to deselect.) - Take a screenshot via
mcp__computer-use__screenshotat each instruction checkpoint (after step 1, after step 2, after step 7). - Display the screenshot inline AND describe what the agent sees in 1-2 sentences. This is mandatory — silent storage helps no one. Pattern:
Readthe saved PNG path so the image shows in the response, then say something like "I see your DevTools is on the Recorder tab, not Network — click>>in the tab strip and pick Network." The screenshot becomes part of the agent's reasoning AND the user-facing feedback. - Save screenshots to
$DISCOVERY_DIR/devtools-help-*.pngduring the session so they're available for inline display and so manuscripts archiving can clean them up. - Phase 5.5 cleanup: the archive-time cleanup must explicitly delete
$DISCOVERY_DIR/devtools-help-*.png— these are ephemeral debug aids, not durable artifacts, and the text-based credential scrubber insecret-protection.mdcannot redact PNG contents. Make sure thecleanupblock in the run wrap-up includesrm -f "$DISCOVERY_DIR"/devtools-help-*.png 2>/dev/nullor equivalent.
If COMPUTER_USE_AVAILABLE=false, the text-only instructions above stand on their own. Do not skip them just because computer-use isn't available — the click path itself is the load-bearing fix.
After any session transfer method, verify cookies transferred before proceeding:
# Verify auth cookies are present for the target domain
COOKIES=$(agent-browser cookies get --json 2>/dev/null)
if echo "$COOKIES" | grep -q "<target-domain>"; then
echo "Session transfer verified — found <target-domain> cookies."
else
echo "WARNING: No <target-domain> cookies found."
fiIf no target-domain cookies are found, present via AskUserQuestion:
"Session transfer failed — no
<target-domain>cookies found in the browser. The browser-sniff would run unauthenticated."
- Log in manually — "I'll open a headed browser. You log in, then I browser-sniff."
- Continue without auth — "Browser-Sniff only public endpoints"
- Provide HAR manually — "Export a HAR yourself from browser DevTools"
After loading a Chrome profile, also verify the session is actually active on the target site. Cookies may exist but be expired:
# Navigate to the site and check for login indicators
browser-use eval "var login=document.querySelector('a[href*=login],a[href*=signin],[class*=sign-in],[class*=login-btn]');
var account=document.querySelector('a[href*=account],a[href*=profile],[class*=logged-in],[class*=user-menu]');
login && !account ? 'SESSION_EXPIRED' : account ? 'SESSION_ACTIVE' : 'UNKNOWN'"If the result is SESSION_EXPIRED (login link visible, no account link), the profile cookies have expired. Present via AskUserQuestion:
"Your browser session for
<site>has expired (login page visible). I need a fresh login to discover authenticated endpoints."
- Open headed browser to log in (Recommended) — "I'll open a visible browser. You log in, then I continue the browser-sniff."
- Continue without auth — "Browser-Sniff only public endpoints"
Do NOT silently proceed without auth when the session has expired. The authenticated surface is often the most valuable part of the API (order history, rewards, saved data).
If cookies are verified, proceed to Steps 2a/2b capture flow with the authenticated session loaded. The session state file is stored at $SESSION_STATE_FILE (under ${TMPDIR:-/tmp}/printing-press-$(id -u)/session/$RUN_ID/, outside $DISCOVERY_DIR, so it cannot reach archived manuscripts).
Step 2a.0: Direct-API-probe fallback (try before browser-use when WAF-protected)
When probe-reachability returns mode: browser_http (Cloudflare, Vercel, or another WAF challenge in front of the site), browser-use can still get blocked at runtime even though surf would clear the challenge. Before launching browser-use, attempt a direct curl probe with a Chrome User-Agent to see whether the site's API surface answers without a resident browser.
This is most likely to succeed against:
- Sites with a discoverable proxy-envelope endpoint (a single internal route such as
/_api/ws/proxy,/api/graphql, or/internal/rpcthat fronts the public API). - Sites whose public API responds to a Chrome UA without JS challenge (the WAF gates HTML pages but exempts
/api/*paths).
Probe pattern:
# Pick a candidate endpoint surfaced by the Phase 1 research brief or
# the proxy-envelope detection step above.
PROBE_URL='https://<site>/<candidate-api-path>'
curl -s -o /tmp/probe-response.json -w '%{http_code}\n' \
-A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36' \
-H 'Accept: application/json' \
"$PROBE_URL"Decision criteria:
- HTTP 200 with structured JSON — the direct probe is viable. Capture a handful of representative endpoint responses to
$DISCOVERY_DIR/direct-probe-*.json, then proceed to Step 2a or fall through to Step 2b (manual HAR via DevTools) for the structured browser-sniff capture. Do not skip the structured capture —cli-printing-press browser-sniffneeds a HAR or enriched-capture JSON, not loose curl responses, and the replayability check still has to run against the captured envelope. - HTTP 403/429 with a Cloudflare challenge body (
<title>Just a moment...</title>,cf_chl_opt, Vercel/Akamai equivalents) — the WAF is blocking direct probes. Fall through to browser-use; the captured surface will need Surf transport and possibly a clearance-cookie step. - HTTP 401/403 with a structured auth error (
{"error":"unauthenticated"},WWW-Authenticate: Bearer) — direct probing works but the path is auth-only. Document the path in the brief and route to the authenticated-flow capture.
The replayability check still applies regardless of probe outcome: any endpoint discovered through direct probing must round-trip through surf with the same Chrome TLS fingerprint the printed CLI will ship, or the captured URLs are unusable in production.
Example: a public read-only catalog site fronted by Cloudflare exposes /_api/ws/proxy (the internal proxy-envelope) and answers a Chrome-UA POST with the right service/method/path body. The direct probe confirmed the envelope shape and request fields in seconds; the structured browser-sniff still ran via DevTools HAR to capture the full set of paths the proxy fronts.
Step 2a: browser-use CLI capture (preferred)
Claude drives browser-use directly via CLI commands — no LLM key needed, no Python API versioning issues. Uses the browser's native Performance API to collect API endpoint URLs from each page.
IMPORTANT: Run the page collection loop in foreground, not background. The loop takes ~60-90 seconds for 10-15 pages. Background execution has unreliable output capture for shell functions that call browser-use. Always run this inline.
browser-use eval returns synchronous values only
The browser-use eval <expr> CLI returns only synchronous JSON-serializable values. Any expression whose last value is a Promise returns None, even though the Promise may fire correctly in the background. This is a common trap when probing authenticated endpoints via inline fetch() from the page context.
Wrong (returns None before the Promise resolves):
browser-use eval "fetch('/api/cookbook', {credentials:'include'}).then(r => r.text())"
# result: None (looks like the fetch failed, but it didn't)Right (kick off the fetch, store the result in a window var, read back on a later eval):
# Step 1: kick off
browser-use eval "window.__probe = 'pending'; fetch('/api/cookbook', {credentials:'include'}).then(r => r.text()).then(t => { window.__probe = t.slice(0, 3000); }).catch(e => { window.__probe = 'ERR: ' + e; }); 'kicked'"
# Step 2: read back (can be immediate if you just want to confirm the interceptor ran;
# do one or two more evals as no-ops to let the Promise resolve)
browser-use eval "1"
browser-use eval "window.__probe"The pattern is safe because subsequent eval calls run in the same JS context — browser-use keeps the tab open — so window state persists between calls.
When the stored value reads as "pending", the Promise hasn't resolved yet. Explicitly handle the error case (.catch(e => window.__probe = 'ERR: ' + e)) so you can distinguish "in flight" from "errored".
For XHR-based interceptor captures, the same pattern applies: install the interceptor once, then issue the fetch without consuming the Promise in the eval return value. Read accumulated logs from window.__apilog on subsequent calls.
Step 2a.1: Build the user flow plan
From the primary browser-sniff goal (Step 0 in the SKILL.md), derive the interactive steps a real user would take to accomplish that goal. This is NOT a list of pages to load -- it is a sequence of actions.
Example for "Order a pizza for delivery" (Domino's):
- Click "Delivery" on homepage
- Enter a delivery address, click "Continue"
- Confirm a store from the results
- Browse the menu (click a category like "Specialty Pizzas")
- Add an item to cart (click "Add to Order")
- View cart (click cart icon)
- Proceed toward checkout (STOP before entering payment)
Example for "Create an issue" (Linear):
- Click "New Issue" from the sidebar
- Fill in title and description
- Assign to a team/project
- Set priority and labels
- Submit (or preview if dry-run)
Example for "Check today's scores" (ESPN):
- Load the homepage (scores are front-page content for read-heavy sites)
- Click a sport (NFL, NBA, etc.)
- Click a specific game for the box score / play-by-play
- Click standings
- Click a team for the team page
Each step triggers API calls that page loads alone would miss. After the primary flow, add 1-2 secondary flows from the research brief's other top workflows (e.g., "Check rewards," "Track an order").
Step 2a.1.5: Authenticated flow (when AUTH_SESSION_AVAILABLE=true)
When the user confirmed a logged-in session (AUTH_SESSION_AVAILABLE=true from Phase 1.6), add authenticated page visits AFTER the primary flow. The primary flow discovers the public API surface; the authenticated flow discovers what's behind the login wall.
Record the public endpoint set. Before visiting auth pages, note which endpoints have been discovered so far. These are the "public set" — reachable without session cookies.
Visit account/profile pages. Navigate to common authenticated URLs. Try these patterns in order, stopping at the first that loads a real page (not a redirect to login):
/account,/my-account,/profile,/settings/orders,/order-history,/my-orders/rewards,/loyalty,/my-deals-and-rewards/addresses,/saved-addresses,/payment-methods
Also derive page URLs from the research brief's top workflows. If the brief mentions "order history" or "rewards" or "saved addresses," visit the corresponding pages even if they don't match the common patterns above.
Interact with auth pages. Apply the SPA interaction rule below — click tabs, expand sections, trigger lazy loads. Auth pages often have sub-sections (e.g., "Recent Orders" tab, "Rewards History" tab) that fire separate API calls.
Classify endpoints. After visiting auth pages, compare the new endpoints against the public set:
- Endpoints that appear ONLY during auth page visits → classify as auth-required
- Endpoints that appear in both public and auth visits → classify as public
- Record the classification in the discovery report's Endpoints table (add an "Auth" column)
Discover the auth header pattern. Many SPAs don't send cookies directly — they read tokens from cookies/localStorage and construct an
Authorizationheader. Install an XHR header interceptor and trigger a client-side navigation (click a link — do NOT usebrowser-use open) to capture the actual request headers:# Install header interceptor browser-use eval "window.__authHeaders={}; const _s=XMLHttpRequest.prototype.setRequestHeader; XMLHttpRequest.prototype.setRequestHeader=function(k,v){ if(k.toLowerCase()==='authorization')window.__authHeaders[k]=v.substring(0,100); _s.apply(this,arguments)};'OK'" # Navigate via SPA click (preserves interceptor) browser-use eval "document.querySelector('a[href*=account],a[href*=orders],a[href*=rewards]').click()" sleep 3 # Collect captured auth headers browser-use eval "JSON.stringify(window.__authHeaders)"If an Authorization header is found:
- Record the scheme (e.g.,
Bearer,CustomerAuth,Token, custom) - Trace values back to cookies. Read
document.cookieand match literal values from the captured header against cookie values:
For each cookiebrowser-use eval "document.cookie"name=value, check ifvalueappears as a substring in the Authorization header. When a match is found, record the cookie name and which part of the header it corresponds to. - Construct the format string. Replace each literal cookie value in the header with
{cookieName}:- Example: header
CustomerAuth 2432962|FD44DA6A-..., cookiescustomerId=2432962; authToken=FD44DA6A-... - Format string:
CustomerAuth {customerId}|{authToken}
- Example: header
- Write composed auth into the spec. When building the spec YAML, include:
This tells the generator to emitauth: type: composed header: Authorization format: "<format string with {cookieName} placeholders>" cookie_domain: <site domain> cookies: - <cookie1Name> - <cookie2Name>auth login --chromethat reads those specific cookies and composes the header. The user never sees the format — the CLI handles it. - Also check localStorage for token sources:
browser-use eval "JSON.stringify(Object.keys(localStorage).filter(k=>k.match(/token|auth|session/i)))" - Use the composed header in Step 2d validation (not cookie replay — construct the actual header from extracted cookies)
- Record the auth scheme in the discovery report
If cookie matching fails (header values don't match any cookie values — possibly URL-encoded or hashed), fall back to recording the auth scheme without composed config. The printed CLI will use generic token auth. Report: "Auth header discovered but could not trace values to cookies."
If no Authorization header found but auth endpoints returned data (from step 4), the API likely uses cookie-based auth directly. Write
auth.type: cookieinto the spec and proceed to Step 2d with cookie replay.If the interceptor captured nothing (page didn't fire API calls), try clicking a different link or scrolling the page. If still nothing after 2 attempts, proceed to Step 2d with cookie replay as a fallback.
- Record the scheme (e.g.,
Trigger auth validation. If any auth-required endpoints were found, Step 2d (Cookie/token auth validation) MUST run. Use whichever auth method was discovered in step 5:
- If an Authorization header was found → replay with that header
- If no header found → try cookie replay
This is what propagates
Auth.Typeand auth config into the spec.
If auth pages redirect to login. The session may have expired between the time the user confirmed login and the browser-sniff reaches this step. Report: "Auth pages redirected to login — session may have expired. Auth-only endpoints not discovered." Do NOT fail the browser-sniff — the public endpoints are still valid. Proceed to Step 2a.2 with the public set only.
SPA interaction rule: On each page/state, take a snapshot first. Look for interactive elements (buttons, forms, dropdowns, tabs). Click through them. SPAs fire API calls on interaction, not on page load. If you load a page and see no XHR activity, that means you need to interact with the page, not that there is nothing to find.
SPA navigation rule: After installing fetch/XHR interceptors, do NOT use browser-use open to navigate between pages unless you immediately reinstall the interceptor before further interactions. A full page reload destroys page-scoped interceptors. Prefer navigating by clicking links after the interceptor is installed:
# Good: SPA navigation preserves interceptors
browser-use eval "document.querySelector('a[href*=\"/orders\"]').click()"
# or
browser-use click "Orders"
# Bad: full reload destroys interceptors
browser-use open "https://site.com/orders"Only use browser-use open for the initial page load (before interceptors exist) or when you intentionally want to re-install interceptors on a fresh page.
Primary request-body capture. After each browser-use open and before walking interactions on that page, install a Request-aware fetch body interceptor. The Performance API gives broad URL coverage, but it does not expose POST bodies. This interceptor preserves bodies for both fetch(new Request(...)) and legacy fetch(url, {body}) calls so the enriched capture can use the real request shape instead of inferring it from responses.
browser-use eval "window.__capture_bodies={};const _f=window.fetch;async function __ppReadFetchRequestBody(args){try{if(args[1]&&args[1].body)return typeof args[1].body==='string'?args[1].body:'[non-string]';if(args[0]&&typeof args[0]==='object'&&args[0].clone)return await args[0].clone().text()}catch(e){}return ''}window.fetch=async function(...args){const url=typeof args[0]==='string'?args[0]:(args[0]&&args[0].url)||'';const method=(args[1]&&args[1].method)||(args[0]&&args[0].method)||'GET';const requestBodyPromise=__ppReadFetchRequestBody(args);const r=await _f.apply(this,args);const c=r.clone();Promise.all([requestBodyPromise,c.text()]).then(([requestBody,responseBody])=>{window.__capture_bodies[method+' '+url]={request_body:requestBody,response_body:responseBody,response_status:r.status,response_content_type:r.headers.get('content-type')||''}}).catch(()=>{});return r}"After browsing, collect these bodies:
browser-use eval "JSON.stringify(window.__capture_bodies)"When building $DISCOVERY_DIR/browser-sniff-capture.json, merge the matching request_body, response_body, response_status, and response_content_type from window.__capture_bodies into each API entry. If a body is missing, fall back to the Step 2b enrichment loop or HAR metadata rather than guessing from the response shape.
Because browser-use installs this interceptor inside the current page, it cannot capture API calls that fired before installation during the initial page load. For page-load POST bodies, use an agent-browser/manual HAR path and prefer HAR request.postData.text; do not infer request shape from the response body.
Step 2a.2: Collect API URLs
Open a headless browser session, then visit each page and collect API URLs using the Performance API:
# Start collection
mkdir -p "$DISCOVERY_DIR"
SNIFF_URLS="$DISCOVERY_DIR/sniff-urls.txt"
> "$SNIFF_URLS"
# For EACH target page (run this loop in foreground — do NOT use run_in_background):
browser-use open "<target-page-url>"
sleep 4 # Wait for initial page load API calls to complete
# Install request/response body capture for interaction-triggered API calls:
# run the "Primary request-body capture" browser-use eval command above here.
# Early interactive-challenge check. If this finds Cloudflare/Vercel/WAF
# challenge assets or a challenge title/body, stop the capture attempt and
# route to the challenge-only recovery prompt below instead of burning the
# browser-sniff time budget waiting for Playwright to auto-solve it.
browser-use eval "var urls=performance.getEntriesByType('resource').map(e=>e.name).join('\n');var text=(document.title+' '+document.body.innerText+' '+document.documentElement.innerHTML).toLowerCase();JSON.stringify({challenge: urls.includes('/cdn-cgi/challenge-platform') || text.includes('just a moment') || text.includes('cf-turnstile') || text.includes('x-vercel-challenge') || text.includes('captcha'), title: document.title});"
# Apply browser-sniff pacing delay (starting at 1s, adapts per Browser-Sniff Pacing rules above)
browser-use scroll down # Trigger lazy-loaded content
sleep 1
# Apply browser-sniff pacing delay before next eval call
# Collect API URLs via Performance API (browser-native, no injection needed)
browser-use eval "var e=performance.getEntriesByType('resource');var u=[];for(var i=0;i<e.length;i++){var n=e[i].name;if(n.indexOf('<api-domain-1>')>-1||n.indexOf('<api-domain-2>')>-1)u.push(n);}u.join('|||');"
# Parse the result and append to collection file
# The eval output is "result: url1|||url2|||url3"
# Split on ||| and append each URL to the fileReplace <api-domain-1>, <api-domain-2> etc. with the API domains discovered in Phase 1 research (e.g., api.espn.com, sports.core.api, site.web.api).
Why Performance API: It is built into every browser, captures all resource loads (including those that fire before any JS interceptor could be injected), survives within a page lifecycle, and returns simple URL strings. Do NOT use fetch/XMLHttpRequest monkey-patching — it breaks on page navigation.
Step 2a.2.5: GraphQL BFF detection
After collecting URLs, check whether the site uses a GraphQL BFF pattern. This is common in modern SPAs (Domino's, Notion, Shopify storefronts) where all API traffic goes through a single /graphql or /api/graphql endpoint.
Detection signal: If >50% of captured XHR/fetch POST URLs resolve to the same path (e.g., /api/web-bff/graphql, /graphql, /api/graphql), classify as a GraphQL BFF.
If GraphQL BFF detected:
Extract operation names from POST bodies. The URL alone tells you nothing — all calls go to the same endpoint. The value is in the request bodies.
For agent-browser:
# List all XHR requests agent-browser network requests --type xhr --json # For each POST to the GraphQL endpoint, get the full request including body: agent-browser network request <request-id> --json # Parse: look for operationName and query fields in the request bodyFor browser-use: inject a fetch interceptor BEFORE browsing auth/interaction pages. This captures POST bodies that the Performance API misses:
browser-use eval "window.__gqlOps=[];const _f=window.fetch;async function __ppReadFetchRequestBody(args){try{if(args[1]&&args[1].body)return typeof args[1].body==='string'?args[1].body:'[non-string]';if(args[0]&&typeof args[0]==='object'&&args[0].clone)return await args[0].clone().text()}catch(e){}return ''}window.fetch=async function(...args){const bodyPromise=__ppReadFetchRequestBody(args);const url=typeof args[0]==='string'?args[0]:(args[0]&&args[0].url)||'';const r=await _f.apply(this,args);bodyPromise.then(body=>{try{if(url.includes('graphql')&&body&&body!=='[non-string]'){const b=JSON.parse(body);if(b.operationName)window.__gqlOps.push({op:b.operationName,vars:Object.keys(b.variables||{})})}}catch(e){}});return r}"After browsing, collect:
browser-use eval "JSON.stringify(window.__gqlOps)"Record operations. For each unique
operationName, record:- Operation name (e.g.,
GetStoreMenu,AddToCart,GetOrderHistory) - Type: query (read) or mutation (write) — infer from the
queryfield prefix or from naming convention (Get*= query,Add*/Create*/Update*/Delete*= mutation) - Variable keys (e.g.,
storeId,productCode) — these become CLI flags - Domain group — group by prefix (e.g.,
Store*,Menu*,Order*,Account*)
- Operation name (e.g.,
Write to discovery report. Replace (or supplement) the "Endpoints Discovered" table with a "GraphQL Operations" table:
| Operation | Type | Variables | Domain | |-----------|------|-----------|--------| | GetStoreMenu | query | storeId, lang | Store | | AddToCart | mutation | productCode, qty | Order |Feed into spec building. When building the OpenAPI spec from discovered operations, each GraphQL operation becomes a spec path:
POST /graphql#OperationName. The operation name goes inoperationId. Variables become request body properties. This is compatible with the existing generator — it sees each operation as a distinct POST endpoint.
If NOT a GraphQL BFF: Skip this step. The existing URL-based discovery flow handles REST APIs.
Step 2a.2.7: JS bundle endpoint extraction (supplementary)
SPA frameworks (Angular, React, Vue, Next.js) compile all API endpoint paths into their main JS bundle. Extracting these paths supplements the browser-sniff with endpoints that no user flow visits (admin features, migration tools, rarely-used settings).
When to run: After completing the interactive browser-sniff (Steps 2a.1–2a.2.5). This is supplementary — the browser-sniff is primary because it provides response shapes, auth patterns, and parameter types. Bundle extraction only gives endpoint paths.
Skip when: The site is server-rendered HTML without JS bundles, or the browser-sniff already discovered 20+ endpoints and the API surface appears complete.
Find the main bundle:
browser-use eval "Array.from(document.querySelectorAll('script[src]')).map(s=>s.src).filter(s=>s.includes(location.hostname)&&(s.includes('main')||s.includes('app'))).join('\\n')"Download and extract API paths:
curl -s "<bundle-url>" | python3 -c " import sys, re content = sys.stdin.read() # Find the API base URL config (common patterns) base_match = re.search(r'(apiUrl|baseUrl|API_URL)[^\"]*\"(https?://[^\"]+)\"', content) if base_match: print(f'API base: {base_match.group(2)}') # Extract capitalized path segments (API routes) paths = re.findall(r'\"(/[A-Z][a-zA-Z]+(?:/[A-Z]?[a-zA-Z]*)*)\"|\"(/[a-z]+/[a-zA-Z]+)\"', content) unique = sorted(set(p[0] or p[1] for p in paths if p[0] or p[1])) for p in unique: print(f' {p}') # Extract HTTP method calls calls = re.findall(r'\.(get|post|put|delete|patch)\([^)]*\"(/[A-Za-z][A-Za-z0-9/\${}]+)', content) for method, path in sorted(set(calls)): print(f' {method.upper()} {path}') "Merge with browser-sniff results. Append bundle-discovered endpoints to
$SNIFF_URLS. Mark their provenance:# Append bundle-only endpoints (not already in sniff-urls.txt) # In the discovery report, mark these as "discovered: bundle"Record API config. If the bundle reveals useful config (API version headers, auth token construction, rate limit hints), note them in the discovery report's Browser-Sniff Configuration section.
Step 2a.3: Deduplicate and normalize
After collecting from all pages:
# Strip query parameters and deduplicate to find unique API path patterns
cat "$SNIFF_URLS" | sed 's/\?.*//' | sort -u > "$DISCOVERY_DIR/browser-sniff-unique-paths.txt"Step 2a.4: Generate enriched capture
The Performance API gives us URLs but not response bodies. To feed cli-printing-press browser-sniff, we need to call each unique API endpoint and capture the response:
# For each unique API URL, fetch it and build a simple capture file
# cli-printing-press browser-sniff accepts HAR or enriched capture JSON
# When fetching each unique API URL to build enriched capture:
# Apply browser-sniff pacing between requests (1s initial, adaptive per Browser-Sniff Pacing rules)
# On 429: double delay, log, continue with remaining URLsAlternatively, if the URL count is small enough, the unique path patterns alone are sufficient to identify what the existing spec is missing — compare against the spec and report the gap without needing full HAR capture.
Step 2a.5: Close browser
browser-use closeStep 2b: agent-browser capture (fallback)
If browser-use is not available, use agent-browser with Claude driving the exploration. Note: agent-browser HAR does not include response bodies. Use the enriched capture workflow to get them.
Browse and capture:
# agent-browser is headless by default; use --headed to show the browser window agent-browser open <target-url> agent-browser network har startWalk the user flow using the snapshot-reason-act loop:
- Use the user flow plan from Step 2a.1 (same flow applies to both backends)
- For each step in the flow:
agent-browser snapshot -ito see the current page state- Find the interactive element for this step (button, form, link, dropdown)
- Click/fill/submit it
agent-browser wait --network-idleafter each interaction- Apply browser-sniff pacing between interactions (1s initial, adaptive per Browser-Sniff Pacing rules)
- After completing the primary flow, run 1-2 secondary flows
- Skip: navigation links, footer links, social media buttons, cookie/consent banners
- Fill forms with realistic sample data based on the domain (real-looking addresses, names, etc.)
Capture response bodies (agent-browser HAR omits them):
agent-browser network requests --type xhr,fetch --jsonFor each API request (filter by JSON content-type, skip analytics domains):
agent-browser network request <request-id> --json # Apply browser-sniff pacing between response body fetches # These are direct API calls and most likely to trigger rate limitsCombine HAR metadata + response bodies into an enriched capture JSON at
$DISCOVERY_DIR/browser-sniff-capture.json.Stop HAR recording:
agent-browser network har stop "$DISCOVERY_DIR/browser-sniff-capture.har"
Step 2e: Claude chrome-MCP capture (failure-recovery fallback)
Use this backend when (a) the user picks chrome-MCP from the Step 2c.5 recovery menu after browser-use or agent-browser was hard-blocked by an anti-bot gate, or (b) the user opted into chrome-MCP up-front from the Step 1b install picker. Detection is set in Step 1 (CHROME_MCP_AVAILABLE). Skip this section entirely if CHROME_MCP_AVAILABLE=false.
Why chrome-MCP exists as a fallback. chrome-MCP drives the user's already-running Chrome via a browser extension, so capture happens inside the user's logged-in session — no cookie transfer step (unlike Step 1d), and the WAF sees a real Chrome instance with the user's normal fingerprint. This is exactly what unblocks targets that hard-blocked headless Chromium-based capture (browser-use, agent-browser) under Akamai / DataDome / sophisticated bot detection.
Auth surface caveat. chrome-MCP captures every request the user's tab fires, including ones carrying Authorization headers, session cookies, OAuth tokens, and JWTs from the live session. The credential strip rules below are mandatory, not advisory.
Prerequisites.
- The Chrome browser extension for chrome-MCP must be installed and connected. If
tabs_context_mcpreturns an error indicating the extension isn't connected, instruct the user viaAskUserQuestionto connect the extension and retry. If they decline, fall back to the Step 2c.5 recovery menu with chrome-MCP removed (the menu re-fires per the menu re-fire rule in Step 2c.5). - Chrome must be visible and unminimized (chrome-MCP cannot drive a hidden window).
- For authenticated discovery: the user should already be logged in to
<site>in their Chrome session. No separate cookie transfer step.
Tab scope rule (mandatory — not optional).
Always create a fresh capture tab via mcp__claude-in-chrome__tabs_create_mcp, even if tabs_context_mcp shows a tab already open at <site>. Reasons:
- The user's existing tab may have unsaved work (a draft message, a partially-filled form, a search the user is mid-way through). Navigating it away is data loss.
- Other open tabs may be sensitive (banking, internal tools, personal email). The agent has no way to classify them; the safe default is to never touch them.
- A fresh tab gives the agent a clean DOM and clean network log scoped to the discovery target.
tabs_context_mcp is called for awareness only — to confirm the extension is connected and to log how many tabs the user has open. Never navigate an existing tab. Always tabs_create_mcp → navigate in the new tab. After capture, close the capture tab via mcp__claude-in-chrome__tabs_close_mcp so the user's tab strip returns to its original state.
Body-capture interceptor (mandatory — install BEFORE navigation).
mcp__claude-in-chrome__read_network_requests returns request metadata (URL, method, request headers, response status) but does not include response bodies. The downstream internal/browsersniff/parser.go consumer requires response_body populated per entry, so an extra step is mandatory.
Two viable approaches:
Recommended: in-page interceptor installed before interactions. Use
mcp__claude-in-chrome__javascript_toolto install afetchandXMLHttpRequestinterceptor in the fresh capture tab after navigating to<site>and before performing the user-flow interactions. The interceptor pushes each completed call's request and response bodies intowindow.__capture_bodieskeyed by URL+method. After capture,javascript_toolreadswindow.__capture_bodiesback. This avoids re-firing requests against a wary WAF.Fallback:
javascript_toolre-fetches each captured URL after the fact. Mirrors theagent-browser network request <id> --jsonenrichment loop in Step 2b. Use this only when option 1 fails (e.g., the page wrapsfetchitself and the interceptor can't shadow it). Re-fetching may trip rate limits — apply the pacing rules below.
The interceptor sketch (illustrative, adapt to the page's actual fetch shape):
// Run via mcp__claude-in-chrome__javascript_tool after navigating to <site>
// and before performing interaction-triggered capture steps.
window.__capture_bodies = {};
const _origFetch = window.fetch;
async function __ppReadFetchRequestBody(args) {
try {
if (args[1] && args[1].body) {
return typeof args[1].body === 'string' ? args[1].body : '[non-string]';
}
if (args[0] && typeof args[0] === 'object' && args[0].clone) {
return await args[0].clone().text();
}
} catch (e) {}
return '';
}
window.fetch = async function(...args) {
const url = typeof args[0] === 'string' ? args[0] : (args[0] && args[0].url) || '';
const method = (args[1] && args[1].method) || (args[0] && args[0].method) || 'GET';
const requestBodyPromise = __ppReadFetchRequestBody(args);
const resp = await _origFetch.apply(this, args);
const cloned = resp.clone();
Promise.all([requestBodyPromise, cloned.text()]).then(([requestBody, body]) => {
window.__capture_bodies[`${method} ${url}`] = {request_body: requestBody, response_body: body};
}).catch(() => {});
return resp;
};
// XHR interceptor analogous; install bothWhen merging window.__capture_bodies with network metadata, copy the stored request_body and response_body fields separately into the enriched capture entry.
This page-scoped interceptor cannot capture API calls that fired before installation during the initial navigation. If a page-load POST body matters, use manual DevTools HAR export or another HAR-producing path and prefer HAR request.postData.text.
Capture flow (full sequence).
mcp__claude-in-chrome__tabs_context_mcp— awareness only; confirm extension is connectedmcp__claude-in-chrome__tabs_create_mcp— fresh capture tabmcp__claude-in-chrome__navigate— open the discovery target URLmcp__claude-in-chrome__javascript_tool— install fetch + XHR body interceptor in the current page- Interaction loop (per the same "click + scroll + interact" guidance the browser-use flow uses):
mcp__claude-in-chrome__read_pageto find interactive elementsmcp__claude-in-chrome__find+mcp__claude-in-chrome__left_click(orform_inputfor forms) to interact- For SPAs that hydrate via JS without a fresh navigation, do at least 3 meaningful interactions before reading the network log
mcp__claude-in-chrome__read_network_requests— pull request metadatamcp__claude-in-chrome__javascript_tool— read backwindow.__capture_bodies- Optional:
mcp__claude-in-chrome__get_page_textfor SSR-rendered content - Merge metadata + bodies into the enriched-capture JSON shape (next subsection)
mcp__claude-in-chrome__tabs_close_mcp— clean up the capture tab
Output format.
Write to $DISCOVERY_DIR/browser-sniff-capture.json using the EnrichedCapture shape from internal/browsersniff/types.go:
- Top-level:
target_url,captured_at(RFC3339),interaction_rounds(count of meaningful interactions),auth(object describing how auth flows for this capture;{"type": "chrome_mcp_session", "site": "<site>"}is fine),entries[] - Per-entry:
method,url,request_headers(object),response_headers(object),request_body(string, may be empty),response_body(string, populated from interceptor),response_status(int),response_content_type(string),classification(leave empty; analyzer fills),is_noise(leave false; analyzer fills)
This is the same shape agent-browser's enriched-capture JSON uses (Step 2b lines 685-695), so downstream parsing is unchanged.
Write-time credential strip (mandatory — not optional).
Before writing the entry to $DISCOVERY_DIR/browser-sniff-capture.json, scrub credentials from request_headers, response_headers, and request/response bodies. Matching on JSON field names such as apiKey is not enough: products that let users store outbound HTTP credentials embed those values under nested header maps (formulaMap.Authorization, headers.Authorization, headers["X-API-Key"]).
- Remove headers with names matching (case-insensitive):
Authorization,Cookie,Set-Cookie,Proxy-Authorization,X-Api-Key,X-Auth-Token,X-Session-Id, and any header matching the regex/^x-.*-(token|key|auth|session|secret)$/i. - In request/response bodies, redact the same header names when they appear as nested map keys, and redact values that are
Basic/Bearer/Tokencredential blobs regardless of the surrounding key name. The durable machine scrub isbrowsersniff.RedactJSONBody(used for<spec-stem>-samples/); apply the same rules when writing the capture yourself. - For URLs containing query parameters that look like tokens (
access_token=,api_key=,token=,key=,signature=,auth=,password=), redact the value toREDACTEDin the stored URL. - Scope guard: scrub header maps and request/response bodies only. Do not run a blind base64 sweep over
url,host, orpathfields — long base64-shaped path segments are endpoint evidence, not credentials.
Cross-reference secret-protection.md for the canonical scrub list — when the canonical list updates, this section must update too. The strip happens at write time so the artifact never sits on disk with live credentials, even briefly. Phase 5.5 archive-time strip is a defense-in-depth backstop, not the primary control for chrome-MCP captures.
Pacing scope.
The adaptive-pacing rules at the top of "If user approves browser-sniff" (start at 1s, ramp down on success, double on 429, hard-stop after 3 consecutive 429s) apply to operations the agent initiates: navigate between pages, javascript_tool-fired fetch() calls, and any re-fetch loop in the fallback body-capture approach. They do not apply to read_network_requests itself — that's observational and reads what the page already fired. Apply pacing where the agent is the request source, not the observer.
Failure detection (multi-trigger).
A chrome-MCP capture has not succeeded just because read_network_requests returned 142 entries — the entries may all be challenge-page noise. Before reporting success, check all three triggers; any one means "WAF is also blocking chrome-MCP, treat as capture failure":
- HTTP status pattern. Count entries by status. If fewer than 30% returned 2xx, OR more than 30% returned 403/429, the capture is failing.
- Body markers. Scan response bodies for known challenge sentinels:
Just a moment(Cloudflare),cf_chl_opt(Cloudflare),_abckreferences in HTML (Akamai),<title>Access Denied</title>(Akamai/F5),Pardon Our Interruption(PerimeterX),<title>Bot or Not?</title>(Akamai). Any sentinel in any body triggers detection. - Page sentinel.
mcp__claude-in-chrome__get_page_textafter the interaction loop — if the visible page contains "verifying you are human", "checking your browser", "captcha" (case-insensitive), or "access denied", treat as challenge.
When any trigger fires, do NOT write the artifact. Re-fire the Step 2c.5 recovery menu with chrome-MCP removed from the option list (per the menu re-fire rule in Step 2c.5). Record the failure in the marker file's reason field: chrome-mcp captured N requests but failure detection triggered: <which trigger fired>.
Replayability check.
Even when failure detection passes, the same replayability constraint from cardinal rule 5 applies: the captured surface must round-trip through the Chrome-compatible transport with the same TLS fingerprint the printed CLI will ship, or the captured URLs are unusable in production. Do not skip this check just because the user's authenticated browser session worked — the printed CLI does not have the user's authenticated browser session.
Step 2c: Thin-results safety check
After completing the primary user flow capture (browser-use or agent-browser), count unique API endpoints discovered. If fewer than 5 unique endpoints:
Diagnose, don't accept. Thin results from an SPA almost always mean the agent loaded pages without interacting. Ask yourself: did I click buttons? Did I fill forms? Did I submit anything? Did I scroll to trigger lazy loads? If the answer is "I mostly just navigated to URLs," that is the problem.
Re-sniff with interaction. Go back to the page where results were thinnest. Take a snapshot. Find interactive elements. Click the most prominent one. Wait for network activity. Repeat for at least 3 interactions before accepting thin results.
Compare against known endpoints. If Phase 1 research found community wrappers documenting N endpoints but the browser-sniff found fewer than N/2, the browser-sniff missed something. Community wrappers are a floor, not a ceiling -- they represent what someone else already reverse-engineered, often years ago. The real API surface is almost certainly larger.
Report the gap honestly. If re-sniffing with interaction still produces thin results, report: "Browser-Sniff captured X endpoints but community wrappers document Y. The site may use WebSocket, protobuf, server-side rendering, or other techniques that resist browser capture." Do NOT conclude "the API has few endpoints" when the real answer may be "I didn't interact enough to trigger them."
If the thin-results check triggers a re-sniff that discovers additional endpoints, merge the new captures with the originals before proceeding to Step 3.
Step 2c.5: Challenge-only capture safety check
After capture, inspect the collected responses before generating a spec. A browser-sniff is not successful if it only captured challenge, login, or access-denied pages.
Treat the capture as failed when all or nearly all captured target-site responses match one of these:
- HTTP
200responses that only contain a content-less shell, interstitial, deterministic-size truncation, or other placeholder body with no real site data - HTTP
403or429HTML with Cloudflare/Vercel/WAF/DataDome/PerimeterX/CAPTCHA markers - titles or body text such as "Just a moment", "Access denied", "Please enable JavaScript", "captcha", "challenge"
- only login redirects/pages when the user expected an authenticated capture
- no API-looking requests, no SSR embedded data, no structured HTML/feed data, and no page-context fetch evidence
Do not treat a 200-served shell as evidence for IP-blocked, rate-limited, or wait it out. It is a clearance challenge until proven otherwise. Escalate through the same recovery menu as explicit 403/429 challenge pages, and use chrome-MCP to inspect the live browser wall when that backend is available, even if the eventual cookie-warm path still needs user help.
When this happens, do not continue to Phase 2 with a challenge-page spec. Compose the recovery menu per the availability of the MCP fallback flags set in Step 1 — the menu shape changes based on what's reachable in this runtime. Use the table below to pick the option set, then present via AskUserQuestion.
Menu composition table. Always read CHROME_MCP_AVAILABLE and COMPUTER_USE_AVAILABLE from your reasoning (set in Step 1). Pick the row matching the flag combination:
CHROME_MCP_AVAILABLE |
COMPUTER_USE_AVAILABLE |
Menu options (in order) |
|---|---|---|
| false | false | (1) Try cleared-browser capture again, (2) I'll provide a HAR from DevTools, (3) Discuss alternate CLI scope — current behavior, no change |
| false | true | (1) Try cleared-browser capture again, (2) I'll provide a HAR from DevTools — I'll guide you with screenshots of your DevTools window, (3) Discuss alternate CLI scope |
| true | false | (1) Try cleared-browser capture again, (2) Try chrome-MCP — recommended on anti-bot trigger, (3) I'll provide a HAR from DevTools, (4) Discuss alternate CLI scope |
| true | true | (1) Try cleared-browser capture again, (2) Try chrome-MCP — recommended on anti-bot trigger, (3) I'll provide a HAR from DevTools — I'll guide you with screenshots of your DevTools window, (4) Discuss alternate CLI scope |
Recommended-badge rule. When the menu fires because of an anti-bot block (the trigger criteria above: HTTP 200 challenge shells or truncations, 403/429 with WAF markers, "Just a moment", challenge titles, login-redirect-only when authenticated capture was expected), chrome-MCP carries the (Recommended) badge whenever it is present in the menu, regardless of whether computer-use is also detected — chrome-MCP is the highest-leverage path against an anti-bot block because it uses the user's real Chrome session. In other failure modes (thin results from the Step 2c check, time-budget bailout), no option carries the Recommended badge — let the user pick based on context.
Question stem (teach the chrome-MCP mechanic when present). When chrome-MCP is in the menu and the user has not seen this menu before in the current session, the question stem must teach the mechanic in one line. The user is mid-failure and may have never encountered chrome-MCP-as-fallback.
When chrome-MCP IS in the menu:
"The browser capture only saw challenge/login pages or a 200-served shell, so it did not discover the real website data/API surface. The Chrome-extension MCP can capture from your existing Chrome window — pick this if your Chrome is open and you're already logged in to the target site. What should we do next?"
When chrome-MCP is NOT in the menu (current 3-option case, unchanged):
"The browser capture only saw challenge/login pages or a 200-served shell, so it did not discover the real website data/API surface. What should we do next?"
Fixed option labels and bodies. Use these exact strings as the AskUserQuestion option labels and descriptions — the implementer should not paraphrase. Labels are short (4-7 words), self-contained (some harnesses render labels without descriptions), and front-load the differentiator. Composition is per the table above; pick the option set for the flag combination, then mark Recommended where the rule above says.
- "Try cleared-browser capture again" — "Open/attach Chrome, solve the challenge, then repeat the browser-sniff with the previous backend."
- "Try chrome-MCP" — "Capture from your already-running Chrome window via the Chrome extension MCP. Uses your logged-in session, no cookie transfer. The agent will create a fresh tab and close it when done."
- "I'll provide a HAR from DevTools" — "You browse the target site in your Chrome and export a HAR. The agent analyzes it for discovery, then keeps only routes that pass replayability checks."
- "I'll provide a HAR from DevTools — I'll guide you with screenshots of your DevTools window" — "Same as 'I'll provide a HAR from DevTools', plus the agent uses computer-use to take screenshots of your Chrome window at each step and tells you what it sees so you don't get stuck on the export flow."
- "Discuss alternate CLI scope" — "Step away from this discovery target and consider RSS/docs/official API/narrower-scope proposals. Only pick this when the other options have been tried."
Routing on selection.
- "Try cleared-browser capture again" → re-run the previously-attempted backend (browser-use or agent-browser) per the existing capture flow. If it fails again with the same trigger, re-fire this menu with this option's body amended to "(this option already failed once; consider another path)" — the option stays in the menu but loses any badge.
- "Try chrome-MCP" → load the chrome-MCP MCP tools via
ToolSearch(select:mcp__claude-in-chrome__*), then proceed to Step 2e for the capture playbook. - "I'll provide a HAR from DevTools" (either variant) → proceed to the manual-HAR flow at Step 1d (the existing branch). When the user picks the augmented variant, the manual-HAR body augmentation in Step 1d kicks in — see Step 1d for the computer-use visual-feedback-loop.
- "Discuss alternate CLI scope" → leave this flow and return to upstream scope discussion. Record
decision: alternate_scopein the marker file.
Menu re-fire on backend failure (session-scoped state). When the user picks a backend from this menu and that backend ALSO fails (per the Step 2c.5 trigger criteria for browser-use/agent-browser/manual-HAR, or per the Step 2e multi-trigger failure detection for chrome-MCP), re-fire this exact menu with the failed option removed. Track which backends have been tried in your reasoning across this same recovery session — this is a session-scoped tracking concern, not a marker-file field. Stop re-firing when only "Discuss alternate CLI scope" remains in the menu, or when the user picks "Discuss alternate CLI scope" themselves.
Marker file reason field convention. When recording the chosen backend in the browser-browser-sniff-gate.json marker, name it explicitly so future audit can identify which backend produced the capture. Examples: reason: "chrome-mcp captured 142 requests; SSR enriched after Akamai block on agent-browser", reason: "manual-HAR with computer-use guidance after browser-use blocked". The reason field is free-form per the existing schema; no migration needed.
Only the "Discuss alternate CLI scope" option may lead to an RSS-first, official API, docs-only, or narrower-scope proposal. Record the failed capture in $DISCOVERY_DIR/browser-sniff-report.md if a report is written.
If direct HTTP is blocked but the page does not require live page-context execution, try the lightweight replay path before proposing a resident browser runtime: Surf/Chrome-compatible HTTP with modern TLS/UA headers, uTLS-style Chrome ClientHello where available, browser-clearance cookie import plus replay, or structured HTML/SSR/RSS extraction. These are discovery and replayability aids, not permission to ship a browser sidecar transport.
Step 2d: Cookie auth validation (authenticated browser-sniff only)
Skip this step if: The browser-sniff was anonymous (no session transfer in Step 1d), or the API uses API key / Bearer token auth rather than cookie-based session auth.
Purpose: Before promising auth login --chrome in the generated CLI, validate that browser cookies actually produce authenticated responses when replayed outside the browser context. Some APIs use CSRF tokens, SameSite cookie policies, or other mechanisms that prevent cookie-only replay.
Validation procedure:
Select a test endpoint. Pick one endpoint from the capture that returned HTTP 200 and appears to require authentication (e.g., a user-specific resource like
/api/me,/account, or/orders).Replay with cookies. Using
curlor the capture tool, replay the request with the captured cookies attached:curl -s -o /dev/null -w "%{http_code}" \ -H "Cookie: <captured-cookie-string>" \ "https://<api-domain>/<test-endpoint>"Expected: HTTP 200 (or the same status as during capture).
Replay without cookies. Replay the same request with no cookies:
curl -s -o /dev/null -w "%{http_code}" \ "https://<api-domain>/<test-endpoint>"Expected: HTTP 401, 403, or a redirect to a login page.
Evaluate results:
With cookies Without cookies Verdict 200 401/403/302 Pass — cookie auth works. Set Auth.Type = "cookie"andCookieDomainin the spec. The generated CLI will includeauth login --chrome.200 200 (same content) Not required — cookies aren't needed for this endpoint. Check other endpoints; if none require auth, set Auth.Type = "none".401/403 401/403 Fail — cookies don't replay (likely CSRF, SameSite, or IP binding). Warn the user and do not offer browser auth. Other Any Inconclusive — try a different test endpoint. If all attempts are inconclusive after 3 endpoints, treat as Fail. On Pass: Proceed to Step 3. The browser-sniff report (Step 5) should note: "Cookie auth validated — the generated CLI will support
auth login --chrome."On Fail: Inform the user via the conversation:
"Authenticated endpoints were discovered, but cookie replay failed (likely CSRF tokens or strict cookie policies). The generated CLI will include these endpoints but won't offer
auth login --chrome. You'll need to manually provide auth tokens via environment variables."Set
Auth.Type = "none"in the capture's auth section. Include the authenticated endpoints in the spec (they're still valid endpoints), but the CLI won't have a browser auth path. Note the failure reason in the browser-sniff report.
Step 3: Analyze capture
Run browser-sniff on the captured traffic. Always write the structured traffic analysis to the discovery directory so it is archived with the manuscript:
cli-printing-press browser-sniff --har "$DISCOVERY_DIR/browser-sniff-capture.har" --name <api> --output "$RESEARCH_DIR/<api>-browser-sniff-spec.yaml" --analysis-output "$DISCOVERY_DIR/traffic-analysis.json"If using agent-browser's enriched capture format instead:
cli-printing-press browser-sniff --har "$DISCOVERY_DIR/browser-sniff-capture.json" --name <api> --output "$RESEARCH_DIR/<api>-browser-sniff-spec.yaml" --analysis-output "$DISCOVERY_DIR/traffic-analysis.json"If $API_RUN_DIR/source-priority.json exists with two or more sources, add --preserve-hosts to the browser-sniff command so combo-CLI captures retain peer API hosts with per-endpoint base_url overrides instead of selecting only the dominant host.
Immediately inspect $DISCOVERY_DIR/traffic-analysis.json for response-body quality before trusting the sniffed spec:
- If
warnings[].typecontainsempty_response_shapes, or everyendpoint_clusters[]entry hassize_class: "empty"andresponse_shape: {}, the HAR/enriched capture did not preserve usable response bodies. Do not proceed to generation with the skeletal sniffed spec. - For each discovered endpoint cluster, use curl or another direct HTTP path with the captured method, URL, safe headers, and body shape to retrieve at least one representative response. Store those response samples in
$DISCOVERY_DIR/direct-response-*.jsonor the browser-sniff report, then repair the sniffed spec's response types from the direct responses. - If direct HTTP is blocked by the same protection that required browser capture, report HOLD or return to the capture recovery menu. Do not invent type definitions from endpoint names alone.
If hand-writing or repairing $DISCOVERY_DIR/traffic-analysis.json, inspect the canonical schema first:
cli-printing-press schema traffic-analysis > "$DISCOVERY_DIR/traffic-analysis.schema.json"Two fields trip up hand-edits often enough to call out:
versionis the literal string"1"— not semver, not"1.0.0". The downstream parser rejects any other value withunsupported traffic analysis version.- Confidence fields are numbers from
0to1, not strings such as"high".
Before hand-writing or repairing the sniffed YAML spec, check
spec-format.md; two common traps are types.X.fields list shape (- name:
items, not a map) and the response_format enum (json, csv, html, or binary;
use html only for GET/HEAD HTML and embedded-JSON surfaces, with
html_extract when the built-in page, links, or embedded-json modes fit).
Step 4: Report and update spec source
Report: "Browser-Sniff discovered N endpoints across M resources. [X new endpoints not in the original spec.]"
Read $DISCOVERY_DIR/traffic-analysis.json before reporting. If it includes:
"warnings": [{"type": "empty_response_shapes", ...}]— report: "Browser-Sniff captured endpoints but no response bodies. I need direct curl/HTTP samples before generation can have useful response types." Then run the response-body quality recovery from Step 3 before updating the spec source."reachability": {"mode": "browser_clearance_http", ...}— report: "Direct HTTP is blocked; generation will use browser-compatible HTTP plusauth login --chromecookie import. After generation, test whether Surf + imported cookies can replay the captured requests without a resident browser."- Useful same-site HTML document captures — report: "Browser-Sniff found replayable HTML pages; generation can emit
response_format: htmlcommands that extract metadata and filtered links without a resident browser." "reachability": {"mode": "browser_required", ...}— report: "The captured surface appears to require live page-context execution. This is not a shippable runtime shape for ordinary printed CLI commands. Return to discovery for a Surf/direct/browser-clearance replayable surface such as HTML, SSR data, RSS, JSON-LD, or a lighter internal endpoint, or HOLD the run."
Also report the runtime shape the printed CLI will use: standard HTTP, Chrome-compatible HTTP, browser-clearance cookie import plus replay, structured HTML/SSR/RSS extraction, or HOLD because no replayable surface was found.
Update the spec source for Phase 2:
- Enrichment mode: Phase 2 will use
--spec <original> --spec <sniff-spec> --name <api>to merge both - Primary mode: Phase 2 will use
--spec <sniff-spec>directly
Step 5: Write browser-sniff discovery report
Write a structured browser-sniff provenance report to $DISCOVERY_DIR/browser-sniff-report.md. This report preserves the discovery evidence so a future maintainer can reproduce or extend the browser-sniff.
The report must contain these sections:
User Goal Flow — The primary browser-sniff goal and each step attempted.
- Goal: [e.g., "Order a pizza for delivery"]
- Steps completed: [numbered list of steps taken, with which API operations each step triggered]
- Steps skipped: [any steps that couldn't be completed, with reason]
- Secondary flows attempted: [any additional workflows beyond the primary goal]
- Coverage: [X of Y planned steps completed]
Pages & Interactions — List every URL browsed and interaction performed during the browser-sniff, in order. Include the page purpose and what was clicked/filled/submitted (e.g., "Homepage -- clicked 'Delivery' button", "Address modal -- entered '350 5th Ave', clicked 'Continue'").
Browser-Sniff Configuration — Backend used (browser-use, agent-browser, or manual HAR), pacing settings (initial delay, final effective rate), and proxy pattern detection result (proxy-envelope detected / not detected, with the proxy URL if applicable).
Endpoints Discovered — A markdown table with columns: Method, Path, Status Code, Content-Type, Auth. One row per unique endpoint observed. The Auth column is "public" or "auth-required" (based on Step 2a.1.5 classification). If no authenticated flow was run, omit the Auth column.
Traffic Analysis — Summarize
$DISCOVERY_DIR/traffic-analysis.json:- Protocols observed (labels + confidence, e.g.,
rest_json,graphql,google_batchexecute,ssr_embedded_data) - Auth signals (candidate types, header/query/cookie names only -- never values)
- Parameter-name evidence from forms, input labels, placeholder text, SDK/source names, and request context. Preserve enough detail to justify any later
flag_nameenrichment. - Protection signals (Cloudflare/CAPTCHA/login redirects/protected-web hints)
- Generation hints (e.g.,
requires_browser_auth,requires_js_rendering,requires_protected_client,has_rpc_envelope). Treatauth_supports_captcha_preflightas informational auth context, not as proof that the runtime needs browser page context. - Candidate commands worth considering
- Warnings such as raw protocol envelopes, GraphQL error-only responses, HTML challenge pages, empty payloads, or weak schema evidence Treat warnings as discovery evidence, not publish blockers.
- Protocols observed (labels + confidence, e.g.,
Coverage Analysis — What resource types were exercised (e.g., "collections, workspaces, teams, categories") and what was likely missed. Compare against the Phase 1 research brief to identify gaps (e.g., "Brief mentions 'flows' but no flow endpoints were discovered during browser-sniff").
Response Samples — For each unique response shape (keyed by status code + content-type category), include a truncated sample:
- JSON/text responses: first 2KB or 100 lines, whichever is smaller
- Binary responses (images, protobuf, etc.): skip content, include a metadata note:
Binary response: <content-type>, <size> bytes - Aim for one sample per unique shape, not one per endpoint
Rate Limiting Events — Any 429 responses encountered, delays applied, and effective browser-sniff rate achieved (e.g., "Sniffed 7 endpoints at ~1.5 req/s effective rate, one 429 at request #4").
Authentication Context — Whether the browser-sniff used an authenticated session. If yes: transfer method used (auto-connect / profile / headed login / HAR), which endpoints were only reachable with auth (e.g., "order history, saved addresses, rewards required login"), the auth header scheme discovered (e.g., "Authorization: CustomerAuth {customerId}|{authToken}", "Bearer token from localStorage"), and confirmation that session state was excluded from manuscript archiving. If no: "No authenticated session used."
Bundle Extraction — If JS bundle extraction ran (Step 2a.2.7), list: the bundle URL analyzed, the API base URL discovered, endpoints found only in the bundle (not during interactive browser-sniff), and any API config extracted (version headers, auth construction patterns). If bundle extraction did not run, omit this section.