Run and Diagnose E2E Tests
Step 0: Determine run notes
Before running the tests, decide on a short note describing why this run is happening. This gets saved alongside artifacts so runs are easy to understand at a glance later.
Determine the note automatically from context:
- Post-merge verification: "Verifying main after merging branch
<branch-name>" (if you just merged or the user asked to run after a merge). - Main health check: "Routine check that main is green" (if already on main with no special context).
- Specific reason from the user or agent: Use whatever reason was stated, e.g. "Checking whether the callout refactor broke publish flow".
- Worktree runs: The worktree name is already captured in the run ID, so you don't need to repeat it. But if there's a specific reason for the run within that worktree, include it.
Keep notes to 1-2 sentences.
Step 1: Run the tests
Pass the note via the --run-notes flag:
./app/acceptance/e2e/_module/scripts/slowcheck --run-notes "<your note here>"This runs the full Playwright test suite (all specs in app/acceptance/e2e/tests/),
assembles artifacts, and reports results.
Running a single test
If arguments were passed to /e2e (e.g. /e2e html-post-processing-hook),
use --grep to filter to only matching test(s):
./app/acceptance/e2e/_module/scripts/slowcheck --run-notes "<note>" --grep "<pattern>"The grep pattern is passed to Playwright's --grep flag, which matches
against test titles. When --grep is active, the artifact count guardrail
is skipped (since not all specs will run).
Running tests for specific concepts
If the user asks to run tests for specific concepts or scenarios (they may not
use exact names), use --scenarios to filter by acceptance concept ID:
./app/acceptance/e2e/_module/scripts/slowcheck --run-notes "<note>" --scenarios callout publishingThis finds all spec files that import the given concepts and runs only those.
Available IDs are exported in app/concepts/index.ts; an invalid name prints
the complete valid list.
This is useful for faster feedback when you know which area of the app changed.
--scenarios and --grep are mutually exclusive.
Highlighting specs for reviewer focus
When you run the full suite from a feature or bugfix branch, use --highlighted
to mark the specs the reviewer should look at first. Unlike --scenarios, this
does not filter the run — every spec still runs. It only annotates the
report viewer so the highlighted specs land in a dedicated "Highlighted" section
above the normal Failing / Passing sections (thumbs, list, and videos tabs),
and their concept chips glow amber in the filter chips.
./app/acceptance/e2e/_module/scripts/slowcheck --run-notes "<note>" --highlighted delete-then-republishPass one or more spec basenames (the filename without .spec.ts). If a name
doesn't match any spec, the CLI prints the full list of valid basenames and
exits. --highlighted and --scenarios are mutually exclusive — highlighting
only makes sense when you're running a large/whole-suite run. Prefer using it
whenever you're working on a specific spec during a feature/bugfix branch so
/packet surfaces the scenario immediately for review.
Step 2: If tests fail, fix them
Flaky tests are never acceptable. A test that passes sometimes and fails sometimes is not "basically passing" — it is broken and must be fixed before you move on. Flaky tests destroy trust in the suite and slow down shipping. They are the single most corrosive thing that can happen to a codebase. Do not dismiss a failure as "just flaky" and proceed. If a test fails, it is your job to make it pass reliably, every time, under full parallel load.
Never conclude a test is "flaky but fine." If a test fails in the full suite but passes solo, that is a real bug — likely a timing issue, a missing wait, or shared state pollution. Diagnose and fix the root cause.
When a test fails, don't stop. Diagnose and fix the failure:
- Read the test output carefully — Playwright prints the failing assertion, the test file, and line number.
- Read the failing test spec in
app/acceptance/e2e/tests/. - Determine whether the failure is in the test or in the application code:
- Timing issue in test: add appropriate waits, fix selectors, or adjust assertions. A test that fails under parallel load but passes solo has a timing bug — find it and fix it.
- Application bug: trace through the relevant app code and fix it.
- Infrastructure issue (Docker not running, port conflict): report to the user rather than trying to fix.
- After fixing, re-run the full suite (not just the single test) to confirm the fix works under parallel load.
- Run
./quickcheckfrom the repository root after changes, following any enclosing checkout's required validation instructions. A scoped acceptance quickcheck may help during iteration but does not replace the required check.
Stability agent task
If you encounter flaky failures and need a dedicated, thorough stability pass,
run the e2e_full_suite_stability agent task:
tools/agent_task/run_agent_task app/acceptance/e2e/_module/agent_tasks/e2e_full_suite_stability.mdThis task is purpose-built for ferreting out flaky tests. It runs the full suite, verifies deterministically with the status command, diagnoses any failures, fixes them, and repeats until the suite is fully stable. Use it whenever you see intermittent failures — do not hand-wave them away.
Key files for debugging
app/acceptance/e2e/tests/*.spec.ts— test specsapp/acceptance/e2e/src/run/test-fixtures.ts— custom Playwright fixtures (artifactDir,checkpoint)app/acceptance/e2e/src/run/pages/— page object models used by testsapp/acceptance/e2e/playwright.config.ts— test configuration, Docker container setup~/meadow-e2e-artifacts/— test run output (videos, logs, captured state repositories)
Checking pass/fail after a run
The Playwright stdout may be truncated for large test suites. To reliably
determine which tests passed or failed, check status.txt inside each
test's artifact directory:
for dir in ~/meadow-e2e-artifacts/<run-id>/*/; do
name=$(basename "$dir")
st=$(cat "$dir/status.txt" 2>/dev/null || echo "no-status")
echo "$st $name"
doneDo NOT look for FAILED marker files — they don't exist. The status.txt
file contains the Playwright status string (passed, failed, etc.).
For failure details, check manifest.json in the test's artifact directory
(it contains the test source, home commits, and logs) and
error-context.md in app/acceptance/e2e/test-results/<test-dir>/
(it contains the page snapshot at the time of failure).