Testing, Corpus, Performance & Scorecard Infrastructure (egui & Rust)
An engine-first multi-crate egui application enforces quality through a 7-tier verification architecture. Use this skill whenever you write tests, run verification gates, update corpus/parity floors, benchmark hot paths, or update docs/scorecard.md.
Related skills:
- egui-app-architecture —
L0–L7crate layering,Testkitrules, and lint/profile configuration. - egui-app-runner — Running the desktop app, CLI, offscreen snapshots, and live MCP/TCP control servers.
- egui-command-engine — Authoring
CommandSpecs validated bypanic_huntand parity checks. - egui-cow-document-model — Copy-on-write chunks, snapshot undo/redo history, and exhaustive format conversion.
1. The 7 Verification Tiers
| Tier | Scope & Mechanism | Primary Files & Commands |
|---|---|---|
| 1. Compile-Time & Static Gates | L0–L7 acyclic layering (cargo xtask layers), wasm32 check (cargo xtask wasm), unsafe_code = "forbid", #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, ...)] across all workspace crates (relaxed in tests via clippy.toml), and exhaustive struct destructuring in crates/format/src/convert.rs. |
xtask/src/layers.rs, clippy.toml, crates/format/src/convert.rs |
| 2. Unit, Property, Mutation & Format Fuzz Tests | Domain unit tests across supported data representations, cargo-mutants mutation coverage (0 missed mutants), proptest! suites (COW chunks/tiles, spatial regions, codecs, file formats), deterministic synthetic document generators (testgen), truncation/corruption sweeps, and cargo-fuzz targets (crates/*/fuzz/). |
crates/*/src/, .cargo/mutants.toml, crates/*/fuzz/ |
| 3. CPU vs. GPU Rendering Parity | CPU reference renderer vs. wgpu WGSL shader output parity within a bounded tolerance. Every test using a wgpu device or egui_kittest GPU harness must hold let _gpu = testkit::gpu_lock(); on its first line. |
crates/testkit/src/gpu.rs, .config/nextest.toml |
4. Pinned Real-File Oracles (corpus) |
Gitignored real-world files in corpus/ fetched at pinned commits and verified against xtask/*.sha256 manifests. Asserts monotonically increasing oracle pass and export round-trip floors, strict structural validation, and mutation safety. |
xtask/src/corpus_pins.rs, crates/io/tests/corpus.rs |
5. Command Registry & panic_hunt |
crates/engine/tests/panic_hunt.rs fuzzes every registered CommandSpec against adversarial JSON params on its own thread with a per-command deadline (e.g., 4 s) and a custom std::panic::set_hook counter that catches even panics swallowed by Session::execute's catch_unwind. |
crates/engine/tests/panic_hunt.rs |
6. Headless UI (egui_kittest) & MCP Acceptance |
Widget/canvas tests in crates/ui-egui, offscreen wgpu PNG snapshots (examples/snapshot.rs), menu parity ratchet (parity::FLOOR), and end-to-end agent workflows driven over in-memory rmcp transport (crates/automation/tests/agent_tasks.rs). |
crates/ui-egui/src/parity.rs, crates/ui-egui/examples/snapshot.rs, crates/automation/tests/agent_tasks.rs |
| 7. Performance Budgets & Scorecard | Release benches report p50/p95/max ms, in-process peak RSS (testkit::perf::RssSampler + Linux VmHWM), and GPU bytes against perf/budgets.toml and perf/baseline.json. cargo xtask scorecard compiles budgets, corpus floors, checklists (scorecard/*.toml), and unread-preference audits into docs/scorecard.md. |
crates/testkit/src/perf.rs, xtask/src/perf.rs, xtask/src/scorecard.rs, perf/budgets.toml |
2. Fast Local Iteration vs. Pre-Finish Gate
Fast Iteration Loops
Avoid compiling all workspace test binaries and examples while iterating:
# 1. Run unit + integration tests for a single crate (skips compiling examples and doctests)
cargo test -p <crate> --lib --tests
# 2. Run a single integration test binary
cargo test -p <crate> --test <test_file_stem>
# 3. Run parallel process-per-test execution via nextest (caps concurrent GPU tests)
cargo nextest run -p <crate> --lib --tests
# 4. If working alongside parallel agents, isolate your build directory
CARGO_TARGET_DIR=target/agent-<name> cargo test -p <crate> --lib --testsMandatory Pre-Finish Verification Gate
Before finishing a task, run the checks corresponding to the crates you touched:
# Always run on touched crates:
cargo test -p <crates>
cargo clippy -p <crates> --all-targets -- -D warnings
cargo xtask layers
# If you touched L0–L6:
cargo xtask wasm
# If you added or modified engine commands:
cargo test -p <app>-engine --test panic_hunt -- --ignored
cargo xtask parity
# If you touched format parsers, io, rendering, or persistence crates:
CARGO_PROFILE_RELEASE_LTO=false CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 \
cargo xtask test-corpus --changed
# If you touched a hot path, fixed a scorecard item, raised a floor, or wired a preference:
cargo xtask perf --quick
cargo xtask scorecard
cargo xtask scorecard --check3. Critical Rules When Writing Tests
- Always Hold
gpu_lock()in GPU Tests: Multiple concurrentwgpudevices in one process crash Windows WARP (STATUS_ACCESS_VIOLATION), Mesallvmpipe/RADV, and multi-ICD Vulkan. Undercargo test(multi-threaded within one process), every test creating awgpudevice oregui_kittestGPU harness must acquiretestkit::gpu_lock()on its first line:
If you add a new GPU test module in#[test] fn my_gpu_test() { let _gpu = testkit::gpu_lock(); // build wgpu device or egui_kittest harness... }ui-egui, ensure its name matches thegputest-group filter in.config/nextest.toml. - Isolate OS Side Effects in Tests & Fuzzers:
Commands that trigger external OS side effects (such as physical printing, system dialogs, or network/external process spawns) must be stubbed or disabled in dry-run/fuzz mode (e.g., calling a
disable_*()hook or setting aNO_*=1environment guard) before runningpanic_hunt. - Test Full Domain Representations & Edge Cases: Domain algorithms and commands should be tested across all supported precision modes, coordinate spaces, and entity variants where applicable.
- Every Crash Fix Requires a Synthetic Regression Test:
Never commit large external corpus files into the repository. Reproduce any corpus or fuzz crash with a small, deterministic synthetic test in the crate (e.g.,
crates/io/tests/corpus_regressions.rs) that failed or panicked before the fix. - Ratchets Only Move Up:
Never lower
parity::FLOORincrates/ui-egui/src/parity.rsor anypass_floor/roundtrip_floorincrates/io/tests/corpus.rs. When your change improves pass counts, raise the floor and regeneratedocs/scorecard.mdviacargo xtask scorecard. - Mutation Testing (
cargo-mutants) & Headlessegui_kittestIdioms:- Prerequisite Installation: Ensure
cargo-mutantsis installed (cargo install --locked cargo-mutants) and documented whenever onboarding or writing verification instructions. - Sequential
cargo mutantsRuns &--timeout 120: On a warmtarget/cache, the unmutated baseline test finishes in< 2s, causingcargo-mutantsto auto-set a20stest timeout. Running multiplecargo testorcargo mutantsjobs concurrently causes linker/lock contention that triggers falseTIMEOUTresults. Always runcargo mutants --in-placesequentially (one crate at a time after allcargo testbuilds finish) with--timeout 120. - Sandboxed Offline Execution on macOS (Optional Agent Workaround): In restricted agent sandboxes where
/tmp(/private/tmp) or~/.cargocannot be canonicalized (Operation not permitted), seed~/.cargo/registryonce via APFS copy-on-write (cp -cR) into a writable scratch directory and pointCARGO_HOMEandTMPDIRthere:
(Note: Cargo hashes theCARGO_HOME=<scratch>/cargo-home TMPDIR=<scratch>/tmp \ cargo mutants --in-place --timeout 120 -d <crate_dir> -o <scratch>/mutants-<name> -C --offlineCARGO_HOMEpath intotarget/debug/.fingerprint/, so if you seedtarget/directories across isolated builds usingcp -cR, keepCARGO_HOMEidentical across them.) - Headless
eframe::AppVerification viaegui_kittest::Harness::new_eframe: Addegui_kittest = { version = "0.37", features = ["eframe"] }to[dev-dependencies]and boot the full app headlessly viaHarness::new_eframe(|_cc| MyApp::default()). OnceHarness::new_eframetests query rendered labels (harness.get_by_label(...)), simulate clicks (.click()), and press keys (harness.key_press(egui::Key::...)), remove"<impl eframe::App for MyApp>::ui"from.cargo/mutants.tomlexclude_resocargo-mutantsverifiesui()and panel UI methods directly, leaving only pure pixel-coordinatePainterhelpers (e.g.,render_canvas,paint_viewport) excluded until offscreen GPU snapshot tests come online. egui::Scene::showClosure Borrow Split (E0500Prevention): Passing&mut self.view.scene_rectdirectly toegui::Scene::new().show(ui, &mut self.view.scene_rect, |ui| { ... })borrowsself.view.scene_rectfor the entire.show(...)call, preventing the inner closure from calling methods onself. Becauseegui::RectisCopy, copylet mut scene_rect = self.view.scene_rect;beforeScene::show, pass&mut scene_rect, and assignself.view.scene_rect = scene_rect;afterward.- Configure
.cargo/mutants.tomlwith Exact--listMutant Names:exclude_rematches the generated mutant description fromcargo mutants --list, not raw Rust syntax:- To exclude
fn main(), match"replace main"(not"fn main"). - To exclude a trait method like
eframe::App::ui(beforeegui_kittestis introduced), match"<impl eframe::App for MyApp>::ui"(not"impl eframe::App for MyApp::ui"). - Set
copy_target = truein.cargo/mutants.tomland gitignoremutants.out/andmutants.out.old/.
- To exclude
- Avoid
..Default::default()Inside Untested Entrypoints (fn main):cargo-mutants(v27.1+) generatesStructField(delete field <field> from struct <Struct> expression) mutants on struct literals using..Default::default(), andStructFieldmutants bypassexclude_refiltering. Infn main(), initializeeframe::NativeOptionsvialet mut options = eframe::NativeOptions::default(); options.viewport = ...;rather than struct update syntax somain()does not produce an un-excludable missed mutant. - Delegate
impl Default/fn newLogic to Named Methods:cargo-mutantsskipsimpl Default(fn default) andfn newcompletely to avoid infinite recursion withDefault::default(). Put non-trivial initialization or state-reset logic in a named method (e.g.,pub fn reset_view(&mut self)) called byDefault::default()socargo-mutantscan mutate and verify it. - Designing Arithmetic, Boolean & Match-Arm Tests for Zero Missed Mutants:
- Arithmetic (
Pos2,Vec2,Rect,TSTransform,hypot): Never test at0.0or1.0coordinates/factors (where0.0 - x == 0.0 / xorx * 1.0 == x / 1.0). Use non-zero, asymmetric values (e.g.,pos2(100.0, 60.0),vec2(16.0, 10.0),zoom_factor = 2.0, and3-4-5right triangles at non-zero offsets) so every+/-/*///%mutant produces a distinct float caught byassert_eq!. - Compound Booleans (
&&vs.||and!): For&&predicates likeself.is_item_visible(item) && item.contains_point(pt), include negative tests for both(true, false)and(false, true). For multi-branch||search filters (e.g.,matches_querycheckingname || category || tag), include one query that matches exclusively via each branch plus a non-matching query so every|| -> &&and!negation mutant is caught. - Exhaustive
matchArms: Assert each enum variant's behavior in isolation so nomatchbranch (e.g., inz_order(),color(),label(), or visibility predicates) can be deleted or collapsed.
- Arithmetic (
- Rust 1.97+
float_literal_f32_fallbackWarning withegui::Stroke::new: BecauseStroke::newtakes genericimpl Into<f32>, untyped float literals likeStroke::new(2.0, color)trigger#[warn(float_literal_f32_fallback)]onrustc 1.97+. Use explicit_f32suffixes (Stroke::new(2.0_f32, color)) or typedf32variables.
- Prerequisite Installation: Ensure
4. Deep-Dive Reference
For detailed workflows on:
- Fetching, authoring, and debugging real-file corpora (
oracle_diff, local corpus testing, pin bumps) - Adding performance scenarios (
testkit::perf,perf/budgets.toml,perf/baseline.json) - Updating
scorecard/*.tomland understanding CI matrix optimizations (ci.yml,perf-nightly.yml)