Index of every page on this site: llms.txt
All skills
Run it yourself

This skill should be used every time you create, update, or review end-to-end tests in this repository.

  • 1 file
  • 9.3 KB
  • Updated 9 months ago
  • GitHub
Use this Skill: https://skilld.dev/gh/alpha-innovation-labs/opennexus-rs/e2e-tests

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈29 tokens for metadata: the name and description. ≈2.3k when used: this file.

Description uses 1.4% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈115 characters, or 1.4%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

E2E Test Standards

Purpose

This project uses exclusively E2E (end-to-end) tests. We do not write unit tests or integration tests. E2E tests verify complete user-facing workflows using real services, real data, and real API calls.

Core Principles

  1. E2E tests only - No unit tests, no integration tests. Every test validates a complete scenario from start to finish.
  2. No mocks - Tests must use real services, real databases, and real API calls. Mocking defeats the purpose of E2E testing.
  3. Real-world data - Prefer production-like data over synthetic test fixtures. Tests should reflect actual usage patterns.
  4. Context-driven - Every test originates from Next Actions defined in a context file (see skills/nexus/context-driven-development/SKILL.md).

Test Source: Context Files

  1. Tests derive from context files - The ## E2E Test Scenarios table in each context file defines what tests to write.
  2. One test file per scenario - Each scenario becomes its own test file.
  3. Scenario table format - Each scenario has a Name (snake_case, no test_ prefix) and Description column. The test_ prefix is automatically added when generating filenames and test functions. Never use test_ in the Name column.
  4. Test folder inside crate's tests/ directory - In a Rust workspace, tests live inside each crate:
    • .context/nexus-tui/TUI_001-sidebar-navigation.md → crates/nexus-tui/tests/tui_001_sidebar_navigation/
    • .context/nexus-llm/RIG_001-oauth-authentication.md → crates/nexus-llm/tests/rig_001_oauth_authentication/
    • Context ID uses lowercase with underscores in folder name (e.g., TUI_001 → tui_001)
  5. Test file naming - Each scenario file is named test_<name>.rs where <name> is from the Name column.
  6. mod.rs required - Each context folder must have a mod.rs that declares all test modules.

Example context scenario table:

## Next Actions

| Description | Test |
|-------------|------|
| User creates session and knowledge is extracted | `session_creates_knowledge` |
| Malformed session is skipped with warning logged | `malformed_session_skipped` |
| Rate-limited API request retries with backoff | `rate_limit_retry` |

This produces the following test structure:

crates/nexus-rag/tests/              # Inside the crate directory
└── rag_002_session_watcher/         # Context ID (lowercase, underscores)
    ├── mod.rs                       # Declares: mod test_session_creates_knowledge;
    ├── test_session_creates_knowledge.rs
    ├── test_malformed_session_skipped.rs
    └── test_rate_limit_retry.rs

Run tests with: cargo test -p nexus-rag --test rag_002_session_watcher

Each file contains a test function with the test_ prefix added:

#[test]
fn test_session_creates_knowledge() {
    // Test implementation
}

Naming Validation

CRITICAL: Never use test_ prefix in the Name column

The test_ prefix is automatically added to:

  • Filenames: session_creates_knowledge → test_session_creates_knowledge.rs
  • Test function names: test_session_creates_knowledge()

Using test_ in the Name column causes double-prefix errors:

  • Wrong: test_menu_opens_view → test_test_menu_opens_view.rs ❌
  • Correct: menu_opens_view → test_menu_opens_view.rs ✓

The test-sync tool validates this and will report mismatches if found.

No Mocks Policy

  1. Real services required - Tests connect to actual services (databases, APIs, message queues).
  2. Real network calls - HTTP requests hit real endpoints, not mock servers.
  3. Real file system - Tests write to and read from actual files, not in-memory fakes.
  4. Real data stores - Use actual databases with test data, not in-memory substitutes.

User Input Exception

  1. Skip or prompt for user-required input - When a test requires explicit user action (e.g., OAuth authentication with a third party, manual approval flows, CAPTCHA):
    • Option A: Skip - Mark the test as skipped with clear documentation explaining why
    • Option B: Prompt - Request the required input from the user at test runtime

Example skip annotation (Rust):

#[test]
#[ignore = "Requires manual OAuth login - run with --ignored when credentials available"]
fn test_third_party_oauth_flow() {
    // Test implementation
}

Example prompt pattern:

fn get_oauth_token() -> String {
    if let Ok(token) = std::env::var("OAUTH_TOKEN") {
        token
    } else {
        eprintln!("OAUTH_TOKEN not set. Please authenticate and provide token:");
        // Read from stdin or fail gracefully
    }
}

Test Implementation Guidelines

  1. Functional program testing - Each test validates that a functional program/scenario works correctly end-to-end.
  2. Setup real environment - Tests may need to provision real resources (start services, create test databases, seed data).
  3. Clean up after tests - Remove test artifacts, reset state, but preserve real service availability.
  4. Accept latency - E2E tests are slower than unit tests. This is expected and acceptable.
  5. Retry for flakiness - Network-dependent tests may use retries with backoff for transient failures.

What Tests Must Verify

  1. User-observable outcomes - Test what users see and experience, not internal implementation.
  2. Complete workflows - From initial input to final output, including all intermediate steps.
  3. Error conditions - Verify graceful handling of real failures (network errors, invalid data, service unavailability).
  4. Edge cases in production context - Test edge cases using real services, not mocked edge responses.

Public API Only

  1. No private member access - E2E tests must only use the public API (pub items exported from lib.rs). Never access private modules, internal constants, or unexported functions.
  2. Test rendered output, not internal state - Instead of checking internal constants like BULLET_MARKERS, verify the rendered output contains the expected characters (●, ○, ◆, ◇).
  3. If a test needs private access, it's testing implementation - Rewrite to test observable behavior, or move to unit tests inside src/ if truly testing internal logic.

Example - Wrong (accesses private constant):

use nexus_tui::markdown::styled_line::BULLET_MARKERS; // ❌ Private module

#[test]
fn test_bullet_markers() {
    assert_eq!(BULLET_MARKERS[0], "● "); // ❌ Testing implementation
}

Example - Correct (tests observable output):

use nexus_tui::markdown::render_markdown_to_styled_lines;

#[test]
fn test_bullet_markers() {
    let lines = render_markdown_to_styled_lines("- Item\n");
    let rendered = lines[0].render(80);
    let text: String = rendered[0].spans.iter()
        .map(|s| s.content.to_string()).collect();
    
    assert!(text.contains("●"), "Should render with bullet marker"); // ✓ Observable
}

Anti-Patterns

  1. No mocking libraries - Do not use mockall, mockito, wiremock, or similar.
  2. No test doubles - No fakes, stubs, spies, or dummies.
  3. No in-memory databases - Use the same database technology as production.
  4. No HTTP record/replay - Tests hit live endpoints every run.
  5. No isolated unit tests - Every test must exercise the full stack.
  6. No private API access - Tests importing private modules (e.g., mod::internal::*) are testing implementation, not behavior.

Test Environment Isolation

  1. Isolate from user data - Tests must NEVER read from or write to real user directories (~/.config/, ~/.local/, etc.).
  2. Use environment variables - Override default paths via environment variables to redirect all I/O to /tmp/ or similar.
  3. Shared test utilities - Use tests/test_utils.rs to set up isolated test environments consistently.
  4. Guard pattern - The setup_test_env() function returns a guard that automatically cleans up environment variables when dropped.

Required Environment Variables

Tests should override these paths to prevent touching real user data:

Variable Purpose Test Value
NEXUS_CONTEXT_DIR Context file storage /tmp/nexus_test_context/
NEXUS_STATE_FILE TUI state persistence /tmp/nexus_test/state.json
NEXUS_SETTINGS_FILE User settings /tmp/nexus_test/settings.json
NEXUS_RIG_TOKEN_PATH OAuth tokens /tmp/nexus_test/tokens.json
NEXUS_RIG_SESSION_PATH Chat session storage /tmp/nexus_test/sessions

Test Setup Pattern

Every E2E test that interacts with persisted state must use the shared isolation:

mod test_utils;

#[test]
fn test_something() {
    let _guard = test_utils::setup_test_env();
    
    // Test code here - all paths automatically redirected to /tmp/
    // Guard cleanup happens automatically when test ends
}

What This Prevents

  • Tests resetting user's OAuth tokens (requiring re-authentication)
  • Tests corrupting user's TUI settings
  • Tests polluting user's session history
  • Tests launching browser for OAuth flow during CI

Validation

All E2E tests run via justfile recipes:

just test-e2e           # Run all E2E tests
just test-e2e-verbose   # Run with detailed output

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 8347f22. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last month.

Dormantupdated 9 months ago
compatibility
opencode

README badge

README badge for alpha-innovation-labs/opennexus-rs/e2e-tests