All skills
mblode avatar

/ax-audit

@57eb304
by Matthew Blodemblode/agent-skills136 stars
12

Audits agentic products for tool parity, authority, approval payloads, recovery, and trust using 27 rules and a ship verdict. Use when asked for an "AX audit", to review an agent approval flow, or whether an agent can operate the product. For human-facing API ergonomics use dx-audit; for ordinary UI use ui-design.

Use this Skill: https://skilld.dev/gh/mblode/agent-skills/ax-audit

This session only. Nothing lands on disk.

referencesagent-native-principles.md

≈1.8k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Agent-Native Principles (Condensed)

Condensed from the Every agent-native guide (https://every.to/guides/agent-native), with Anthropic's tool-writing and context-engineering guidance folded in where it sharpens a rule.

<!-- TOC --> <!-- /TOC -->

Core Principles

  • Parity: every UI capability has a tool; if not, add one.
  • Granularity: atomic primitives, one action per tool; decision logic lives in prompts, so behavior changes are prompt edits, not refactors.
  • Composability: atomic tools plus parity make new features new prompts.
  • Emergent capability: ship atomic tools, watch requests, add domain tools for common patterns.
  • Improvement over time: context files plus refined prompts improve the app without code; self-modification needs audit logs and rollback.

Tool Design

Atomic primitives first. One action per tool, scoped to a domain noun: read_note, update_note, list_projects. Prove the architecture on these before bundling them into workflow tools.

Atomic is not raw. A domain primitive has a blast radius you can state in a sentence. Raw substrate access over production data (arbitrary code eval, a shell on the live host, a SQL or GraphQL passthrough, an untyped SDK call) has the blast radius of the whole system and makes every tool boundary above it advisory. Ship the first. Some long-tail requests then stop being servable; that is a refusal, not a reason to add the substrate tool back.

CRUD completeness. Verify every entity has create, read, update, delete. Common failure: create_note + read_notes exist but update_note and delete_note are missing.

Domain tools. Add deliberately for vocabulary anchoring, guardrails (validation not left to judgment), or bundling a multi-step operation.

Consolidate mechanics, not judgment. Anthropic's guidance for agent tools runs the other way from "one tool per endpoint": a few tools aimed at whole workflows beat many thin wrappers, and bloated, overlapping tool sets are the first failure mode it names. The two agree once the axis is clear. Chaining mechanical steps into one user action (schedule_event finds availability and books) is consolidation the agent loses nothing to; folding a decision into code (which role, what counts as stale) moves judgment out of the prompt and is what granularity-workflow-shaped-tool catches.

Tool results are a contract. MCP gives the shape a name: structuredContent under an outputSchema on success, isError: true on failure (assumed false when unset), and annotations that describe stakes (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) which clients treat as hints, not authority. parity-unstructured-tool-output and comm-no-approval-gate audit against that shape whatever the stack.

Dynamic capability discovery. For evolving type systems (CMS, CRM custom objects), expose list_available_types() + read_data(type) over one wrapper per type. A product agent whose tools are the product nouns (create_issue, update_document) does not need to discover them at runtime.

MCP standardizes discover and access for external services. Prefer those servers over hand-coded wrappers.

Graduation. Hot paths can move to optimized code, but the agent still triggers them and falls back to domain primitives for edge cases.

Context Patterns

Entity-scoped directories. {entity_type}/{entity_id}/; separate ephemeral (AgentCheckpoints/, AgentLogs/) from durable (Research/).

The context.md pattern. Read at session start, updated as state changes (agent identity, user knowledge, what exists, recent activity, current state). Portable working memory, no code changes.

Context injection. System prompts include three sections, scoped to this run, not the whole product:

  1. Available resources: what data exists, where
  2. Capabilities: what the agent can do
  3. Recent activity: what happened since last session

Context is a budget with two ends. Too little and the agent asks redundant questions. Too much (every tool, every procedure, every run) and it picks the wrong one from a longer list.

Durable state outside the transcript. A session that outgrows its window silently drops the earliest decisions, so anything the agent must still honour at step 40 belongs in a file it re-reads, not in message history (see rules-arch/context-no-checkpoint-resume). Audit the store, not the window-management technique.

Agent-UI Communication

Completion signals. Every model API returns a terminal reason for a turn (stop_reason in the Messages API, finishReason in AI SDK). The audit question is whether the orchestrator reads that field, or infers "done" from idle time, an empty delta, or a timeout. Reading it is not enough on its own: pause_turn means continue and max_tokens means truncated, so a loop that treats every non-tool reason as completion has smuggled the heuristic back in. Orchestrators that add their own states (pause, escalate, retry) must emit them as explicitly as completion, or the UI guesses again.

Partial completion tracking. Per-task status (pending, in_progress, completed, failed, skipped); show 3/5 tasks complete (60%) with error notes.

Agent event types. Emit typed events (thinking, toolCall, toolResult, textResponse, statusChange); an ephemeralToolCalls flag hides noisy internals.

Shared workspace. Agents and users share one data space, each building on the other's work. Sandbox only when security or data integrity requires it.

Stable names on the pixel surface. When an agent must operate the same UI a human sees, expose durable names in the DOM (data-agent-id, aria names that match the tool nouns) rather than making the agent guess from CSS and copy. Screenshot-only targeting drifts when layout shifts; a named control is a tool. This is the product-side half of annotating a UI for agents, not a visual-polish concern.

Approval gates. Match approval to stakes and reversibility:

Stakes Reversibility Pattern
Low Easy Auto-apply
Low Hard Quick confirm
High Easy Suggest + apply (show diff)
High Hard Explicit approval

An explicit user request is already approval. Self-modification always requires explicit approval + audit log + rollback.

Provenance is the third axis. Stakes and reversibility classify by tool name, so a gate built from them treats every delete the same. Two questions adjust the row:

  • Did the agent create this in the current conversation? Deleting a draft it just made is not the same act as deleting a record that existed before the session.
  • Does the target reach outside the workspace? Posting to an internal thread and posting to one synced with a public repo differ in blast radius, not in the tool name.

A gate that cannot read those two facts can only be tuned by making it stricter, and a gate strict enough to catch the pre-existing delete will also interrupt the draft cleanup. Pass provenance into the checkpoint alongside stakes.

Source: SKILL.md on GitHub

No alerts13d3 checks · Risk SAFE
  • Gen Agent Trust Hub13d

    The skill is a specialized auditing framework for AI agent products, focusing on architectural integrity and user trust. It uses standard shell tools for static analysis of codebases. The analysis found no malicious behavior, obfuscation, or data exfiltration risks.

  • Socket13d

    No alerts

  • Snyk13d

    Risk: LOW · No issues

Signed by skilld at 57eb304. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 5 hours ago.

Activeupdated 2 weeks ago

README badge

README badge for mblode/agent-skills/ax-audit