All skills
mblode avatar

/ax-audit

@57eb304
by Matthew Blodemblode/agent-skills134 stars
12

Audits agentic products for tool parity, authority, approval payloads, recovery, and trust using 27 rules and a ship verdict. Use when asked for an "AX audit", to review an agent approval flow, or whether an agent can operate the product. For human-facing API ergonomics use dx-audit; for ordinary UI use ui-design.

Use this Skill: https://skilld.dev/gh/mblode/agent-skills/ax-audit

This session only. Nothing lands on disk.

rules-axcontext-memory-not-visible.md

≈769 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Agent uses context the user can't see or edit

Agent injects preferences, past interactions, or learned patterns into its prompt, but the user can't see what the agent "knows" about them. Opaque memory feels invasive. Memory in Motion requires every piece of stored context to have a user-facing view and edit path.

What goes wrong

Agent says "Based on your preference for concise answers..." and the user thinks "What preference? I never said that." The system built a profile from past interactions and injected it into the system prompt with zero user visibility: no settings page, no memory panel, no way to correct it.

Detection

Surfaces: agent-chat, agent-config, agent-dashboard

Auditability: code-auditable

Static signals:

  1. Find context injection points (prompt builders, context loaders, preference injectors).
  2. Search for UI that exposes this context (settings pages, memory panels).
  3. Flag injected context with no user-facing view or edit path.

Concrete commands:

rg '(systemPrompt|buildPrompt|contextLoader|injectContext|userPreferences|userMemory)' --type=ts -l src/
rg '(MemoryPanel|PreferencesView|WhatIKnow|MemorySettings)' --type=ts -l src/
rg '(savePreference|updateMemory|storePattern|learnFrom)' --type=ts -l src/

False-positive guards:

  • Skip files with // ax-audit-ignore:context-memory-not-visible.
  • Skip test and Storybook fixtures.
  • Skip internal admin-only agent tools where the operator is the developer.

Fix

For every context item injected into the prompt, provide UI to view and edit it: a "Memory" or "What I know about you" panel with edit/delete per item.

Default tier and overrides

Defaults to: fix-this-sprint

Surface Tier
Agent chat fix-this-sprint
Agent config fix-this-sprint
Agent dashboard backlog

Examples

Anti-pattern (fails):

async function getAgentContext(userId: string) {
  const prefs = await redis.get(`user:${userId}:prefs`);
  const history = await redis.get(`user:${userId}:patterns`);
  return { preferences: prefs, patterns: history }; // never shown to user
}

Applied (passes):

// Context store is shared: same data feeds the agent AND the settings UI
async function getAgentContext(userId: string) {
  return await getVisibleMemory(userId); // MemorySettings reads the same store
}

function MemorySettings() {
  const memory = useMemory();
  return memory.items.map((m) => (
    <li key={m.id}>{m.summary} <button onClick={() => memory.delete(m.id)}>Delete</button></li>
  ));
}

Suppression

{/* ax-audit-ignore:context-memory-not-visible, internal dev tool, operator is the developer */}
<AgentPromptBuilder />

Source: SKILL.md on GitHub

No alerts13d3 checks · Risk SAFE
  • Gen Agent Trust Hub13d

    The skill is a specialized auditing framework for AI agent products, focusing on architectural integrity and user trust. It uses standard shell tools for static analysis of codebases. The analysis found no malicious behavior, obfuscation, or data exfiltration risks.

  • Socket13d

    No alerts

  • Snyk13d

    Risk: LOW · No issues

Signed by skilld at 57eb304. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 2 days ago.

Activeupdated 2 weeks ago

README badge

README badge for mblode/agent-skills/ax-audit