All skills
mblode avatar

/ax-audit

@57eb304
by Matthew Blodemblode/agent-skills134 stars
12

Audits agentic products for tool parity, authority, approval payloads, recovery, and trust using 27 rules and a ship verdict. Use when asked for an "AX audit", to review an agent approval flow, or whether an agent can operate the product. For human-facing API ergonomics use dx-audit; for ordinary UI use ui-design.

Use this Skill: https://skilld.dev/gh/mblode/agent-skills/ax-audit

This session only. Nothing lands on disk.

rules-axcontrol-thin-approval-payload.md

≈1.5k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Approval prompt names the action but not what it will do

The gate is there. It fires on the right actions, it matches the stakes, and it asks "Allow the agent to send an email?" with a recipient, a subject, and a body the user never sees. There is nothing to weigh, so the answer is yes every time, which makes the gate a click-through rather than a decision.

As the interface shrinks, the approval moment is one of the few surfaces left. It has to carry exactly enough of the action for a confident yes or no, and a tool name is not enough.

What goes wrong

The agent drafts a reply to a customer, gets it wrong, and asks for approval. The dialog reads "Send email? Allow / Deny". The user, six approvals into a session where the previous five were fine, allows it. The wrong reply goes to the customer. The gate did its job and prevented nothing, because the one fact that would have changed the answer, the body of the message, was the fact the dialog left out.

The reverse also happens: the dialog is so thin the user denies everything, and the agent becomes unusable for exactly the tasks it exists to do.

Detection

Surfaces: agent-chat, agent-tool-execution

Auditability: code-auditable

Static signals:

  1. Find approval and confirmation components: ApprovalDialog, ConfirmAction, ToolApproval, PermissionPrompt, anything rendering an allow and deny pair.
  2. Read the props each one accepts, and the props the call site passes.
  3. Flag components whose only action-describing input is a name or type (toolName, action, actionType, title) with no channel for the call arguments.
  4. Flag call sites that have the arguments in scope and pass only the name.
  5. For destructive actions, check that the target is named specifically (which record, how many rows), not by category.
  6. In AI SDK 7 renderers, find the branch on part.state === "approval-requested": part.input is in scope there, and a branch that prints the tool name without it is the finding. In Claude Agent SDK canUseTool(toolName, input) handlers, check that input reaches the prompt, not only toolName. MCP asks clients to show tool inputs before calling the server; a host that shows the tool title is not doing that.

Concrete commands:

# candidate approval surfaces: record this list, it is the evidence
rg -l 'Approval|ConfirmAction|PermissionPrompt|requiresApproval' --type=ts src/

# per candidate, does the surface ever reference the call's arguments?
# absence is the finding, so check each file rather than grepping for a negative
for f in $(rg -l 'Approval|ConfirmAction|PermissionPrompt' --type=ts src/); do
  rg -q 'args|payload|preview|diff|target|params|describeIntent' "$f" \
    || echo "thin approval surface: $f"
done

# the prop surface, to see what a call site is even able to pass
rg -n -A 8 'interface .*(Approval|Confirm).*Props' --type=ts src/

# gates that stringify the tool and stop there
rg -n 'confirm\(|window\.confirm' -A 2 --type=ts src/

# framework approval branches: read each for input/args rendering
rg -n 'approval-requested|canUseTool' -A 8 --type=ts src/

Judgment signals:

  • A payload rendered but truncated to one line for a multi-paragraph action is a partial pass; note it as warn, not fail.
  • A diff view for edits and a plain summary for sends both count, as long as what changes is legible before the answer.

False-positive guards:

  • Skip gates for low-stakes reversible actions where a receipt with undo is the right pattern instead. That is control-no-approval-gate territory, and this rule should not push toward more friction than the stakes need.
  • A component that accepts a children or preview slot passes if the call sites populate it. Check the call sites, not just the type.
  • Approve-with-changes passes: a gate whose fields are editable and flow back as updatedInput (Agent SDK) or an approval reason (AI SDK) carries more than the payload this rule asks for.
  • Skip files with // ax-audit-ignore:control-thin-approval-payload near the match.
  • Skip test and Storybook fixtures.

Fix

Pass the call arguments through to the gate and render the ones that would change the answer.

// before: the user approves a category
<ApprovalDialog
  toolName="send_email"
  onApprove={() => execute(call)}
  onDeny={reject}
/>

// after: the user approves this specific act
<ApprovalDialog
  toolName="send_email"
  summary={`Reply to ${call.args.to} about "${call.args.subject}"`}
  onApprove={() => execute(call)}
  onDeny={reject}
>
  <RecipientList to={call.args.to} cc={call.args.cc} />
  <BodyPreview body={call.args.body} expandable />
</ApprovalDialog>

Default tier and overrides

Defaults to: fix-this-sprint

Surface Tier
Agent tool execution release-blocker
Agent chat fix-this-sprint
Agent config fix-this-sprint
Agent dashboard fix-this-sprint

Examples

Anti-pattern (fails):

function ToolApproval({ call, onApprove, onDeny }: Props) {
  return (
    <Prompt>
      <p>Allow the agent to run {call.name}?</p>
      <Button onClick={onApprove}>Allow</Button>
      <Button onClick={onDeny}>Deny</Button>
    </Prompt>
  );
}

call.args is in scope and never rendered.

Applied (passes):

function ToolApproval({ call, onApprove, onDeny }: Props) {
  return (
    <Prompt>
      <p>{describeIntent(call)}</p>
      <ArgumentPreview args={call.args} highlight={call.destructiveFields} />
      <Button onClick={onApprove}>Allow</Button>
      <Button onClick={onDeny}>Deny</Button>
    </Prompt>
  );
}

Suppression

{/* ax-audit-ignore:control-thin-approval-payload, no arguments, the action is the whole payload */}
<ApprovalDialog toolName="end_session" onApprove={stop} onDeny={reject} />

Source: SKILL.md on GitHub

No alerts13d3 checks · Risk SAFE
  • Gen Agent Trust Hub13d

    The skill is a specialized auditing framework for AI agent products, focusing on architectural integrity and user trust. It uses standard shell tools for static analysis of codebases. The analysis found no malicious behavior, obfuscation, or data exfiltration risks.

  • Socket13d

    No alerts

  • Snyk13d

    Risk: LOW · No issues

Signed by skilld at 57eb304. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 2 days ago.

Activeupdated 2 weeks ago

README badge

README badge for mblode/agent-skills/ax-audit