Index of every page on this site: llms.txt
All skills
pydantic avatar

/pydantic-ai-harness

@b2d64a3
by pydanticpydantic/pydantic-ai21k stars
3,221
Run it yourself

Extend Pydantic AI agents with capabilities from pydantic-ai-harness -- the Coder coding agent, file and shell tools in local or sandboxed workspaces (Modal, E2B, Render, Sprites, Daytona, boxd, smolvm, BoxLite), Code Mode, sub-agents and planning, model routing, memory and skills, context compaction, guardrails and spend limits, web research and browsers, hosted SaaS integrations, step persistence, and background agents on Render Workflows. Use when the user mentions pydantic-ai-harness or pydantic_ai_harness, imports a harness capability such as Coder, CodeMode, FileSystem, Shell, SubAgents, ModelRouter, Memory, or ToolGuardrail, or wants a Pydantic AI agent that edits files, runs commands or agent-written Python, delegates, routes work between models, remembers, manages long context, stays within limits, or runs model and tool operations as chained Render task runs.

  • 11 files
  • 334 KB
  • MIT
  • Updated 2 days ago
  • GitHub
Use this Skill: https://skilld.dev/gh/pydantic/pydantic-ai/pydantic-ai-harness

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesRUNTIME-AND-EXTENSION.md

≈10k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Runtime and Extension

How a harness agent is routed, persisted, made durable, configured, extended, and served. This covers model-based selection (ModelRouter), saving and resuming runs (StepPersistence), AWS Lambda durable functions (AWSLambdaDurability), Absurd durable tasks (AbsurdDurability), pgtask durable tasks (PGTaskDurability), Render Workflows, harness capabilities under core durable execution, Logfire-managed instructions (ManagedPrompt), agent-written capabilities (CapabilityCreation), loading harness capabilities from YAML/JSON specs, serving an agent to editors over ACP, and running one as a GitHub Agentic Workflow.

Choose

I want to ... Use
Let another model choose the main agent's model from a named menu ModelRouter
Resume, continue, or fork a run from saved history; audit tool side effects after a crash StepPersistence
Survive worker crashes with automatic replay (Temporal, DBOS, Prefect) core durability capability; most harness capabilities work inside it
Checkpoint every model/tool step on AWS Lambda durable functions AWSLambdaDurability
Checkpoint every model/tool step in Postgres with Absurd AbsurdDurability
Checkpoint every model/tool step in Postgres with pgtask PGTaskDurability
Run agent operations as activities in native Azure Durable Task orchestrators AzureDurableTaskDurability and run_durable (pydantic_ai_harness.azure_durable_task, [azure-durable-task])
Use or evaluate Render Workflows for background agents with separate model/tool retries, timeouts, and compute RenderWorkflows; retrying the entry task run restarts the agent
Edit, version, and roll out the system prompt from Logfire without redeploying ManagedPrompt
Let the agent write new capabilities that load on the next run CapabilityCreation
Define the agent in YAML/JSON with harness capabilities Agent.from_file(..., custom_capability_types=[...])
Use the agent inside Zed or another ACP editor pydantic_ai_harness.experimental.acp
Run the agent headless on GitHub issues/PRs/schedules the gh-aw pydantic-ai engine

A model error's retry_after (a provider's Retry-After) is waited durably before the engine's next attempt at the model request: on AWS Lambda as the step retry's timed suspension, on Azure Durable Task as a durable timer within the model RetryPolicy, on Absurd and pgtask as a recorded failed attempt and a sleep_for that suspends the task, at most as many times per model request as the task has attempts left. Render Workflows can't, so its task Retry applies. Set model_settings={'max_retries': 0} so the provider SDK doesn't also retry inside the durable unit.

ModelRouter

ModelRouter asks a Pydantic AI model to choose from named ModelChoice entries, then uses the selected model for the main agent request. The default 'once' mode pays for one routing request per run; mode='per_step' routes again before each logical model request.

from pydantic_ai import Agent

from pydantic_ai_harness import ModelChoice, ModelRouter

agent = Agent(
    capabilities=[
        ModelRouter(
            choices={
                'fast': ModelChoice('openai:gpt-5.6-luna', 'Routine lookups and extraction.'),
                'capable': ModelChoice('openai:gpt-5.6-sol', 'Complex or high-stakes work.'),
            },
            router_model='openai:gpt-5.6-luna',
            default='capable',
        )
    ]
)

The router receives the parent run's message history, including files, and its request counts toward the same usage limits. Keep it in the same trust boundary as the selectable models: model selection runs before input guardrails and request wrappers. Router failures and picks below probability_threshold use default; configuration errors and request-time UserErrors propagate. Under core durable execution the routing call is durable, and every selectable model must also be registered in the durability capability's models= mapping. ModelRouter is Python-only and cannot be loaded from an agent spec.

ModelChoice

Each ModelChoice(model, description) pairs a model name or configured Model instance with the criteria the router should use for that option. Choice keys and descriptions must be non-empty, and the configured default must name one of the keys.

AzureDurableTaskDurability

Attach AzureDurableTaskDurability() to a named agent. Retrieve the bound copy with AzureDurableTaskDurability.from_agent(agent) and enter its register(worker) context before starting the native SDK worker; exit after the worker stops and drains. Register your own orchestrator and call yield from run_durable(ctx, lambda: agent.run(prompt)). Start, inspect and signal runs through the SDK client. Dependencies must be JSON-serializable and are reconstructed using deps_type.

Handle DeferredToolRequests in your orchestrator: wait for an external event and resume with full message history, cumulative usage and DeferredToolResults. Publish a fresh event name per pause in the custom status, and send decisions to the event the reviewer saw, keyed by tool-call ID. Orchestrator-side code must be deterministic and I/O-free. Tool effects must be idempotent. Set native retry policies with model_options, tool_options, event_options and capability_options. resolve_tool_options returns a native policy, None to keep tool_options, or False to run a function tool inline. Function tools can also opt out with metadata={'azure_durable_task': False}, or async ones with the engine-neutral durable=False. Inline tools must be deterministic and I/O-free. See Azure Durable Task.

StepPersistence

Records an append-only event log, full-history snapshots at settled tool-cycle boundaries, and a ledger of tool calls (started/completed/failed). It stores message history only: it does not restore capability state, graph-node state, or in-flight streams, and it does not replay anything by itself. You continue a run by passing a snapshot's messages to Agent.run(message_history=...).

It is not the recommended session store: to store conversations, use core's ConversationStore (pydantic_ai.conversation_store), which also keeps usage and deferred tool requests. To move sessions already stored with StepPersistence, save each SqliteConversationStore head, or else each session ID's newest complete snapshot, as a Conversation, skipping conversations the store already has. Settle a session whose newest run ends on unanswered tool calls (a pending approval or deferred result) first: its last complete snapshot leaves that turn out (see "Moving sessions to a ConversationStore" on the Step Persistence page).

import asyncio

from pydantic_ai import Agent

from pydantic_ai_harness import StepPersistence
from pydantic_ai_harness.step_persistence import InMemoryStepStore, continue_run

store = InMemoryStepStore()
agent = Agent(
    'test', capabilities=[StepPersistence(store=store, agent_name='librarian')]
)


async def main():
    await agent.run('Find the bug', conversation_id='conv-1')
    run_id = (await store.list_runs(conversation_id='conv-1'))[-1].run_id
    history = await continue_run(store, run_id=run_id)
    await agent.run('Now fix it', message_history=history, conversation_id='conv-1')
    runs = await store.list_runs(conversation_id='conv-1')
    print(len(runs), runs[-1].agent_name)
    #> 2 librarian


asyncio.run(main())

Key parameters: store (default InMemoryStepStore()), agent_name (prefix for the derived run id), run_id (explicit, single-use), parent_run_id (auto-filled when a tool of one persisted agent runs another in-process), metadata: dict[str, str], and capture_frontier=False (also checkpoint the prompt and the model's proposed tool calls before execution, so a first-request failure or a kill mid-tool-cycle keeps them).

Stores, all in pydantic_ai_harness.step_persistence, all async, all accepting max_snapshots_per_run=None (unbounded by default):

  • InMemoryStepStore(): process-local; tests.
  • FileStepStore(directory): <directory>/<run_id>/ with JSON/JSONL files.
  • SqliteStepStore(database='runs.db'), or connection= a sqlite3.Connection opened with check_same_thread=False. Stores each message once rather than repeating the history in every snapshot, and deletes stored messages along with the last snapshot that references them.
  • MongoStepStore(client=AsyncMongoClient | db_url=..., database=...): needs uv add "pydantic-ai-harness[mongodb]". Exactly one of client/db_url, and database is required. With db_url the store owns the client, so call await store.aclose(). It creates indexes on its first write, so the user needs index-creation privileges.
  • PostgresStepStore(pool, *, table='step_persistence'): pool is your own asyncpg pool (the driver-neutral PostgresPool protocol); no harness extra, and you own the pool lifecycle. table is a prefix of at most 40 characters. It creates its tables on first use unless they all exist, so only that first call needs CREATE on the schema.

File, SQLite, Mongo, and Postgres stores move BinaryContent and any text part of 64 KiB or more to a MediaStore (disk, same DB, same Mongo client, or same pool by default). Pass media_store=None to keep them inline, or media_store=S3MediaStore(...) from pydantic_ai_harness.media.

Reading back:

  • continue_run(store, run_id=...) / fork_run(...) return list[ModelMessage] from the latest complete snapshot, and raise LookupError if there is none. fork_run returns the same data; start it under a new conversation.
  • include_interrupted=True also considers snapshots taken mid-tool-cycle. Check await store.list_unresolved_tool_effects(run_id=...) first: a started effect with no terminal record may or may not have happened.
  • store.list_runs(parent_run_id=..., conversation_id=...) is sorted oldest first. list_events, latest_snapshot, and recovery.inspect_recovery(store=..., run_id=...) inspect a run.
  • In a tool that writes external state, call await annotate_tool_effect(store, ctx, idempotency_key=..., effect_summary=...) so an orchestrator can tell whether a replay is safe.

What exists after a crash: each finished tool cycle saves a complete snapshot that already includes the tool returns, so a crash after it resumes from there. If a tool raises and fails the run, on_run_error also saves an interrupted snapshot with the open tool calls. A process kill mid-cycle runs no hook, so only the last complete snapshot exists, plus an interrupted one if capture_frontier=True. The default read skips interrupted snapshots.

async def resume(run_id: str):
    unresolved = await store.list_unresolved_tool_effects(run_id=run_id)
    for effect in unresolved:  # status 'started': may or may not have happened
        print(effect.tool_name, effect.tool_call_id, effect.idempotency_key)
    # Take the interrupted point only when no call is in doubt: resuming from it
    # re-runs its open calls or closes them out with synthesized returns.
    history = await continue_run(store, run_id=run_id, include_interrupted=not unresolved)
    return await agent.run('Continue.', message_history=history)

Gotchas:

  • Shapes a test sees: when a streamed model request fails, the saved history (default read included) ends with an empty or partial-text ModelResponse. With Planning, each plan tool's ToolReturnPart ends with the plan in a <plan_reminder> tag and carries metadata={'origin': 'pydantic-ai-harness.planning.plan_reminder.v1', 'plan': ...}; a PlanReminderPart (from pydantic_ai_harness.planning) records the plan only when the latest plan isn't visible (after compaction, say). Assert on the parts you need, not exact message shapes.
  • run_id is per Agent.run. Reusing one instance with the same explicit run_id raises ValueError. Group turns with conversation_id= on Agent.run.
  • A derived run_id longer than 200 characters raises ValueError, so keep agent_name short. FileStepStore only accepts ids matching [A-Za-z0-9_.-]{1,200}.
  • Events are never pruned. Snapshots are pruned only with max_snapshots_per_run, and pruning never deletes media blobs. PostgresStepStore and MongoStepStore store each message once too, but keep stored messages when the snapshots referencing them are pruned.
  • Put StepPersistence before capabilities whose after_run rewrites history. After-hooks run in reverse order.
  • The default id is 'step_persistence', and store writes are durable operations, so it works alongside a durability capability. Pass id= only for a second instance.
  • Spec form: {'StepPersistence': {'backend': 'memory' | 'file' | 'sqlite', 'directory': ..., 'database': ..., 'max_snapshots_per_run': ...}}. Any other backend raises ValueError. Mongo is Python-only.

Harness capabilities under core durable execution

Most harness capabilities run under core TemporalDurability, DBOSDurability, and PrefectDurability (pydantic_ai.durable_exec.*), but support is per capability:

  • Agent(...) with TemporalDurability raises UserError ("Toolsets that are 'leaves' ... need to have a unique id") for AskUser, ExaSearch, ExaAgent, YouSearch, YouResearch, Researcher, PydanticAIDocs, LocalStack, Macroscope, and BrowserUse (either session_scope): their toolsets carry no stable id, and passing id= to the capability does not reach the toolset.
  • PlaywrightBrowser raises UserError at Agent(...) with any durability capability (a live browser cannot survive replay); a TrajectoryJudge without an id raises at run start inside a durable workflow.
  • CodeMode runs but skips its speculate= early tool launches. BackgroundTools, guardrails, SpendLimits, Memory, Planning, SubAgents, the sandboxes, and the hosted integrations build under Temporal.

Attach the workspace, the harness capabilities, and the durability capability at construction, and give the agent a name:

from pydantic_ai import Agent
from pydantic_ai.capabilities import LocalWorkspace
from pydantic_ai.durable_exec.temporal import TemporalDurability

from pydantic_ai_harness.coder import Coder

agent = Agent(
    'anthropic:claude-opus-5-5',
    name='coder',
    capabilities=[LocalWorkspace('.'), Coder(), TemporalDurability()],
)

Rules that apply to every harness capability:

  • Every leaf toolset needs a stable, unique id. The capabilities not listed above ship fixed or derived ids; two instances of one class need distinct id=.
  • Per-run inputs (tokens, user ids) come from deps, never globals: tools and auth functions can execute on another worker.
  • Inside a Temporal or DBOS workflow, agent.run(..., capabilities=...) raises UserError for any capability except a few core ones such as Instrumentation. A Prefect flow accepts per-run capabilities that contribute no function, MCP, or dynamic toolset. Temporal also refuses toolsets added at run time (for example SubAgents with agent_folders). Load authored capabilities at construction; they take effect on the next worker start.
  • Removing or adding a capability changes the replay shape. Drain in-flight workflows first.

The per-engine feature matrix for workspace tools (commands, background jobs, live events, delegation, timeouts) is in CODING-AND-WORKSPACES.md and the durable execution docs page.

RenderWorkflows

When the user is using or evaluating Render, consider RenderWorkflows for background agent jobs that need separate retries, timeouts, or compute for model requests and tool calls. Retrying the entry task starts the whole agent again and can repeat completed work; the integration does not checkpoint progress or replay completed steps. Choose it when the application can handle repeated work. A plain Render task around agent.run(...) may be enough when the user only needs background execution. The calling application chooses which requests to submit, even if it runs on AWS or another platform. A submission creates a root-level task run; the capability starts chained task runs for supported operations through TaskContext.run().

Use the Render Workflows guide for installation, a runnable example, local execution, and deployment. Check its capability support and execution limits before composing an agent. The setup and constraints that affect implementation are:

  • Construct one RenderWorkflows(app) per agent and attach it in Agent(capabilities=[...]). Define agents and tools at module load time, using the same Render Workflows app. Call agent.run(...) inside that capability's @workflows.task entry function; a plain @app.task leaves agent operations inline. The live app object requires Python construction rather than a JSON or YAML agent spec.
  • Give agents and registered leaf toolsets stable names and IDs, including capability-owned toolsets. Task options are fixed per named toolset at registration. Use separate named toolsets for different policies; individual function tools can opt out, but cannot override their shared task settings. Return False from resolve_tool_options for delegate_task, read_tool_result and grep_tool_result when using SubAgents and ToolOutputLimits, so those helpers retain the parent task's live state. For a delegated agent's operations to run as chained task runs, attach its own RenderWorkflows instance using the same app. A chained task run retry can repeat that operation's side effects, so make external writes safe to repeat.
  • Inputs and results crossing a task boundary must be JSON encodable, including deps. Workers need access to their own model credentials and external resources. Keep large artifacts in shared storage and pass references that another worker can read; process-local files and live clients are not shared. Model streaming and forwarded capability events are buffered at task boundaries. Tools that need an immediate capability-event decision must run inline.
  • Generated operations are Render task definitions. Callers with Render API access to submit task runs can invoke them directly, bypassing checks that exist only in the entry task. Keep Render credentials on the server and authorize application users before submitting validated inputs. Treat Workflow API callers as trusted, and account for Render's task-data retention when deciding what to send in task payloads.

AWSLambdaDurability

Checkpoints every model request, function tool call, MCP call, and dynamic-toolset resolution as an AWS Lambda durable step. A resumed invocation replays completed steps from the log.

uv add "pydantic-ai-harness[aws-lambda]" "pydantic-ai-slim[bedrock]"   # Python 3.11+
from typing import Any

from aws_durable_execution_sdk_python import DurableContext, durable_execution
from pydantic_ai import Agent

from pydantic_ai_harness.aws_lambda import AWSLambdaDurability, durable_agent_handler

agent = Agent(
    'bedrock:us.amazon.nova-pro-v1:0',
    name='support',
    capabilities=[AWSLambdaDurability()],
)


@durable_execution
@durable_agent_handler
async def handler(event: dict[str, Any], context: DurableContext) -> str:
    result = await agent.run(str(event['prompt']))
    return result.output

Parameters: AWSLambdaDurability(*, models=None, event_stream_handler=None, name=None, step_config=None). models maps ids to extra models for agent.run(model='<id>'). step_config sets the base retry_strategy/step_semantics/serdes for all steps. Per-tool metadata={'aws_lambda': {...}} overrides it key by key, and metadata={'aws_lambda': False} opts a function tool out (MCP tools cannot opt out). The engine-neutral durable=False opts an async function tool out on every engine; an engine's own key on the same tool wins.

Approvals: add core HandleDeferredToolCalls(ask, timeout=timedelta(...)) (pydantic_ai.capabilities), whose ask takes a resume token, next to AWSLambdaDurability() instead of ending the run with DeferredToolRequests output and starting a second run. A call that needs approval, an answer or external execution creates a Lambda callback and runs ask(ctx, requests, callback_id) once in a step ({name}__external_event.deferred_tool_calls.ask). If ask returns DeferredToolResults, the step records it and the run continues (the callback is left open, unused); if it returns None, the execution suspends. AskUser questions use the same mechanism ({name}__external_event.ask_user_question). The approver sends the JSON form of DeferredToolResults ({"approvals": {"<tool_call_id>": true}}, or {"kind": "tool-denied", "message": ...} / {"kind": "tool-approved", "override_args": {...}}) with aws lambda send-durable-execution-callback-success --callback-id ... --cli-binary-format raw-in-base64-out --result '<json>' (boto3 send_durable_execution_callback_success). On replay the model is not called again and ask does not rerun. A timeout or send-durable-execution-callback-failure denies every pending approval and fails every pending external call with a message, and the run continues; an invalid decision or a stale tool call ID raises UserError. Other deferred-call handlers (a token-less HandleDeferredToolCalls, ApprovalReviewer, AskUser) run first. An ask that returns None only delivers the question; the answer comes from your approval endpoint. Outside a durable handler (local development) the same agent waits in process and pydantic_ai.external_events.resolve_external_event(token, decision) resolves it; from inside a checkpointed tool, the wait raises UserError.

Gotchas:

  • A run is durable only inside durable_agent_handler or run_durable(lambda: agent.run(...), context=context). agent.run_sync(...) or your own asyncio.run runs normally but checkpoints nothing, and raises no error to say so.
  • @durable_execution must be the outermost decorator. The reverse order raises UserError. run_durable cannot be called from inside a running event loop.
  • The agent needs a name (or name=), and every toolset needs a unique id. Both are checked at Agent(...).
  • Steps are at-least-once, and the SDK retries six times by default. For a tool that must not repeat, set both StepSemantics.AT_MOST_ONCE_PER_RETRY and retry_strategy=RetryPresets.none(). Step retries stack with Pydantic AI and provider-client retries, so disable one side.
  • Tool calls run sequentially inside a durable handler. Nested durable runs are rejected.
  • Await agent runs one after another in a handler, never concurrently with asyncio.gather or tasks: replay matches checkpoints by position, so concurrent runs can be served each other's results.
  • Changing the number or order of steps (tools, MCP servers, aws_lambda: False opt-outs, event_stream_handler) breaks in-flight executions, and so does renaming the agent, a toolset id, a tool, or the id a non-default model runs under (a models= key or the model= passed to agent.run), which is part of that model's step names. SDK 2.0+ fails those executions with NonDeterministicExecutionError; SDK 1.x does not detect a mismatch. Deploy a new published version.
  • Checkpoints hold model responses, tool results, and step error messages for the retention period.
  • ctx.enqueue() is unavailable inside a durable step. Do not detach work with asyncio.create_task().
  • Budget: 30,000 operations and 100 MB of checkpointed state. Return references, not blobs.

AbsurdDurability

Checkpoints every model request, function tool call, and MCP call as a step of an Absurd task, stored in Postgres. When a worker dies, the task re-runs from the top and completed steps return their stored results.

uv add "pydantic-ai-harness[absurd]" "pydantic-ai-slim[openai]"
from absurd_sdk import AsyncAbsurd, AsyncTaskContext, JsonValue
from pydantic_ai import Agent

from pydantic_ai_harness.absurd import AbsurdDurability

absurd = AsyncAbsurd('postgresql://localhost/absurd', queue_name='agents')
agent = Agent('openai:gpt-6.1-sol', name='analyst', capabilities=[AbsurdDurability()])


@absurd.register_task(name='analyse')
async def analyse(params: JsonValue, ctx: AsyncTaskContext) -> JsonValue:
    assert isinstance(params, dict)
    result = await agent.run(params['prompt'])
    return {'output': result.output}

Parameters: AbsurdDurability(*, models=None, event_stream_handler=None, name=None, parallel_execution_mode='sequential'). models maps ids to extra models for agent.run(model='<id>'). parallel_execution_mode also accepts 'parallel_ordered_events', but not 'parallel'.

Gotchas:

  • A run is durable only inside an async Absurd task handler; a synchronous TaskContext raises UserError.
  • The agent needs a name (or name=), and every function, MCP, or dynamic toolset needs a unique id. Both are part of every step name, so renaming either makes in-flight tasks re-run those steps.
  • Tool return values are stored as JSON. ModelRetry, ToolFailed, CallDeferred, and ApprovalRequired are not checkpointed, and DynamicToolsets are not wrapped, so that work re-runs.
  • Function, MCP, or dynamic toolsets passed with run(toolsets=...) inside a task raise UserError; ExternalToolset is fine.
  • Keep tool side effects idempotent: a crash after a tool runs but before its step is saved re-runs it.

PGTaskDurability

Checkpoints every model request, tool call, and MCP call as a step of a pgtask task, stored in Postgres. When a worker dies, the task re-runs from the top and completed steps return their stored results.

uv add "pydantic-ai-harness[pgtask]" "pydantic-ai-slim[openai]"
from pgtask import Task, TaskRegistry
from pydantic_ai import Agent

from pydantic_ai_harness.pgtask import PGTaskDurability

tasks = TaskRegistry(queue_name='agents')
agent = Agent('openai:gpt-6.1-sol', name='analyst', capabilities=[PGTaskDurability()])


@tasks.task('analyse')
async def analyse(task: Task, payload: dict[str, str]) -> dict[str, str]:
    result = await agent.run(payload['prompt'])
    return {'output': result.output}

Parameters: PGTaskDurability(*, models=None, event_stream_handler=None, name=None, parallel_execution_mode='sequential'), the same as AbsurdDurability.

Gotchas:

  • Run await Client.connect(url) then await client.migrate() once per database before the first task.
  • The agent needs a name (or name=), and every function, MCP, or dynamic toolset needs a unique id. Both are part of every step name, so renaming either makes in-flight tasks re-run those steps.
  • Tool results are stored as JSON, ModelRetry and ToolFailed included; DynamicToolset calls are checkpointed too. A checkpoint that no longer decodes raises CheckpointDecodeError instead of re-running the tool.
  • Function, MCP, or dynamic toolsets passed with run(toolsets=...) inside a task raise UserError; ExternalToolset is fine.
  • Keep tool side effects idempotent: a crash after a tool runs but before its step is saved re-runs it.

ManagedPrompt

Resolves a Logfire-managed prompt once per run and uses it as the agent's instructions, with the label and version attached as baggage to every span of the run.

uv add "pydantic-ai-harness[logfire]"
from pydantic_ai import Agent
from pydantic_ai.messages import ModelRequest
from pydantic_ai.models.test import TestModel

from pydantic_ai_harness import ManagedPrompt

agent = Agent(
    TestModel(),
    capabilities=[
        ManagedPrompt('support_agent', default='You are support.', label='production')
    ],
)
request = agent.run_sync('hi').all_messages()[0]
assert isinstance(request, ModelRequest)
print(request.instructions)
#> You are support.

Parameters: name (positional; declared as the variable prompt__<name>, with hyphens turned into underscores; ASCII letters, digits, hyphens, and underscores only), or a prebuilt logfire.variables.Variable (not a TemplateVariable); default (required with a name); label; targeting_key and attributes (static or ctx -> value); render_template=False (Handlebars against deps); logfire_instance. Call logfire.configure() in your app to get remote values. Until then, and whenever no remote value is published, default is used.

Gotchas: pin label='production' to keep the provider prompt cache warm, because rollouts across labels split it; a pinned label that doesn't exist falls back to the rollout. A label change mid-run applies from the next run; in a conversation continued from message_history, a new version is appended as a change instead of rewriting the cached prefix. The id is derived from the name (managed-prompt-support_agent), so two of one name on an agent raise UserError. With render_template=True, anyone who can edit the prompt in Logfire can place any deps field in the system prompt, and a remote prompt that isn't a valid template falls back to default with a RuntimeWarning and a Logfire warning log. Logfire's Resolve variable span records the prompt text even with include_content=False. The capability orders itself outermost, wrapping Instrumentation. ManagedPrompt.resolved exposes the current run's ResolvedVariable, and is None outside a run.

CapabilityCreation

Gives the model author_capability(name, code), list_authored_capabilities(), and disable_authored_capability(name). Authored code is written to <directory>/<name>.py, imported, and validated (exactly one AbstractCapability subclass that constructs with no arguments). It becomes active on the next run, and only if you pass it in:

from pathlib import Path

from pydantic_ai import Agent
from pydantic_ai.workspaces import LocalWorkspaceBackend

from pydantic_ai_harness import CapabilityCreation

workspace = LocalWorkspaceBackend('.')
creation = CapabilityCreation(directory=Path('.authored'))
agent = Agent('anthropic:claude-opus-5-5', capabilities=[creation])



async def main():
    history = None
    for prompt in ['Add a capability that logs tool calls.', 'Now refactor utils.py.']:
        extra = await creation.load_active(workspace)
        result = await agent.run(prompt, message_history=history, capabilities=extra, workspace=workspace)
        history = result.all_messages()

Gotchas: a run raises UserError unless its workspace is a writable LocalWorkspace, because authored code executes in your process and would bypass a sandbox. A run also raises UserError unless ctx.workspace.authority_of(directory) is 'operator', so point directory at code the operator controls, not a clone or an upload; a directory outside the working directory needs path_authority={path: 'operator'}. With defer_loading=True these checks refuse each authoring tool call instead. creation.load_active(workspace) makes them against workspace before loading, and only against it, so pass the same workspace as agent.run(workspace=...) instead of a LocalWorkspace capability (for a directory outside the working directory, AuthorityWorkspace(Workspace(LocalWorkspaceBackend('.')), authority='operator', path_authority={'/abs/path': 'operator'})); creation.store.load_active() has no workspace and skips them, so only use it where you would run the model's code yourself. Authored code runs during author_capability validation, in the current run. load_active() runs the same checks and skips entries that fail them. Inside a Temporal or DBOS workflow, capabilities= on a run is refused, so authored capabilities cannot be activated there; a Prefect flow accepts only authored capabilities that contribute no function, MCP, or dynamic toolset. The authoring tools still work. A failed re-author keeps the current version when its entry has no recorded error and its file still exists, and saves the rejected code as <name>.rejected.py. To have a person approve authored code before it runs, add ToolGuardrail(guard=lambda call: GuardrailResult.approve(), tools=['author_capability']) with output_type=[str, DeferredToolRequests]. guidance='' drops the built-in system-prompt guidance. CapabilityCreation is not spec-serializable.

Harness capabilities in agent specs

Core resolves spec capability names from a closed built-in registry. A harness class loads only when you pass it in custom_capability_types. There is no auto-registration, and no harness-wide list to pass. The spec name is the class name unless the class overrides get_serialization_name(). A name not in the list raises ValueError naming the valid choices.

from pydantic_ai import Agent

from pydantic_ai_harness import Planning, StepPersistence

spec = {
    'model': 'test',
    'capabilities': [
        'Planning',
        {'StepPersistence': {'backend': 'memory', 'agent_name': 'triage'}},
    ],
}
agent = Agent.from_spec(spec, custom_capability_types=[Planning, StepPersistence])

Agent.from_file('agent.yaml', custom_capability_types=[...]) works the same way (install pydantic-ai-slim[spec] for YAML). Capability entries take the forms Name, {Name: positional_arg}, or {Name: {kwargs}}, and each class's from_spec builds the instance. Opted out (they hold callables or live agents, and raise ValueError if listed): AWSLambdaDurability, AbsurdDurability, PGTaskDurability, CapabilityCreation, DynamicWorkflow, InputGuardrail, OutputGuardrail, SubAgents, SystemReminders, ToolGuardrail, TrajectoryJudge. AskUser has a spec name but needs an answerer callable, so in practice it is passed in code too. CompactOnRequest, DeduplicateFileReads, FallbackCompaction and TieredCompaction have spec names too, but each requires strategies or a file_key callable that JSON cannot express, so the spec schema leaves them out and they are Python-only. Pass those instances with the capabilities= keyword of Agent.from_spec/Agent.from_file, which adds them to the spec's list. Keep secrets out of spec files and let capabilities read their env vars.

ToolCallJudge and ApprovalReviewer are spec-loadable when the model is a string and tools is 'all', a name list, or a metadata match. A live model, predicate selector, and the on_verdict or on_review callback are code-only.

A spec whose model is a provider string ('openai:gpt-6.1-sol') needs that provider's API key when the agent is built. In tests, pass defer_model_check=True and run under agent.override(model='test').

ACP server (experimental)

Serves any Agent over the Agent Client Protocol on stdio, so Zed and other ACP clients can drive it with streamed text, diffs, tool approval, and sessions. It lives under pydantic_ai_harness.experimental and may change or be removed in any release, without deprecation. Importing it emits HarnessExperimentalWarning.

uv add "pydantic-ai-harness[acp,anthropic]"
from pydantic_ai import Agent
from pydantic_ai.workspaces import LocalWorkspaceBackend

from pydantic_ai_harness.experimental.acp import (
    ACPSession,
    ACPSessionConfig,
    InMemorySessionStore,
    run_acp_stdio_sync,
)
from pydantic_ai_harness.file_system import FileSystem
from pydantic_ai_harness.shell import Shell

agent = Agent('anthropic:claude-opus-5-5')


def session_config(session: ACPSession) -> ACPSessionConfig[None]:
    return ACPSessionConfig(
        deps=None,
        capabilities=[FileSystem[None](), Shell[None]()],
        workspace=LocalWorkspaceBackend(session.cwd),
    )


if __name__ == '__main__':
    run_acp_stdio_sync(
        agent, session_config=session_config, session_store=InMemorySessionStore()
    )

ACPSessionConfig requires only deps (None for no deps); capabilities, toolsets, and workspace default to None, so an agent without file or shell tools needs no workspace.

Gotchas: root tools at session.cwd through session_config, not a static FileSystem. Tools marked requires_approval=True become client approval prompts. acp_filesystem(session) and acp_terminal(session) return editor-backed tools, or None when the client does not offer them, so use acp_filesystem(session) or FileSystem[None](). If a client sends MCP servers and there is no session_config to connect them, the session is rejected. Prompt content is text-only unless you pass prompt_capabilities. models=[...] exposes a model picker. Use async run_acp_stdio inside a running loop. Silence the warning with warnings.filterwarnings('ignore', category=HarnessExperimentalWarning) (importing pydantic_ai_harness.experimental itself does not warn).

GitHub Agentic Workflows

The gh-aw pydantic-ai engine runs a Pydantic AI agent in GitHub Actions on issues, PRs, or a schedule. In the workflow .md, import pydantic/pydantic-ai/src/pydantic_ai_harness/gh-aw/pydantic.md@main, set engine: {id: pydantic-ai, model: openai/gpt-6.1-sol} (provider/model is required), and point engine.env.PAI_AGENT at module:variable (for example my_agent:agent), at pydantic_ai_harness.researcher:researcher_agent, or at a .yml/.json spec. Omit PAI_AGENT to run Coder. Use gh-aw v1.0.1 or newer with AWF v0.29.0 or newer. Run gh aw compile and commit the regenerated .lock.yml, including for existing workflows. Upgrading the CLI alone leaves existing locks unchanged. Omit evals, which protected capture rejects. Treat PAI_AGENT modules and caller-supplied PYTHONPATH as trusted engine configuration. Point both settings only at code you trust.

For authenticated terminal metadata, set engine.env.PAI_RESULT_METADATA to a trusted MODULE:ATTRIBUTE dictionary. Keep its keys as strings and its values JSON-serializable; update the same dictionary during the run. Read the signed value from verified agent-session.jsonl under session.result.data.metadata. OTLP endpoints and credentials stay on the host, so the launcher's direct OTLP export does not run under gh-aw. See the engine README's observability section.

Gotchas: leave the model off the Agent, because the engine's -m overrides it. Writes go through safe outputs (for example the safeoutputs_add_comment tool). A step that appends {"type":"noop","message":"..."} to $GH_AW_SAFE_OUTPUTS makes the engine exit 0 before starting the agent; gh-aw rejects a noop without message. Install extra dependencies with a workflow steps: pip install --user. The provider key is a repository secret, never engine.env. Threat detection on safe outputs runs on copilot and reads COPILOT_GITHUB_TOKEN (or github.token under copilot-requests: write); without either, set safe-outputs.threat-detection.engine to codex, claude, or false. A spec agent cannot use harness capabilities, because the CLI passes no custom_capability_types. Use a Python module for those.

See also

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at b2d64a3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 days ago
compatibility
Requires Python 3.11+
metadata
{
  "version": "0.2.0",
  "author": "pydantic"
}

README badge

README badge for pydantic/pydantic-ai/pydantic-ai-harness