Index of every page on this site: llms.txt
All skills
pydantic avatar

/pydantic-ai-harness

@b2d64a3
by pydanticpydantic/pydantic-ai21k stars
3,221
Run it yourself

Extend Pydantic AI agents with capabilities from pydantic-ai-harness -- the Coder coding agent, file and shell tools in local or sandboxed workspaces (Modal, E2B, Render, Sprites, Daytona, boxd, smolvm, BoxLite), Code Mode, sub-agents and planning, model routing, memory and skills, context compaction, guardrails and spend limits, web research and browsers, hosted SaaS integrations, step persistence, and background agents on Render Workflows. Use when the user mentions pydantic-ai-harness or pydantic_ai_harness, imports a harness capability such as Coder, CodeMode, FileSystem, Shell, SubAgents, ModelRouter, Memory, or ToolGuardrail, or wants a Pydantic AI agent that edits files, runs commands or agent-written Python, delegates, routes work between models, remembers, manages long context, stays within limits, or runs model and tool operations as chained Render task runs.

  • 11 files
  • 334 KB
  • MIT
  • Updated 2 days ago
  • GitHub
Use this Skill: https://skilld.dev/gh/pydantic/pydantic-ai/pydantic-ai-harness

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesDELEGATION-AND-PLANNING.md

≈7.5k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Delegation and Planning

Capabilities that structure long or multi-part work: Planning (a model-owned task list), SubAgents (one delegate_task tool over named child agents), Peers (messages between independently run sessions), DynamicWorkflow (the model scripts sub-agent calls in a sandbox), Advisor (consult a second model mid-run), and BackgroundTools (slow tools run while the agent keeps going). All import from pydantic_ai_harness; only DynamicWorkflow needs an extra.

Which one to use

Need Use
Parent calls one specific agent from a tool it writes itself, full control over prompt and deps Core agent delegation (await other.run(..., usage=ctx.usage) inside a tool; see the building-pydantic-ai-agents skill, ORCHESTRATION-AND-INTEGRATIONS.md)
Model picks among several named specialists, occasionally, judging each result before the next SubAgents
Independently run sessions discover or message each other without controlling each other's runs Peers
Fan-out, chaining, voting, or retry loops over sub-agents, where intermediate results should not enter the parent context DynamicWorkflow
A second opinion from another (often stronger) model, not a task hand-off Advisor
Keep a checklist across a long run, or hand a plan from a planner run to an executor run Planning
A slow tool should not block the agent BackgroundTools

Start with SubAgents when unsure; its sub-agents move to a DynamicWorkflow catalog unchanged.

Planning

Gives the model plan tools and keeps the current plan in front of it: every tool that changes the plan ends its result with the whole plan in a <plan_reminder> tag, and a fixed note in the static instructions says what the tag holds. Only when the latest plan isn't visible (after compaction, in a run started without it, or after an outside store change) does Planning record it as user-channel data, a PlanReminderPart (from pydantic_ai_harness.planning; find records by type). The plan never goes in system voice, and the history only grows, so plan edits don't break the prompt cache or preserved thinking.

from pydantic_ai import Agent

from pydantic_ai_harness import Planning

agent = Agent('test', capabilities=[Planning()])

Tools: write_plan(items) (whole-list replace), read_plan(), add_task(content, active_form), update_task_status(task_id, status), update_task_statuses(updates) (all-or-nothing), remove_task(task_id). enable_subtasks=True adds add_subtask, set_dependency, get_available_tasks and the blocked status. Statuses: pending, in_progress, completed, cancelled (+ blocked).

One write_plan item (the PlanItem model from pydantic_ai_harness.planning); parent_id and depends_on (a list of ids) need enable_subtasks=True:

from pydantic_ai_harness.planning import PlanItem

item = {
    'id': 'mig',  # optional; generated when omitted
    'content': 'Add the database migration',  # the only required field
    'status': 'in_progress',  # default 'pending'
    'active_form': 'Adding the database migration',  # shown while in progress
}
print(PlanItem.model_validate(item).status.value)
#> in_progress

Key parameters:

  • guidance=None: None = built-in guidance, '' = none, a string replaces it.
  • store=None: None = fresh in-memory plan per run. Pass a PlanStore to persist.
  • store_resolver=None: Callable[[RunContext], PlanStore], wins over store (per-tenant stores).
  • enable_subtasks=False, inject=True (show the model the plan). cache_ttl is deprecated and ignored (an explicit '5m'/'1h' warns); turn on caching on the model (such as anthropic_cache_messages).
  • tools=None: allowlist, e.g. tools=['write_plan']. Naming a tool the mode does not register raises ValueError; so does an unknown key in descriptions.

Persistence and planner/executor hand-off through a shared store:

from pydantic_ai import Agent

from pydantic_ai_harness import Planning
from pydantic_ai_harness.planning import SqlitePlanStore

store = SqlitePlanStore('plan.db', session='issue-403')
planner = Agent('test', capabilities=[Planning(store=store)])
executor = Agent('test', capabilities=[Planning(store=store)])
# await planner.run('Write a plan. Do not implement.'); await executor.run('Implement the plan.')

Stores: InMemoryPlanStore(), SqlitePlanStore(database='.agent-plan.db', *, session='default', table='plan_items'), PostgresPlanStore(pool, *, session=..., table=...) over your asyncpg pool, RedisPlanStore(client, *, session=..., key_prefix='plan', expire_seconds=None) over your redis.asyncio client. The harness installs no driver.

Gotchas:

  • SqlitePlanStore(':memory:') raises ValueError; use InMemoryPlanStore. The SQLite file lives on the agent's host, not in the workspace.
  • The reminder reads the store on every model request; a store that raises fails the run. Wrap the store yourself for fallback behaviour.
  • parent_id, depends_on, and blocked are rejected by write_plan unless enable_subtasks=True.
  • Events: PlanCreatedEvent, PlanUpdatedEvent, PlanStatusChangedEvent, PlanCompletedEvent, PlanDeletedEvent from pydantic_ai_harness.planning, via @agent.on_event(...). Only mutations made through the tools emit them; direct PlanStore calls do not. Store event_emitter= is deprecated.
  • The planner's "do not implement" discipline comes from its instructions and toolsets, not from Planning.

SubAgents

One delegate_task(agent_name, task) tool. Each call runs the named agent in a fresh run (no parent history) and returns str(result.output). The roster is listed in the system prompt.

from pydantic_ai import Agent
from pydantic_ai.usage import UsageLimits

from pydantic_ai_harness import SubAgent, SubAgents

researcher = Agent('test', name='researcher', description='Researches a topic')
writer = Agent('test', name='writer', description='Turns notes into prose')

orchestrator = Agent(
    'test',
    capabilities=[
        SubAgents(
            agents=[
                SubAgent(researcher, usage_limits=UsageLimits(request_limit=20)),
                SubAgent(writer, timeout_seconds=300, max_calls=2),
            ],
        )
    ],
)

SubAgent(agent, name=None, description=None, models=None, usage_limits=None, timeout_seconds=None, max_calls=None, on_failure=None, contain_errors=None, on_model_error=None) sets per-delegate controls.

SubAgents parameters that change behaviour:

  • agent_folders=None (default): no disk loading. Pass 'agents' to load every Claude-style *.md and Codex-style *.toml under .agents/agents/, .claude/agents/, and .codex/agents/ (in that order) from the run's workspace. TOML needs name, description, and developer_instructions; unknown keys, including sandbox or permission settings, skip the file with a warning. A run without a workspace skips it. A sequence of paths reads exactly those (and fails at run start without a workspace). Pass workspace=LocalWorkspaceBackend('/app') to read definitions from elsewhere.
  • forward_usage=True: children share the parent's usage, so a parent usage_limits bounds the tree. A SubAgent.usage_limits gives that child isolated accounting (its tokens stop aggregating).
  • inherit_tools=False: True is deprecated. Bind tools directly to explicit child agents, use include_self=True for a fresh run of the agent with all capabilities and tools bound to it, or use tool_resolver to give disk agents tools.
  • shared_capabilities=(): capabilities applied to every child run.
  • models={}: menu of key -> model | ModelOption(model, description=, settings=). When set, delegate_task gains a model enum argument; SubAgent(models=['fast']) restricts a delegate.
  • workspace_resolver=None: an async (ctx, request: DelegationWorkspaceRequest) -> Workspace callback enables delegate_task(working_directory=...). Import DelegationWorkspaceRequest from pydantic_ai_harness.sub_agents. Validate permitted delegates and paths in the callback. Keep returned workspaces alive until background children finish. Parent/delegate read-only restrictions still apply; supply trusted read-only tools too. Managed tasks save the resolved directory and reject directory changes on resume. Remote and durable delegations reject this argument.
  • include_self=False: True lists the running agent as delegate self (fresh run with all bound capabilities). max_depth=3 caps nesting, counting the top-level run.
  • tool_retries=2, contain_errors=False, event_stream_handler=None, tool_name='delegate_task'.
  • Disk agents: agent_overrides={'name': AgentOverride(model=..., effort='high')} and tool_resolver (maps frontmatter tools names to toolsets); from pydantic_ai_harness.sub_agents.

Gotchas:

  • Every delegate needs a name (the agent's name or SubAgent(name=...)), else ValueError at construction. Duplicate names raise; with include_self=True, so does a delegate named self (a disk agent named self is skipped with a warning).
  • Children receive the parent's deps (same AgentDepsT) and inherit its workspace unless a directory is selected.
  • In tests, override a child's model by nesting overrides: with parent.override(model=m1), child.override(model=m2): (SubAgent runs the same Agent).
  • Timeout, own-budget exhaustion, max_calls exhaustion, and a child run that ends with DeferredToolRequests or PendingModelResponse (the parent cannot resume it) are soft: a steering message (or on_failure) comes back as the tool result, with outcome failed. A model request the child left pending (e.g. in a BatchModelRequests batch) is cancelled first, best effort, on a remote SubAgentWorker too; a batch is only cancelled once no other request in it still waits. Child ModelRetry/UnexpectedModelBehavior becomes a parent ModelRetry; on_model_error='return' returns the error plus the steering message as a tool result instead, and 'retry' keeps the ModelRetry even with on_failure set. If it's unset, setting on_failure returns that message alone, without the error. Other crashes abort the parent unless contain_errors=True.
  • SubAgent(keep_partial_output=True) appends the child's last text response, framed as <partial_output>, to the steering message when its timeout or own usage budget stops it. The text has not passed output validation and can be text the child rejected; leave it off for a child whose guardrails keep content from its parent.
  • A locally run child inheriting count_tokens_before_request=True from the run's limits that raises NotImplementedError before a request is counted in its usage (such as a model with no count_tokens) is started once more without it (under a new run ID when recorded as a task). A SubAgent.usage_limits that sets the flag itself is kept and fails like a top-level run, and a remote worker applies the announced limits as received.
  • include_self=True passed in agent.run(capabilities=...) raises UserError; bind it on the Agent.
  • Not agent-spec serializable (holds live agents). Events: DelegationStartedEvent (not DelegationStartEvent) and DelegationFinishedEvent (not DelegationEndEvent), whose outcome is ok/timeout/budget/failed/contained.

Task control

SubAgents(task_control=TaskControl()) turns delegations into tasks the model manages. Off by default.

from pydantic_ai import Agent

from pydantic_ai_harness.sub_agents import SubAgent, SubAgents, TaskControl

researcher = Agent('test', name='researcher', description='Researches a topic.')
orchestrator = Agent(
    'test',
    capabilities=[SubAgents(agents=[SubAgent(researcher)], task_control=TaskControl(max_running_tasks=4))],
)
  • delegate_task takes background=True and resume=<task_id>; the model also gets list_tasks, check_task (a running task's recent activity, from DelegationTasks.snapshot(task_id)), message_task (which resumes a finished task), answer_task, and cancel_task(force=False). Sub-agents get ask_parent (SubAgent(can_ask_questions=False) removes it; SubAgent(resumable=False) makes one one-shot, as DelegationTasks(one_shot=...) does by name for an owner the application binds).
  • Without a bound DelegationTasks, each run opens its own owner: background reports arrive as <task_output> data and questions as <task_question> data, the top-level run waits for its background tasks before it ends, and tasks still running when it ends are cancelled. Bind a DelegationTasks to keep tasks across runs; its observer then gets every task event as DelegationTaskEvent(event=...).
  • TaskControl fields auto_background_after=None, max_running_tasks=None, forward_events=False are opt-in.
  • Every step is a SubAgentMessageEvent capability event (TaskAssignedEvent, TaskQuestionAskedEvent, TaskCompletedEvent, ...) plus TaskBackgroundedEvent; @on_event listeners can rewrite or veto() the ones sent to a task, and answer a TaskQuestionAskedEvent by setting answer.
  • Remote agents: TaskControl(remote_agents=[RemoteSubAgent(name=...)], transport=...) (SubAgentTransport; InMemoryTransport in process), served by SubAgentWorker(agents={name: agent}, transport=..., deps=...) in the other process. accept_timeout=30.0 fails a task no worker picks up.
  • dynamic_agents=DynamicAgents(models=['fast']) adds create_agent, with a vetoable AgentCreatedEvent; works with or without task control.
  • One SubAgents per agent may set task_control (or dynamic_agents). Don't use either with the deprecated wrapper agents (TemporalAgent, ...). On Temporal and DBOS, dynamic_agents and task control run each delegation other than to self in a durable unit of its own; a durable task can't ask questions, take messages or be resumed, cancel_task stops it right away, and its report arrives when the run would otherwise end. UserError under durability: task control with include_self, remote_agents or forward_events, auto_background_after or max_running_tasks outside Temporal, other durable engines, and a bound DelegationTasks owner.
  • Non-self delegations run in an activity on Temporal and inline in the workflow on DBOS. To keep a delegate's own durability, add SetToolMetadata(tools=['delegate_task'], temporal={'child_workflow': {}}) (or dbos=...): every delegate must be durable on that engine, and on Temporal registered with the worker (AgentPlugin). Combined with include_self, inherit_tools, models, shared_capabilities, event_stream_handler, dynamic_agents, task_control, or max_calls, a delegation raises UserError. The delegate's usage isn't reliably counted in the parent's usage or bounded by its limits. Depth isn't carried across, so nested opt-ins reset max_depth per child workflow.

DelegationReports

Delivers finished background-task reports from a DelegationTasks owner into a parent run. Each report is a TaskOutputPart (a DataCapabilityPart from pydantic_ai_harness.sub_agents): the child's output as <task_output task_id=... agent_name=...> data, with outcome=... only when the task didn't succeed, and a note naming the sub-agent as its author. A question is a TaskQuestionPart (<task_question>). The output goes out as written; text in it that looks like the frame's tag or an application frame gets a further note instead. Find both by type, or with is_task_report(part), which also matches the metadata-marked user text an earlier version delivered, so hosts can keep them out of user-prompt listings. The Vercel AI and AG-UI message conversions leave capability parts out.

from pydantic_ai_harness.sub_agents import DelegationReports, DelegationTasks

tasks = DelegationTasks()
reports = DelegationReports(tasks, conversation_id='conversation-1')
  • priority='when_idle' (default) lets an active parent finish its current work first.
  • For a host-started, report-only continuation, use priority='asap' with agent.run(None, ...), so the reports reach the first model request with no synthetic user prompt.
  • Reports are marked delivered once consumed and are not replayed. The host owns idle wake-up.
  • With DelegationTasks(directory=...), a damaged saved task file is skipped and listed in owner.problems instead of stopping opened(). await DelegationTasks.discover(directory) reads the saved task tree (tasks by ID, plus problems) without opening an owner or resuming anything.

Peers

Use Peers for independently run sessions, not delegated child runs. InMemoryPeerNetwork connects sessions within one application: Peers(network) joins it when the agent is built, under the agent's name (or handle=), receiving in a new LocalPeerInbox (or inbox=). An agent with no name raises UserError, and handles are unique within a network.

from pydantic_ai import Agent

from pydantic_ai_harness import Peers
from pydantic_ai_harness.peers import InMemoryPeerNetwork

network = InMemoryPeerNetwork()
reviewer = Agent('test', name='reviewer', capabilities=[Peers(network)])
author = Agent('test', name='author', capabilities=[Peers(network)])

A host network instead gives each peer a PeerConnection for discovery and delivery, plus an optional PeerInbox its transport delivers to: Peers(connection, inbox=inbox). Call InMemoryPeerNetwork.join(handle=..., inbox=..., workspace=..., title=...) yourself to set the workspace or title list_peers shows; pass the returned connection and the same inbox to Peers.

  • Tools: list_peers(query='') lists other sessions, filtering handle, title, workspace, or branch; send_message(to, text, reply_to=None) addresses a listed handle or ID.
  • Peers defaults: max_messages_per_run=10, max_message_bytes=16384 UTF-8 bytes, max_listed=30. LocalPeerInbox defaults: max_pending=50, max_message_bytes=16384. Set both byte limits when changing allowed message size.
  • A peer with no active run holds the message; InMemoryPeerNetwork never starts its run, and a host network's own policy decides whether a message wakes it. A running peer reads the message at its next model request. Sending never waits for a reply.
  • PeerConnection.send takes a key (Peers passes run_id:run_step:tool_call_id); a retried send with the same key must not deliver twice.
  • For host storage, implement PeerInbox: take(run_id) before and after each model request, acknowledge(message_id) once in history, release(run_id) at run end.
  • Use LocalPeerInbox(path=...) for pending messages across restarts. The host owns transport, identity, authentication, and any cross-process coordination; the in-memory network does not supply those. A connection without Peers(connection, inbox=...) does not read an inbox.
  • Peer text is untrusted data, not user approval: it arrives as a PeerMessagePart with no authority. Find it by that type, or use read_peer_message(part) to read the typed message, not by parsing displayed text. A host delivering peer messages itself sends each as its own part. If peers can reach the agent, require approval for tools with side effects: a peer message can steer the run into any unapproved tool.
  • Joining InMemoryPeerNetwork without an inbox is send-only; incoming messages are rejected. LocalPeerInbox(accepting=False) refuses new messages and retains already-pending messages.
  • Do not combine Peers with durable-execution capabilities. Construction raises UserError because the engine records neither inbox reads nor sends, so a replay could diverge.

DynamicWorkflow

The model gets one run_workflow tool and writes Python (Monty sandbox) in which each sub-agent is an async function called as await name(task='...'). Only the script's last expression returns to the parent.

uv add "pydantic-ai-harness[dynamic-workflow]"
from pydantic_ai import Agent

from pydantic_ai_harness import DynamicWorkflow

reviewer = Agent('test', name='reviewer', description='Reviews code for bugs.')
summarizer = Agent('test', name='summarizer', description='Summarizes findings.')

orchestrator = Agent(
    'test',
    capabilities=[DynamicWorkflow(agents=[reviewer, summarizer], max_agent_calls=20)],
)

What the model writes (illustrative):

import asyncio

reports = await asyncio.gather(reviewer(task='Review auth.py ...'), reviewer(task='Review db.py ...'))
await summarizer(task='Summarize:\n' + '\n\n'.join(reports))

All parameters are keyword-only: agents (required), tool_name='run_workflow', max_agent_calls=50 (exact host-enforced ceiling on sub-agent runs per parent run, shared across all run_workflow calls), max_retries=3, forward_usage=True, inherit_model=False (children keep their own models; True makes them follow the parent run's resolved model, e.g. after a per-run model override), sub_agent_usage_limits=None (applied to each child run), resource_limits=None (backstop 256 MB, no time cap; dict merges, 'unlimited' disables), call_scheduler=None (a (RunContext) -> WorkflowCallScheduler factory, called once per run_workflow call, for integrations that decide when sub-agent calls start; not called under DBOSDurability), id/description/defer_loading.

Gotchas:

  • Agent name must be a valid Python identifier; WorkflowAgent(agent, name=..., description=...) (from pydantic_ai_harness.dynamic_workflow) renames without editing the agent.
  • task is keyword-only in the script. A structured output_type arrives as a dict: r['field'].
  • Without sub_agent_usage_limits, children sharing usage run under the parent run(usage_limits=...); with forward_usage=False they get pydantic-ai's default request_limit=50. For an exact ceiling use max_agent_calls. With shared usage and concurrent fan-out, limits are best-effort.
  • Workflows do not nest: do not give catalog agents DynamicWorkflow.
  • Sandbox: no third-party imports, no clock/randomness/filesystem/env. A child failure raises RuntimeError in the script; uncaught, the whole script retries. A child run that ends with DeferredToolRequests or PendingModelResponse is a failure too (a model request it left pending is cancelled first, best effort), and the catalog leaves both types out of its signature.
  • Past max_agent_calls a call raises in the script, and whatever the script does next (even a try/except that finishes), run_workflow returns {'error': '...exhausted its sub-agent call budget...', 'last_error': ..., 'completed': [...]}; later calls in that run are refused.
  • Children do not inherit the parent's workspace: give each its own workspace capability. Children gathered in parallel on one checkout can overwrite each other's edits.
  • defer_loading=True requires a stable id. workflow.reveal(agent) adds a sub-agent mid-run.
  • A WorkflowCallScheduler gets each call as a WorkflowCall (agent_name, start) and must release held calls in flush. Under AWSLambdaDurability a released call has to await call.start() without suspending first (no semaphore or other await before it), or it misses the parallel batch and the script hangs.
  • Not agent-spec serializable.

Advisor

Lets the executor consult another model. Uses the provider-native advisor tool when executor and advisor share an Anthropic or OpenRouter provider (string model names only), else a local advisor function tool backed by a separate agent.

from pydantic import BaseModel
from pydantic_ai import Agent

from pydantic_ai_harness import Advisor


class Decision(BaseModel):
    proceed: bool
    risk_score: float


agent = Agent('test', capabilities=[Advisor('anthropic:claude-opus-5-5', max_uses=1)])
typed = Agent('test', capabilities=[Advisor('openai:gpt-5.6-sol', output_type=Decision)])

Advisor(model, *, native=True, local=True, output_type=str, max_uses=None, max_tokens=None, caching=None, forward_history=False, id=None, defer_loading=False, description=None). By default it uses the provider tool where it applies (history gets provider-native parts) and the advisor function tool otherwise (history gets a ToolCallPart and ToolReturnPart). local=False requires the provider tool; native=False always uses the function tool. mode= is deprecated.

<!-- TODO(harness 1.0, #12997): remove the `mode=` deprecation sentence above. -->

The executor decides when to consult. To require a consultation before a verdict, use native=False and check this run's history in an output validator:

@agent.output_validator
def advised(ctx: RunContext[None], output: str) -> str:
    responses = [m for m in ctx.messages if isinstance(m, ModelResponse) and m.run_id == ctx.run_id]
    if output == 'APPROVE' and not any(p.part_kind == 'tool-call' and p.tool_name == 'advisor' for m in responses for p in m.parts):
        raise ModelRetry('Call `advisor` before approving.')
    return output

Gotchas:

  • Raise at construction: max_uses < 1, max_tokens < 1024, a non-str output_type with local=False, local=False without an anthropic:/openrouter: string, OpenRouter native with max_uses. local=False with an executor on a different provider raises UserError at run time.
  • A Model instance, a non-default output_type, or max_uses on OpenRouter select local execution.
  • max_uses can be a callable (ctx) -> int, resolved per executor request, for a budget across runs kept in deps. Below 1, the native tool is omitted and the local tool returns its limit message.
  • Local path: the advisor sees only the consultation prompt (plus completed history, including system_prompt parts, tool calls, and results, with forward_history=True), never the executor's deps or tools. Its usage counts toward parent limits. A failed consultation still counts toward max_uses. A parent count_tokens_before_request=True is kept on an advisor model that implements count_tokens and retried once without the up-front count (limits checked after the response) on any pre-request NotImplementedError, unless a request was counted in the shared usage since the consultation started, in which case the error propagates.
  • Two Advisors with the same id merge, a None option taking the other instance's value, two list output_types combining, and any other option the later one's. With different ids, their built advisor tools conflict.
  • Durable runs: DBOS records each local consultation under the default id (advisor; keep it stable once deployed). Under Temporal and Prefect use local=False: their activity- or task-local usage doesn't merge back into the run, so local execution is unsupported there.

BackgroundTools

Selected tools start, return a "running in background (task id)" message immediately, and deliver the result as a follow-up message when done.

Each follow-up, including a handled failure notification, is a BackgroundToolResultPart (a DataCapabilityPart from pydantic_ai_harness.background_tools, with tool_name, tool_call_id and outcome fields); find follow-ups by that type. The Vercel AI and AG-UI message conversions leave capability parts out. The tool's return value and files, then any ToolReturn.content, go in the same frame; ToolReturn.metadata is discarded.

The follow-up wraps what the tool returned in <tool_result tool_name="..." tool_call_id="..."> tags, like the <pyai:tool_result> tags Pydantic AI puts around tool files it sends on the user channel, with a note that the tool wrote it, not the user or the application. This tells the model the text is the tool's output rather than something the user said, and which pending call it completes. The output goes out as written: text in it that looks like <tool_result> or an application frame such as <system> gets a further note naming the tool as its author, so it cannot end the frame early. A failed call is framed the same way with outcome="failed", wrapping the message from ModelRetry or ToolFailed.

import asyncio

from pydantic_ai import Agent

from pydantic_ai_harness import BackgroundTools

agent = Agent('test', capabilities=[BackgroundTools()])


@agent.tool_plain(metadata={'background': True})
async def slow_research(query: str) -> str:
    await asyncio.sleep(60)
    return f'Findings for {query!r}'

BackgroundTools(tools={'background': True}) takes any ToolSelector: 'all', names, metadata dict, or lambda ctx, td: .... metadata={'background': 'optional'} adds a run_in_background argument the model sets per call (your function never receives it; a tool with its own run_in_background parameter raises UserError).

Gotchas:

  • run_stream()/run_stream_sync() wait for background tools but do not deliver results; use run(), run_sync(), run_stream_events(), or fully consume agent.iter().
  • The run waits for pending tools; a pause or early stop cancels them. Async tools must honour cancellation; sync tools cannot be interrupted.
  • Delivered results skip tool-result/tool-error hooks; an unexpected tool failure shows the model only the error type.
  • Sequential tools and realtime sessions run normally. Do not call ctx.enqueue() inside a durable activity.

See also

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at b2d64a3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 days ago
compatibility
Requires Python 3.11+
metadata
{
  "version": "0.2.0",
  "author": "pydantic"
}

README badge

README badge for pydantic/pydantic-ai/pydantic-ai-harness