Agents Core
Read this file when the user needs the core Agent workflow: creating agents, choosing output types, using dependencies, defining specs, selecting models, reusing an OpenAI Responses WebSocket, or choosing how to run/stream an agent.
Create a Basic Agent
Include Logfire instrumentation in new applications from the start, so every run is visible; see Debug and Validate Agent Behavior for credentials and other OpenTelemetry backends.
import logfire
from pydantic_ai import Agent
logfire.configure()
logfire.instrument_pydantic_ai()
agent = Agent(
'anthropic:claude-fable-5-2',
name='hello_world_agent',
instructions='Be concise, reply with one sentence.',
)
result = agent.run_sync('Where does "hello world" come from?')
print(result.output)Pass an explicit name= to each agent: it labels the agent's run span in Logfire. When omitted, the name is inferred from the variable the agent is assigned to and falls back to 'agent' when it can't be (e.g. agents kept in a list or dict), which matters once more than one agent runs in the same app.
Structured Output with Pydantic Models
Use output_type=MyModel when the model should return validated structured data.
from pydantic import BaseModel
from pydantic_ai import Agent
class CityLocation(BaseModel):
city: str
country: str
agent = Agent('google:gemini-3-flash-preview', name='city_location_agent', output_type=CityLocation)
result = agent.run_sync('Where were the olympics held in 2012?')
print(result.output)If the user is choosing between output modes:
output_type=strfor plain textoutput_type=MyModelfor structured outputTextOutputfor custom text parsingNativeOutputorToolOutputwhen they need explicit output-mode control
Picking One of a Run-Time Set
Use Choices({key: description}) when the model has to pick one of a set that only exists once the run is under
way — the records a search returned, the actions available on a screen. Each option carries its meaning into the
schema, the output is validated against the keys, and it is a type, so the same value also works as a model field
or a tool parameter — there, write Annotated[str, Intent] so type checkers see a str. For a set you know when you write the code, use a Literal or an Enum (with
UseEnumMemberDocstrings for per-member descriptions), which give exhaustiveness checking.
from pydantic_ai import Agent, Choices
agent = Agent('openai:gpt-5.2', name='triage_agent')
result = agent.run_sync(
'The blender arrived smashed. Just send me another one.',
output_type=Choices(
{'refund': 'They want their money back.', 'replace': 'They want a working unit.'},
description='What the customer is asking for.',
),
)
print(result.output)Choice(description, value=...) makes an option stand for something other than its key. When that value is
callable, picking it calls it — sync or async, with no arguments — so the run's output is what the action
returned, the way an output function's is, and ModelRetry from it sends the model back for another pick. A set
with a callable value is an output_type only; as a field or a parameter it raises UserError.
Validate Output Against Your Domain
The output type's Pydantic validation checks shape. Checks that need the run's deps or I/O — an id the model returned
must be one the app gave it, a referenced record must exist — go in an @agent.output_validator. Raising ModelRetry
there sends the message back to the model, which corrects its output within the same run. Domain checks belong in
the validator, not after run(): code that rejects result.output once the run has returned has lost the retry and
can only start over.
from dataclasses import dataclass
from pydantic import BaseModel
from pydantic_ai import Agent, ModelRetry, RunContext
@dataclass
class TriageDeps:
open_ticket_ids: set[str]
class Triage(BaseModel):
ticket_id: str
summary: str
agent = Agent('openai:gpt-5.2', name='triage_agent', deps_type=TriageDeps, output_type=Triage)
@agent.output_validator
def check_ticket_id(ctx: RunContext[TriageDeps], output: Triage) -> Triage:
if output.ticket_id not in ctx.deps.open_ticket_ids:
raise ModelRetry(f'{output.ticket_id!r} is not an open ticket. Use one of {sorted(ctx.deps.open_ticket_ids)}.')
return outputValidators can be async, so a lookup can use a database client from deps. Look up all the ids in one query, so the number of queries doesn't grow with the size of the model's output:
from dataclasses import dataclass
from typing import Protocol
from pydantic import BaseModel
from pydantic_ai import Agent, ModelRetry, RunContext
class OrderStore(Protocol):
async def existing_order_ids(self, order_ids: list[str]) -> set[str]: ...
@dataclass
class RefundDeps:
orders: OrderStore
class Refund(BaseModel):
order_id: str
reason: str
agent = Agent('openai:gpt-5.2', name='refund_agent', deps_type=RefundDeps, output_type=list[Refund])
@agent.output_validator
async def check_orders_exist(ctx: RunContext[RefundDeps], output: list[Refund]) -> list[Refund]:
if ctx.partial_output: # streamed runs also validate partial output; check only the final one
return output
order_ids = [refund.order_id for refund in output]
missing = set(order_ids) - await ctx.deps.orders.existing_order_ids(order_ids)
if missing:
raise ModelRetry(f'No orders with ids {sorted(missing)}. Refund only orders that exist.')
return output- Each
ModelRetryuses one output retry; the default budget is 1, raised withretries={'output': N}on the agent or the run. When it runs out, the run raisesUnexpectedModelBehavior. - The validator may return a modified output; what it returns is
result.output. - When the valid ids are known as the run starts and the model must pick one,
Choices(above) puts them in the schema itself; a validator also covers ids nested in larger outputs and checks that need I/O.
Dependency Injection
Use deps_type=... plus RunContext[...] when tools or instructions need app state.
from pydantic_ai import Agent, RunContext
agent = Agent('openai:gpt-5.2', name='greeting_agent', deps_type=str)
@agent.instructions
def add_user_name(ctx: RunContext[str]) -> str:
return f"The user's name is {ctx.deps}."Use @agent.tool when the tool needs RunContext. Use @agent.tool_plain when it does not.
For instructions built from operator-controlled state that changes mid-conversation (todo lists, memory, per-run context), use on_change='append' to preserve the instruction prefix and the cache behind it; the cost is the block's complete new text added to history per change.
Use @agent.instructions(name='state', on_change='append') on the agent.
The callback still runs before each request; changed values become full replacements in history, sent as a diff when that's shorter.
Return None or an empty string to withdraw the block.
Use the same decorator on an always-on capability with an explicit id.
Toolsets can return InstructionPart(content=..., name='state', on_change='append') from get_instructions() under their own id, or use @toolset.instructions(name='state', on_change='append') on a FunctionToolset with an id.
Any instruction function (passed to Agent(instructions=...), returned from a capability's get_instructions(), ...) can return InstructionPart(..., name='state', on_change='append') to opt in from inside the function.
Per-run context passed to agent.run(..., instructions=...) appends when it's a named part: instructions=InstructionPart(..., name='page', on_change='append') is keyed run:page; unnamed runtime instructions always rewrite.
A custom capability's get_instructions() can also return such an InstructionPart directly, for text that changes between runs when for_run returns a replacement carrying the new text (otherwise the construction-time result is reused).
Literal text in Agent(instructions=...), Capability(instructions=...) and FunctionToolset(instructions=...) opts in with InstructionPart('...', name='config', on_change='append').
Template strings can't opt in; use an instruction function that returns an InstructionPart instead.
Build instruction text from operator-controlled state, never from tool arguments, tool output, user input or retrieved content, even in append mode; send untrusted values in user messages or tool results.
Keep serialized message history in trusted server storage, including its InstructionBaselineParts and InstructionDeltaParts.
Compaction or loss of the baseline starts a new prefix from current instructions.
Use a durable operation for instruction callbacks that read external state during durable execution.
Deferred capabilities and realtime sessions do not support this update policy.
Leave the default on_change='rewrite' when prefix preservation is unnecessary.
A @agent.system_prompt(dynamic=True) function is re-run for each later run of a conversation and, by default, rewrites its part in history in place, which busts the prompt cache. Prefer instructions; when a system prompt is needed (other agents continue the history), use @agent.system_prompt(on_change='append') (which implies dynamic=True): a changed value is appended as a new SystemPromptPart sent as <pyai:context id="system_prompt:<name>">, where <name> (the function's __name__, or name=) is visible to the model. History ties the prompt to the function's __qualname__, so renaming or moving the function disconnects it from existing conversations, even with name=.
The deferred capability catalog and Harness SubAgents and RepoContext instructions already append their changes.
So do MCP server instructions from MCPToolset(..., include_instructions=True), keyed toolset:<id> (a URL toolset's id defaults to its hostname; durable execution needs an explicit one); stdio and in-process servers need an id= to append. Only enable include_instructions for servers you trust. The MCP capability includes them by default; pass MCP(..., include_instructions=False) for a server you don't trust, such as one your end users supply.
Define Agents Declaratively with Specs
Use YAML or JSON specs when configuration should live outside Python code.
model: anthropic:claude-opus-4-6
instructions: "You are helping {{user_name}} with research."
capabilities:
- WebSearch
- Thinking:
effort: highfrom dataclasses import dataclass
from pydantic_ai import Agent
@dataclass
class UserContext:
user_name: str
agent = Agent.from_file('agent.yaml', deps_type=UserContext)
result = agent.run_sync('Find recent papers on AI safety', deps=UserContext(user_name='Alice'))Template strings are part of the spec flow, so route template-string questions here too.
A spec has the same authority as the code that builds the agent: it picks the model, the capabilities, and the endpoints they connect to, such as an MCP server URL, localhost and private addresses included. Load specs only from trusted sources; if users supply them, check each spec's capabilities (name and kwargs, such as an MCP url) against what that user may use before building the agent.
Choose or Configure Models
Model strings use the "provider:model-name" format.
Examples:
openai:gpt-5.2anthropic:claude-sonnet-4-6google:gemini-3-pro-preview
Through the Pydantic AI Gateway, one PYDANTIC_AI_GATEWAY_API_KEY reaches models from several providers, with spending limits and cost monitoring in Logfire. Prefix the model string with gateway/, using the provider's API format:
gateway/anthropic:claude-fable-5-2gateway/openai:gpt-6-sol
Use a model instance instead of a string when the user needs provider-specific constructor arguments.
A request to a model its provider has deprecated or retired emits a DeprecatedModelWarning naming the retirement date and the provider's recommended replacement. Move to that replacement rather than silencing the warning.
Keep Anthropic Inference in the US
For a data residency requirement on Anthropic, pass AnthropicModelSettings(anthropic_inference_geo='us'); don't use extra_body. It's sent on the Claude API, Claude Platform on AWS, Claude on Google Cloud (Anthropic managed) and Message Batches, and reported back in ModelResponse.provider_details['inference_geo'].
Models older than Claude 4.6 reject it with a 400, which surfaces as ModelHTTPError rather than the setting being dropped. Bedrock, Vertex AI and Foundry clients ignore it with a UserWarning: there, pick the region through the endpoint, inference profile or deployment.
Use Anthropic data residency for pricing and details.
Reuse an OpenAI Responses WebSocket
Install pydantic-ai-slim[openai-realtime]. Pass OpenAIResponsesModelSettings(openai_transport='websocket') as settings= on OpenAIResponsesModel, or as model_settings= on an agent or run. A plain dict fails type checking because openai_transport is not a ModelSettings key.
Other models ignore the setting, so code that takes a model string doesn't change.
Each run opens a socket on its first request that can use one and closes it at the end; wrap back-to-back runs, or an app's lifetime, in async with agent: or async with model: to keep one socket.
WebSocket-mode requests, including HTTP overflow, default openai_previous_response_id to 'auto', continuing only from reachable state. A single-request run pays the handshake for nothing.
Runs given the same model instance share one socket per event loop: on api.openai.com each concurrent request gets its own stream_id lane; other hosts run one at a time unless the profile sets openai_responses_supports_websocket_stream_id=True.
These requests use HTTP without a warning: a direct model.request() outside a run or async with, durable execution units without an outer model or agent context on their loop, background or resumed requests, no free lane, and headers that differ from the handshake's. To share a connection across Temporal activities, enter the registered model around Worker; the activity result still checkpoints independently. Inspect pydantic_ai.transport on the chat span. Under Temporal, inspect the model activity span instead. WebSocket mode records 'websocket' or 'http'. HTTP-only mode leaves the attribute unset.
Early rejection frames with status 408, 409, 429 or 5xx use the effective max_retries budget. Set it to zero when the durable engine owns retries; a disconnect or error after response events start propagates to the caller.
A missing extra, non-asyncio backends, and models whose profile sets openai_responses_supports_websocket=False warn and use HTTP. profile=OpenAIModelProfile(openai_responses_supports_websocket=True) overrides the flag for Azure and the gateway with static credentials. The OAuth providers, OpenAICodexModel, and clients with API key callbacks, workload identities, Azure AD token providers or HTTP client authentication always use HTTP, so credentials resolve for every request.
A closed or failed socket, or one nearing OpenAI's 60-minute limit, is replaced on the next request; a failure mid-request raises ModelConnectionError, so ModelFallback works.
A response stored server-side (the default) continues from any socket or HTTP; with openai_store=False, keep the full message history, since only the socket that created it can continue from it.
Use OpenAI Responses WebSockets for the full list of fallbacks and limits.
Run Methods and Streaming
Pick a run method based on the interaction pattern:
run()for async runs that complete normallyrun_sync()for synchronous scripts and notebooksrun_stream()for streaming final outputrun_stream_sync()for sync streamingrun_stream_events()when the caller needs the typed event stream directlyiter()when the caller needs step-by-step control over the agent loop
Use event_stream_handler= with run() or run_stream() when the user wants progress updates without manually consuming the event stream. The stream includes model deltas, tool call/result events, and framework events such as EnqueuedMessagesEvent when queued messages enter run history.
Realtime sessions do not use event_stream_handler; iterate the session to consume realtime-only RealtimeEvent members.
from collections.abc import AsyncIterable
from pydantic_ai import Agent, AgentStreamEvent, FunctionToolCallEvent, RunContext
agent = Agent('openai:gpt-5.2', name='streaming_agent')
async def stream_handler(ctx: RunContext, events: AsyncIterable[AgentStreamEvent]):
async for event in events:
if isinstance(event, FunctionToolCallEvent):
print(f'Calling {event.part.tool_name}...')
async def main():
await agent.run('Do the task', event_stream_handler=stream_handler)Deferred tool calls also surface as batch-level events: DeferredToolRequestsEvent (once per batch of deferred calls, before any HandleDeferredToolCalls handler runs) and DeferredToolResultsEvent (when a handler resolves requests inline). Use these to tell a frontend the run is paused waiting for approvals or external calls.
To surface progress or intermediate results from an async tool into the same event stream without polluting the model's context, define a dataclass subclass of CustomEvent (its fields are the payload; the event name derives from the class name) and await ctx.emit(event). Sync tools cannot emit events. It reaches the event_stream_handler, run_stream_events(), iter() streaming, the AG-UI/Vercel AI adapters, and a realtime session's event stream (where events emitted from before_run lead the stream, and those emitted from after_run, after the stream has ended, aren't delivered, as in iter()); when emitted from a tool, its tool_call_id and tool_name are auto-stamped, and consumers use isinstance() against the class. Code driving agent.iter() can inject events by awaiting AgentRun.emit(). The payload can't reuse the envelope's own field names: data, tool_call_id, tool_name, and event_kind are rejected at class definition.
CustomEvent is for application-owned code only. Code that lives inside a capability must define namespaced CapabilityEvent subclasses instead; emitting either family from the other's side raises UserError. See CAPABILITIES-AND-HOOKS.md.
Custom events reach the AG-UI and Vercel AI frontends by default. For an event that should stay server-side (metrics, audit logs), declare the class ui=False — class IndexProgressEvent(CustomEvent, ui=False) — and every UI adapter skips it while in-process consumers still receive it. Declaring a ui field or ClassVar on an event class is rejected, since it would shadow that flag. The flag is class-level, not on the wire, so adapters also skip an UnknownCustomEvent (a class this process never imported): when events reach the frontend from another process, import their defining modules there or none of them are forwarded.
from dataclasses import dataclass
from pydantic_ai import Agent, CustomEvent, RunContext
agent = Agent('openai:gpt-5.2', name='progress_agent')
@dataclass(kw_only=True)
class ProgressEvent(CustomEvent):
done: int
total: int
@agent.tool
async def process(ctx: RunContext, count: int) -> str:
for i in range(count):
await ctx.emit(ProgressEvent(done=i + 1, total=count))
return 'done'Handle Provider Failures
Use the ModelFallback capability when the user wants automatic provider or model failover. The chain is the model selected for the step (the agent's, run(model=...)'s, or SelectModel's), then the capability's models.
from pydantic_ai import Agent
from pydantic_ai.capabilities import ModelFallback
agent = Agent(
'openai:gpt-6.1-sol',
name='fallback_agent',
capabilities=[ModelFallback('anthropic:claude-fable-5-2')],
)fallback_on defaults to (ModelAPIError, ContentFilterError, OutputTokenLimitExceeded): API errors, refusals (any finish_reason='content_filter' response, empty or with partial text) and cut-off responses (any finish_reason='length' response, including cut-off text that would otherwise be the output). The backup model applies its own safety policy and may answer within the token limit. Leave ContentFilterError or OutputTokenLimitExceeded out to opt out of either, e.g. when max_tokens is a deliberate cap. FallbackModel defaults to (ModelAPIError,) and falls back on refusals or cut-off responses only when ContentFilterError or OutputTokenLimitExceeded is named. Streamed refusals and cut-off responses are never fallen back from. fallback_on also takes exception types, exception handlers, and response handlers ((ModelResponse) -> bool, which reject a successful response). Each candidate gets its own prepare_model_request pass and native tool factory call (with the candidate as ctx.model), so provider-specific preparation is redone per model, and outer capabilities only see the accepted response. Use FallbackModel instead only outside an agent (direct model requests or a standalone Model).
Each step restarts the chain from the step's model. Pass sticky=True to keep a conversation on the candidate it fell back to, which keeps that provider's prompt cache warm and stops retrying a failing primary every step. The agent records the answering model as each response's model_id and a fallback as its failed_attempts, so stickiness carries over to later runs continuing that history. Only a fallback makes a conversation sticky: a candidate picked with run(model=...) that answered without one doesn't. It only replaces the agent's default model, not run(model=...), override(model=...) or SelectModel, and if the sticky candidate fails, the chain continues after it and wraps around.
In a YAML/JSON agent spec, declare it as - ModelFallback: [anthropic:claude-opus-5-5], or - ModelFallback: {models: [...], fallback_on: [ModelRateLimitError, ModelUnavailableError], sticky: true}. A spec's fallback_on takes only names of ModelAPIError classes from pydantic_ai.exceptions, ContentFilterError and OutputTokenLimitExceeded; handlers need Python. Model names resolve lazily, so ResolveModelId aliases work there too.
To retry the same model on transient failures (rate limits, unavailable/overloaded models, 5xx, connection errors and transport timeouts), add the RetryOnModelErrors capability, e.g. RetryOnModelErrors(max_attempts=4). It sends its requests with max_retries=0 so the provider SDK doesn't retry underneath it (an explicit max_retries is kept), so don't set that yourself. It waits the error's retry_after or an exponential, jittered backoff from initial_delay up to max_delay, gives up (raising the error) when the wait would exceed max_delay or end after the request's request_timeout (which spans all retries of a model) or the run's deadline, and never retries an error with an x-should-retry: false header. retry_on takes the same forms as fallback_on; its default is TRANSIENT_MODEL_ERRORS (from pydantic_ai.capabilities), which you can extend, e.g. retry_on=(*TRANSIENT_MODEL_ERRORS, MyError). Refusals (finish_reason='content_filter') aren't retried by default, since the same model usually refuses again, and ModelFallback falls back on them; naming ContentFilterError in retry_on retries them. With ModelFallback too, in either order, each model is retried up to max_attempts times before falling back, and failed_attempts lists retries (kind='retry') and fallbacks (kind='fallback') in one list. A stream is retried only before its first event reaches the consumer. Spec form: - RetryOnModelErrors: {retry_on: [ModelRateLimitError], max_attempts: 5}.
Good defaults:
- primary expensive/strong model, cheaper fallback for resilience
- same prompt/output contract across both models
- per-model settings only when the user actually needs them
model_settings={'max_retries': 0}on the agent, or on every candidate but the last, so the provider SDK's own retries (2 by default on OpenAI, Anthropic, Groq and Cohere) don't hold up the fallback; Bedrock and xAI ignore it with anUnsupportedSettingWarning, so with one of those in the chain set it on the other candidates'settings
The answering response's failed_attempts lists the attempts the fallback moved on from (model, provider,
'error' or 'rejected' outcome, error, start time, duration, usage); when every model fails they're on
FallbackExceptionGroup.attempts. A rejected response's tokens and cost count in RunUsage and towards token and
cost limits, but not in the answering response's usage. Neither they nor failed attempts count towards
UsageLimits.request_limit, which bounds the model responses the agent acts on, not requests sent to the provider.
To react to a specific kind of failure regardless of provider, catch an error category rather than inspecting status codes or provider exceptions: ModelRateLimitError (with retry_after), ModelQuotaExceededError (exhausted quota or billing, only when the provider says so; waiting won't help), ModelUnavailableError and its ModelOverloadedError (only when the provider says overloaded, e.g. HTTP 529), ModelServerError (other 5xx), ModelConnectionError (and its ModelTimeoutError, which includes an HTTP 408), or ModelRequestRejectedError (a 400/413/422 or other rejection that resending unchanged won't fix) and its ModelAuthenticationError (401/403, bad or unauthorized credentials), ModelNotFoundError (404, unknown model or resource) and ModelContextWindowExceededError. A specific category wins over the generic rejection: a 400 context overflow is a ModelContextWindowExceededError, and Anthropic's out-of-credits 400 is a ModelQuotaExceededError, not a rejection. A retry predicate can skip permanent failures with isinstance(e, ModelRequestRejectedError | ModelQuotaExceededError). All are ModelAPIErrors, so ModelFallback and FallbackModel fall back on them by default; one that came with an HTTP status is also a ModelHTTPError, and a custom model can raise such a combined error with ModelHTTPError.for_category(category, status_code=..., model_name=...). An error inside an already open stream gets the status the same error has before the stream opens, with in_stream=True, so errors don't depend on whether the request was streamed. Provider details are on provider_error_code, provider_error_type, and body; retry_after is on every ModelAPIError, ModelHTTPError.provider_retry_hint reads x-should-retry, and ModelConnectionError.phase ('pool', 'connect', 'write', 'read') says whether the request may have reached the provider. Embedding models (Embedder) raise the same errors, and so does a realtime model whose connection the provider rejects with an HTTP status.
When a completed model response omits usage, configured token or cost limits emit UsageLimitUnavailableWarning and return the result with known totals as lower bounds. Promote the warning to an error when the application must abort if a limit cannot be fully checked:
import warnings
from pydantic_ai import UsageLimitUnavailableWarning
warnings.filterwarnings('error', category=UsageLimitUnavailableWarning)To bound how long one model can hold up the fallback, set model_settings={'request_timeout': seconds}. Unlike
timeout, which the provider SDK applies to each attempt and re-arms on every retry, it is one deadline enforced by
Pydantic AI over all attempts at a request to one model, and over reading a streamed response through its last chunk
(consumer time included). Expiry raises ModelRequestTimeout (a ModelAPIError), which ModelFallback and FallbackModel fall back on
by default; each model they try gets a fresh deadline. It applies to any model, with or without a fallback. Under
Temporal, a model's deadline runs as a workflow timer that cancels its model activity, including each model
ModelFallback tries, as each attempt is an activity of its own; a FallbackModel's per-model deadlines run inside
its one activity instead.
To fall back when a streamed response doesn't start in time, without cutting off a long response once it has, set
model_settings={'first_event_timeout': seconds}. It bounds the wait from making a streamed request until its first
event (a part starting or a delta, thinking included; OpenAI's response.created doesn't count), across SDK retries
and the continuations of a paused or background response, and stops applying once that event arrives. Expiry raises
ModelRequestTimeout with setting='first_event_timeout', which FallbackModel and ModelFallback fall back on by
default. With request_timeout also set, whichever passes first wins. It has no effect on non-streamed requests.