Code Mode
CodeMode wraps eligible tools into a single run_code tool so the model can write Python that loops,
branches, aggregates, and parallelizes multiple tool calls inside a sandbox.
Use it when the agent needs to call several tools, transform intermediate results, run concurrent
tool work with asyncio.gather, or anywhere a simple Python script is more reliable than the model alone, such as for mathematics.
Install
Code Mode needs the Monty sandbox, pulled in by the codemode extra:
uv add "pydantic-ai-harness[codemode,anthropic]" # `code-mode` is an alias; `anthropic` for the examplesWithout it, importing CodeMode raises ImportError: pydantic-monty is required for CodeMode.
Basic Pattern
from pydantic_ai import Agent
from pydantic_ai_harness import CodeMode
agent = Agent('anthropic:claude-opus-5-5', capabilities=[CodeMode()])
@agent.tool_plain
def get_weather(city: str) -> dict:
return {'city': city, 'temp_f': 72, 'condition': 'sunny'}
@agent.tool_plain
def convert_temp(fahrenheit: float) -> float:
return round((fahrenheit - 32) * 5 / 9, 1)The model could generate code like:
import asyncio
paris, tokyo = await asyncio.gather(
get_weather(city='Paris'),
get_weather(city='Tokyo'),
)
paris_c = await convert_temp(fahrenheit=paris['temp_f'])
tokyo_c = await convert_temp(fahrenheit=tokyo['temp_f'])
{'paris': paris_c, 'tokyo': tokyo_c}Choose Which Tools Are Sandboxed
The tools parameter (a Pydantic AI ToolSelector) controls which tools move behind run_code.
from pydantic_ai_harness import CodeMode
CodeMode(tools='all')
CodeMode(tools=['search', 'fetch'])
CodeMode(tools=lambda ctx, td: td.name in {'search', 'fetch'})
CodeMode(tools={'code_mode': True})Metadata-based selection is useful when the project already groups tools into toolsets:
from pydantic_ai import Agent
from pydantic_ai.toolsets import FunctionToolset
from pydantic_ai_harness import CodeMode
search_tools = FunctionToolset(tools=[search, fetch]).with_metadata(code_mode=True)
agent = Agent(
'anthropic:claude-opus-5-5',
toolsets=[search_tools],
capabilities=[CodeMode(tools={'code_mode': True})],
)Non-matching tools remain regular tool calls. Some tools always stay native even with tools='all':
framework control tools, undiscovered deferred (defer_loading=True) tools, requires_approval=True and external tools, native fallbacks, and other
code-execution tools (any tool with code_arg_name metadata) -- including Shell's run_command, start_command, and shell, and CapabilityCreation's
author_capability. Shell's check_command/stop_command and all FileSystem tools are folded in.
Provider-native tools (native=True) execute server-side and never reach run_code.
With the default tools='all', every tool from MCP or hosted-integration capabilities (for example
GitHub or Linear write tools) moves behind run_code too. Only tools registered with requires_approval=True and external tools stay
native: tools gated with .approval_required() are still sandboxed, where a gated call fails as a retry
unless a HandleDeferredToolCalls capability resolves it inline. When only some tools should be batched, tag those tools'
toolset with .with_metadata(code_mode=True) and use CodeMode(tools={'code_mode': True}), so
approval-gated and integration tools stay regular tool calls.
Return Values
run_code captures the last non-None expression automatically; an assignment on the last line returns nothing.
| Scenario | Return |
|---|---|
| Final expression, no print output | Last expression value |
Final assignment or None, no print output |
{} |
| Print output, no final expression | {'output': '<printed text>'} |
| Print output and final expression | {'output': '<printed text>', 'result': <last expression>} |
| Multimodal final expression | Returned natively for model processing (with prints: a list, text first) |
If the user expects a raw dict or list back, avoid unnecessary print() statements. Printed output is
capped at 10 MiB. Tool results reach the sandbox as JSON: Decimal, UUID and datetime arrive as
strings and mapping keys are stringified (bytes cross unchanged). A nested tool's ToolReturn
gives the sandbox its return_value; its content (e.g. a screenshot) skips the sandbox and reaches
the model with the run_code result, as for a direct call.
Limits and Retries
from pydantic_ai_harness import CodeMode
CodeMode(
tools='all',
max_retries=3,
max_tool_calls=25,
resource_limits={'max_duration_seconds': 10, 'max_memory': 128 * 1024 * 1024},
)max_retries(default 3): execution errors (including syntax/type errors) are sent back to the model to redraft, up to this many times. A nested tool'sToolFailedthe snippet does not catch is the exception:run_codereturns a failed result, as when the model calls that tool directly, and spends no retry. Pool creation, connection, and checkout errors propagate to the code calling the agent, not the model. Inside Temporal they follow the workflow failure policy below. Capacity refusals get one checkout retry after a short backoff before propagating.max_tool_calls(default 100): nested calls perrun_codecall. A call past the budget fails at its call site; if uncaught, the model gets a retry listing the calls that already started.- The run's
UsageLimits(tool_calls_limit=...)counts nested calls and therun_codecall that makes them. A nested call that would exceed it ends the run withUsageLimitExceeded, even when the snippet catches the error. A call launched early byspeculatecounts from the moment it starts, including one the snippet never claims. A launch the limit refused runs cold if the snippet asks for it. resource_limits(default: 30 s execution and 256 MiB heap per snippet): keysmax_duration_seconds,max_memory,max_suspensions(default 1,000 host interactions per REPL session, shared across snippets). Unknown keys raiseUserError. Time awaiting tools does not count; sleeps get a separate allowance ofmax_duration_seconds. A snippet that hits the time limit is stopped and its REPL session is reset.'unlimited'removes time and memory caps only when another boundary supplies them;max_suspensionscan never be disabled.
REPL State
State persists between run_code calls during the same agent run. Imports, variables, and helper
functions carry over until the run ends. The model passes restart: true in the run_code arguments to
reset state. A timeout, worker crash, or interrupted eager statement also resets the session, and the
retry message says so -- the next snippet must recreate its state.
A pool's request_timeout and a remote turn_timeout ask the model to do less work per call.
Remote session and idle timeouts explain that state is lost. Sandbox memory-limit errors and
MemoryErrors without a traceback reset the session and ask the model to use smaller batches.
Other snippet or nested-tool MemoryErrors keep state unless their diagnostic is indistinguishable
from a sandbox memory limit: Monty does not guarantee heap integrity after a limit fires.
Exceeding the 10 MiB printed-output cap also resets the session because it has the same diagnostic.
Remote host-side error details are omitted
from retries because they may contain pool URLs or credentials; the original exception remains
the cause for application-side diagnostics. Retries list nested calls that already started so
the model can avoid repeating their side effects.
State does not carry over to the next agent run unless repl_store is set: an object with async
load(ctx) -> bytes | None and save(ctx, state: bytes | None) methods
(pydantic_ai_harness.code_mode.CodeModeReplStore), usually
keyed by ctx.conversation_id. A run that calls run_code restores the saved state before its first
snippet and saves the REPL when it ends; restart: true replaces it (save(ctx, None) when nothing
replaces it), and a state saved under other resource_limits or that Monty cannot load is dropped so
the run starts fresh. Return only states the application saved: Monty does not authenticate them. Restored state skips
the first-snippet type check. Store errors are logged, not raised; a run whose load failed
saves nothing unless it restarts, so the unread state survives. The store is not used under durable
execution.
Sandbox Restrictions
Code runs inside Monty, an implementation of Python which intentionally supports only a subset of features.
Key restrictions:
- No third-party imports
- No
import * - Only a small stdlib subset is allowed, and each must be imported before use:
sys,typing,asyncio,math,json,re,unicodedata,datetime,time,random,os,pathlib,collections,itertools,functools,dataclasses,copy,base64,binascii,bisect,csv,decimal,fnmatch,hashlib,os.path,tomllib,annotationlib asyncio.gather(...)accepts positional awaitables but no keyword arguments; other task creation and wait APIs are unavailable- No clock or randomness by default:
datetime.datetime.now(),datetime.date.today(),time.time(), and unseededrandomrequire anos_accesshandler (or, for the clock,os_policy);time.sleepandasyncio.sleepreally wait, within the sleep allowance - Filesystem I/O requires an
os_accesshandler or amount;os.getenvandos.environrequire anos_accesshandler - A sandboxed tool named after a Python builtin (
list,open,print, ...) raisesUserError; rename it or usetoolset.prefixed(...) - Approval-required (
requires_approval=True) and external (kind='external') tools stay native undertools='all', so their deferred calls pause the run as usual. An explicit selector can still sandbox them. A sandboxed call that needs approval or is deferred, including one from a toolset wrapped with.approval_required()or a tool that raisesApprovalRequiredorCallDeferredfrom its body (both keepkind='function'), needs aHandleDeferredToolCalls(or equivalent) capability to resolve it inline; without one, it raises an error that surfaces to the model as a retry
The sandbox constrains the model-generated Python, not the implementation of the tools it calls. Wrapped tools retain their normal host and network access, so expose only tools with the authority and input validation appropriate for model-generated calls.
When a generated example keeps failing, check these restrictions before changing the rest of the agent.
Host Access: mount and os_access
Leave both unset unless the task requires host access, and grant only what the task needs. Both are fixed
when CodeMode is built, so construct it per request to scope access to that request.
from pathlib import Path
from pydantic_monty import MountDir, OSAccess
from pydantic_ai_harness import CodeMode
work = Path('agent-work')
work.mkdir(exist_ok=True) # the host directory must exist
CodeMode(mount=MountDir(virtual_path='/work', host_path=str(work), mode='read-write'))
CodeMode(os_access=OSAccess(environ={'API_BASE': 'https://api.example.com'}))mounttakes aMountDiror a list of them;host_pathmust already exist (MountDirraisesTypeErrorotherwise). The defaultmode='overlay'is copy-on-write: writes are visible only within the currentrun_codecall and never reach the host. Usemode='read-write'when writes must persist (including acrossrun_codecalls),mode='read-only'to forbid them.os_accesstakes anAbstractOS(e.g.OSAccess, isolated in-memory filesystem and env, host clock) or a keyword-argument callbackdef handler(*, name, args, kwargs, **_)(may beasync). Return any value (includingNone) to answer the call; returnpydantic_monty.NOT_HANDLEDto refuse it (raises in the sandbox and burns a retry). The positional(name, args, kwargs)form is deprecated.- Mounts are directories on the machine running the agent, not the run's workspace. With a remote sandbox such as
ModalSandbox,FileSystemandShellact in the sandbox while mountedpathlibcode still touches the host -- for files the model shares with its commands, sandbox the workspace tools (FileSystem) instead of mounting.
Other Options
CodeMode(
tools: ToolSelector = 'all',
max_retries: int = 3,
*,
max_tool_calls: int = 100,
os_access: CodeModeOS | None = None,
mount: CodeModeMount | None = None,
os_policy: CodeModeOSPolicy | None = None, # Monty clock/timezone policies over the defaults
resource_limits: CodeModeResourceLimits | Literal['unlimited'] | None = None,
eager: bool = False, # run complete streamed statements before the call finishes
speculate: Sequence[str] | Literal['declared'] | None = None, # start read-only calls while streaming
monty_sandbox: str | Callable[[], AsyncMontyRemote | Awaitable[AsyncMontyRemote]] | None = None, # an http(s):// server root URL, or a function returning the pool
repl_store: CodeModeReplStore | None = None, # keep the REPL across agent runs
dynamic_catalog: bool = True, # keep the tool catalog in an appending instruction block
)dynamic_catalog=True(default): moves sandboxed-tool signatures out ofrun_code's description into an instruction block (toolset:code_mode:catalog,on_change='append'), keeps deferred tools thatrun_codewill take over off the wire until they're found, and appends the updated catalog when one is found (plus a shortDiscoveryAnnouncementPart, anInstructionCapabilityPartwhosetoolsfield lists what it announces, when Tool Search found it), so discoveries don't rewrite the cached prompt. Discovery then goes through the localsearch_toolsorload_capability, which also searches deferred tools a selector keeps native (the native tool search isn't sent then); with the'bm25'/'regex'strategies, such tools are still declared for the native search. The catalog takes the toolset idcode_mode, so another instruction-contributing toolset with that id raises a duplicate-idUserError; rename it. A callable selector must give the same answer for a tool before and after it's revealed, or the cache moves.dynamic_catalog=Falsekeeps the catalog inrun_code's description instead (the default in earlier releases), so a reveal rewrites the description and the cached prompt.eager=True: side effects from early statements cannot be rolled back, and hooks/approval onrun_coderun only after the call finishes streaming. Only applies whenrun_codeis the first tool call in the response.speculate: pass tool names that are safe to run early, or'declared'to trustTool(metadata={'read_only': True})and MCPreadOnlyHint. Only calls with literal keyword arguments start early; unclaimed launches still cost what they cost.CodeMode.speculation_statsreports launched/adopted/evicted.eagerandspeculateput runs in streaming mode, need asyncio, and are inactive under durable execution (Temporal, DBOS, Prefect).os_policy: Monty'sOSPolicy, merged key by key over the defaults (clock, sleeps, and random seeding routed toos_access).{'datetime': 'system', 'timezone': 'Europe/Paris'}gives the sandbox the worker's clock in that zone with no handler;{'timezone': ...}alone keeps the clock onos_accessbut makesastimezone()and%Zreport the zone. Keep the defaults under Temporal: a'system'clock orrandom_startis re-read on replay. Fixed values stay fixed.- With
InputGuardrail(parallel=True), nested tool calls fromeagerandspeculatewait for the guard's verdict, so a blocked prompt runs no tool. monty_sandbox: anhttp:///https://server root URL (no key; no path, query, or fragment;ws:///wss://still work) or a sync or async function returning anAsyncMontyRemote(passmetadata=for anauthorizationheader), to run code on a remote Monty worker over gRPC. A factory pool adds itsrequest_timeout(10 s default) per-turn transport deadline; the configuredmax_duration_secondsalso applies, except in Temporal workflow code. The factory runs once per run, or once per turn when turns run as activities. Only execution moves; tools,os_access, and prints stay host-side, and so do mounts unless turns run as activities, where the activity worker reads them. Run the worker onpydantic-monty>=1.5.0.monty_sandbox_urlis the deprecated form.- Configure credentials for a remote
monty_sandboxin the pool factory, not in the model's instructions or tool arguments.
Durable Execution
CodeMode works next to TemporalDurability() (install pydantic-ai-harness[codemode,temporal]), but
run_code executes in workflow code and is re-run on replay. Keep snippets deterministic: put external
reads, writes, and clock access in wrapped tools (which become activities), not in mount/os_access.
Inside a Temporal workflow max_duration_seconds is disabled for code run in workflow code (memory and suspension caps still apply) and a
workflow task that does not yield within 2 seconds fails, so move heavy computation into a tool.
With a remote monty_sandbox, each turn (the code between two waits on the
host: nested calls, sleeps, deferred calls) runs as its own activity instead, so sandbox computation
does not count against the 2 seconds and replay reads turn results from history. This needs a server
that stores sessions (Logfire Agent Sandboxes does); on one that stores none, the run falls back to
workflow code, and a storing server that cannot store a session's state raises a non-retried
UserError. Runs stay workflow-side with id=None, under the deprecated
TemporalAgent, with os_access set, with an 'overlay' mount (the default mode; use 'read-only'
or 'read-write') or a write_bytes_limit, or when the workflow started before this change. Each
turn takes and returns at most 1 MiB: longer output is cut, and longer tool results fail the snippet
with a ValueError that gives the size and resets the session.
Turns use TemporalDurability's activity_config; a turn that still fails after its retries fails
the workflow with ActivityError. In a turn activity max_duration_seconds applies again, so keep
start_to_close_timeout above it: Temporal's timeout does not stop a running turn. A retried turn
repeats its 'read-write' mount writes. Recorded pool errors keep their type name, so
non_retryable_error_types still match. A CodeMode a DynamicCapability returns keeps its turns
in workflow code. Tool results and call arguments keep their types across turns, and the model sees the
same result; nested-call arguments of a type Monty does not hand the host, or over 512 KiB, raise TypeError.
Setup errors reach workflow code with their original types. PydanticAIPlugin treats UserError
as a workflow failure. Uncaught RuntimeError, MontyRefusedError, and other unclassified factory
errors fail the workflow task instead: Temporal retries the task while the workflow remains running,
so the client keeps waiting if setup keeps failing. Inspect the task history or worker logs for the original error. To end the
workflow, catch expected setup exceptions and raise
temporalio.exceptions.ApplicationError(..., non_retryable=True) from the original exception.
Agent Specs
When the user defines the agent in YAML or JSON, the loader needs to know how to build CodeMode.
YAML files also need PyYAML: uv add "pydantic-ai-harness[codemode,anthropic]" "pydantic-ai-slim[spec]".
model: anthropic:claude-opus-5-5
capabilities:
- CodeMode: {} # or with arguments: CodeMode: {tools: ['search', 'fetch'], max_retries: 5}from pydantic_ai import Agent
from pydantic_ai_harness import CodeMode
agent = Agent.from_file('agent.yaml', custom_capability_types=[CodeMode])Observability
With Logfire or another OpenTelemetry backend, nested tool calls inside run_code produce child spans.
The run_code tool return also carries history-only metadata keyed by nested call id (plus a
speculation summary when speculate is set):
for msg in result.all_messages():
for part in msg.parts:
if isinstance(part, ToolReturnPart) and part.tool_kind == 'code_mode.run_code':
tool_calls = part.metadata['tool_calls'] # dict[str, ToolCallPart]
tool_returns = part.metadata['tool_returns'] # dict[str, ToolReturnPart]For an offline test that drives run_code with a FunctionModel, see Testing and Debugging.
Recognizing run_code in Hooks
run_code declares the 'code_mode.run_code' tool kind, so its call parts are promoted to
RunCodeCallPart (from pydantic_ai_harness.code_mode). Match it with isinstance, not by tool name;
code holds the submitted snippet, and typed_args all the arguments validated as RunCodeArgs:
async def wrap_tool_execute(self, ctx, *, call, tool_def, args, handler):
if isinstance(call, RunCodeCallPart) and call.code is not None:
... # inspect or record call.code
return await handler(args)run_code parts recorded before the kind existed load as plain ToolCallParts;
ToolCallPart.narrow_type(part, tool_kind='code_mode.run_code') promotes one.
IPython Code Mode (local agents only)
IPythonCodeMode gives the model the same run_code tool backed by a persistent IPython kernel: full
CPython, any installed package, and state kept for the whole run. It is not a sandbox: model-written code
runs with the agent process's permissions. Never use it in a webserver or on untrusted input unless the
whole agent runs in a container or VM; use CodeMode there.
uv add "pydantic-ai-harness[ipython]"from pydantic_ai import Agent
from pydantic_ai_harness import IPythonCodeMode
agent = Agent('anthropic:claude-opus-5-5', capabilities=[IPythonCodeMode()])- Tools become synchronous, keyword-only functions in the kernel; calls run one at a time and a failed
call raises
ToolCallError.tools=andmax_tool_calls=work as inCodeMode. timeout=(default 60 s, tool time excluded) interrupts long cells and keeps the kernel's state; a cell that ignores the interrupt or a kernel that exits gets a fresh kernel on the next call.- Each run starts a kernel on first use and stops it at the end. To keep state across runs, pass
kernel=IPythonKernel()(frompydantic_ai_harness.ipython_code_mode) and shut it down yourself, e.g. withasync with IPythonKernel() as kernel:. - Not supported:
eager,speculate,dynamic_catalog, durable execution,mount/os_access.