---
name: adding-a-provider-api-feature
description: Add a new provider API capability (prompt caching, strict/structured tool calling, thinking/reasoning effort, service tier, safety settings, logprobs, etc.) to Pydantic AI. Use when wiring a provider feature through the library — it enforces reasoning from the existing cross-provider abstraction before designing anything, and picking default-on vs opt-in deliberately. Not for adding a new model id (that's a different flow) or a bug fix.
user-invocable: true
allowed-tools: Bash(git:*), Bash(gh:*), Bash(rg:*), Bash(ls:*), Bash(uv:*), Read, Write, Edit, Glob, Grep, WebFetch, AskUserQuestion, Agent
title: adding-a-provider-api-feature
canonical_url: https://skilld.dev/gh/pydantic/pydantic-ai/adding-a-provider-api-feature
last_updated: 2026-10-24T19:05:00.000Z
---

> Index of every page on this site: [llms.txt](https://skilld.dev/llms.txt)

> **Skill from skilld.dev.** Follow the user's requested action.
> If the user asked to fork this Skill, follow the fork workflow below. Do not execute the Skill instructions while copying it.
> Otherwise, follow the instructions below for this session. You do not need to install anything.
>
> If the user asked to install this Skill, run `npx skilld install pydantic/pydantic-ai/adding-a-provider-api-feature`. Install writes the Skill files into the project, so every session loads them.
>
> ## Fork workflow
>
> A fork creates an editable local Skill with its original author and licence. The request authorizes copying and local installation.
> 1. Check `./skills/adding-a-provider-api-feature`, the project lockfile, and selected Agent targets together. If the local directory or installed Skill exists, stop. Never overwrite an existing directory or Agent target.
> 2. Read [source metadata](https://skilld.dev/api/v1/skills/pydantic/pydantic-ai/adding-a-provider-api-feature) once. Use sourceUrl, sourceCommit, skillPath, sourceGone, and license. If the source is gone or its path is missing, stop. If license is null, read licence files at the source commit.
> 3. Fetch only the source commit into a temporary Git repository. Do not clone full history. Derive repository_url from sourceUrl, including repository renames. If sourceCommit is absent, resolve the sourceUrl ref once. Set source_commit to that actual commit. Run these commands in one shell call:
>
> ```sh
> git init --quiet "$temporary_dir"
> git -C "$temporary_dir" fetch --quiet --depth=1 "$repository_url" "$source_commit"
> git -C "$temporary_dir" checkout --quiet --detach FETCH_HEAD
> ```
>
> Read applicable licence declarations and notices at that commit. If copying is not permitted, report the restriction and stop.
> 4. Inspect source entries together, then copy the directory containing skillPath into `./skills/adding-a-provider-api-feature`. Use the user's path if selected. Keep the original SKILL.md, relative links, scripts, binary assets, and executable modes. Exclude .git metadata. Reject symlinks and paths outside the Skill directory. After checking entries, use cp -a where available. A regular source directory needs no custom copy script. Do not save this page wrapper as SKILL.md.
> Preserve author credit, notices, and applicable licence files from repository or parent directories. Add PROVENANCE.md with the Skill page, source URL, actual commit, original path, and licence. Retain any existing PROVENANCE.md and record new provenance separately. Batch source inspection, copying, and provenance work where practical.
> 5. In the project root, run `skilld install ./skills/adding-a-provider-api-feature --mode copy --plain`. If skilld is unavailable, use `npx skilld install ./skills/adding-a-provider-api-feature --mode copy --plain`. This known command needs no help lookup. Install does not support --json. Use detected Agent targets, or add --agent for the targets the user selected. Install the local path, never the upstream selector. If installation fails, preserve the local copy and report the exact failure.
> 6. Confirm the local lockfile source and installed Agent copies once. Report the local path, actual commit, and Agent targets. After edits, reinstall the same local path. Upstream updates must not replace it. Do not publish or push unless the user asks.

# Adding a provider API feature

Use this when exposing a **new provider API capability** through Pydantic AI — prompt caching, strict/structured tool calling, thinking/reasoning effort, service tier, safety settings, logprobs, cache breakpoints, and the like. The output is a change that is *consistent with how sibling providers already expose the same concept*, defaults deliberately, and gates support with a capability flag.

Not for: adding a new model id (that's `add-new-model`), a bug fix, or a refactor.

## The one rule that prevents the most rework

**Before designing anything, find the existing cross-provider abstraction that governs this capability and let its shape decide the API.** Most "how should I expose this?" questions are already answered by an abstraction the codebase has — reaching for a new provider-specific knob when one exists is the single most common thing maintainers reject. When a feature routes through an existing abstraction, the abstraction's shape pre-decides the API surface, the opt-out, and often the default.

The tell that you skipped this: you find yourself listing 2-3 "options" for how a user controls the feature. If one of those options duplicates an existing cross-provider control, it isn't a real option — the existing abstraction wins.

## Step 0 — Enumerate sibling precedent

For the capability you're adding, list **how every provider that already has an analog exposes it**, and name the governing existing abstraction. It is one of:

- a **per-tool flag** — `ToolDefinition.strict: bool | None` (`tools.py`), resolved in `models/__init__.py::_customize_tool_def`;
- a **shared `ModelSettings` field** — `thinking`, `service_tier` (`settings.py`), each with per-model resolvers mapping to native concepts;
- a **provider-prefixed `{Provider}ModelSettings` field** — `anthropic_cache`, `openai_prompt_cache_key`, `groq_reasoning_effort`;
- a **message-stream marker** — `CachePoint` in `UserPromptPart.content` (`messages.py`);
- a **`ModelProfile` capability flag** — `openai_supports_strict_tool_definition`, `bedrock_supports_prompt_caching`.

Read the actual sibling implementations and any review threads on the PRs that added them (`gh pr view <n> --comments`). Contributors who skipped this got redirected: raw `google_tool_config` → use `strict` ([#5366](https://github.com/pydantic/pydantic-ai/issues/5366#issuecomment-4909047000)); string-prefix model detection → use a profile flag ([#4604](https://github.com/pydantic/pydantic-ai/pull/4604#discussion_r2967152134)). **Only open a design fork if no existing abstraction covers the capability.**

## Step 1 — Pick the API shape

1. **An existing cross-provider abstraction covers it → reuse it.** Add a provider mapping (a `_translate_*` resolver, a `JsonSchemaTransformer` subclass, a `CachePoint` translation). Do **not** add a provider-prefixed knob for something the shared abstraction already expresses.
2. **No shared abstraction, but ≥3 providers now have the concept → promote to a shared `ModelSettings` field** with a deliberately narrow common vocabulary and per-model resolvers, keeping per-provider fields underneath as precedence-winning escape hatches (the `service_tier` promotion, [#4926](https://github.com/pydantic/pydantic-ai/pull/4926#discussion_r3028738692)). Don't delete provider knobs; deprecate only genuinely-misnamed ones with a `TODO(v3)`.
3. **The provider's native values can't be expressed by the shared enum** (family-disjoint value sets, platform-only request shaping) → a `{provider}_*` knob is justified, and it **coexists with and outranks** the unified field (`groq_reasoning_effort`, [#5797](https://github.com/pydantic/pydantic-ai/pull/5797)).
4. **Choose the locus by "can the user naturally point at it?":** a boundary *inside* the message stream → a marker (`CachePoint`); a structural region (system prompt, tool defs, whole-request setting) → a setting ([#3363](https://github.com/pydantic/pydantic-ai/pull/3363#discussion_r2505366576)).
5. **Wire it into every request path** the provider has (chat *and* responses APIs); put a setting shared across a provider's APIs on the **base** settings class ([#3678](https://github.com/pydantic/pydantic-ai/pull/3678#discussion_r2602911025)).
6. **Type it.** Reuse the provider SDK's own types where they exist; type knobs as `Literal`, never `extra_body` or untyped `**kwargs` (`models/AGENTS.md`; *"kwargs are a big no no… I'd rather be repetitive but type safe"* — DouweM, [#3457](https://github.com/pydantic/pydantic-ai/pull/3457#discussion_r2547862149)).

## Step 2 — Default: default-on (silent) vs opt-in

Default the feature **on** only when enabling it **cannot change observable behavior and cannot cost the user** — a pure, backward-compatible improvement (caching only *lowers* cost; a validation mode that needs no schema rewrite and can't reject a previously-valid request). A backward-compatible, purely-better default is welcome and needs no opt-in.

Keep it **opt-in** when it:

- changes observable output or wire behavior;
- is a **preview** feature (provider may change semantics);
- can *raise* cost — and choose the default *value* so a shared field never silently upgrades anyone to a pricier tier (`service_tier` `'auto'` vs `'default'`, [#4926](https://github.com/pydantic/pydantic-ai/pull/4926));
- can hit provider limits at scale (auto-promoting strict silently broke agents with >20 tools; reverted in [#5580](https://github.com/pydantic/pydantic-ai/pull/5580));
- applies lossy schema rewrites (OpenAI/Anthropic strict transforms).

When the default isn't obvious, **decide it with a live probe, not an opinion** ([#5897](https://github.com/pydantic/pydantic-ai/pull/5897) flipped a default on 0/5 → 5/5 recovery). Beware "automatic" language — verify whether it means a Pydantic AI default or just provider-side management of an opt-in feature.

## Step 3 — Capability gating

Detect support with a provider-prefixed **`ModelProfile` flag**. Where the feature is used, read only the flag or the adapter's own declaration — never an inline `isinstance`, model-name or `base_url` check (`profiles/AGENTS.md`). Gate at the layer(s) that actually vary:

- **model/family** → a flag in the family's profile function (`google_supports_strict_tool_definition` in `profiles/google.py`);
- **one provider's API** → a flag set in that provider's `Provider.model_profile()` (`bedrock_supports_prompt_caching` in `providers/bedrock.py`);
- **transport** (the Bedrock, Vertex and Foundry client classes of one SDK, or a client's own mode such as `genai.Client.vertexai`) → a flag set in the model's `_instance_profile_defaults()`, or in its `profile` property when the client can't send the feature at all. `Provider.model_profile()` is static and sees only the model name (`models/AGENTS.md`, the client-class bullet);
- **per-schema** → the `JsonSchemaTransformer.is_strict_compatible` signal (default conservative unless the mode needs no rewrites);
- **SDK version** → probe the SDK shape and degrade with a `UserWarning` ([#5580](https://github.com/pydantic/pydantic-ai/pull/5580), botocore strict param);
- **unknowable client-side** → defer to the runtime API error.

**Unsupported → silently ignore** the setting (best-effort so as many requests as possible succeed), documented in the docstring — never hard-error. Conflicting user settings → `UserWarning`, not `UserError`. Profiles are layered: developer-keyed base + thin provider overlay resolved per family, with the provider overlay layering **last** ([#5934](https://github.com/pydantic/pydantic-ai/issues/5934), [#6231](https://github.com/pydantic/pydantic-ai/pull/6231)).

## Step 3b — Where the wire mapping goes

Read `agent_docs/provider-adapters.md` before you edit an adapter's mapper. Place the feature by what it adds:

- **a new part type** → one arm in each message kind's dispatch, calling the part's own method on the model class. When most APIs can't render the part, lower it in `Model.prepare_messages` instead; an adapter that doesn't render it raises `_unprepared_part_error(part)` in its one arm.
- **a new native tool kind** → one entry in the adapter's `kind`-to-wire-name table. Add a branch only where the wire shape differs.
- **a new response wire item** → one function, called from both `_process_response` and the stream.
- **a new flag or setting that changes how an existing part maps** → read it inside that part's method. Where the adapter still maps that part inline in its loop, read it where the part is mapped today, and leave extracting the method to its own structural PR.

## Step 4 — Tests

Pyright does not flag a `Literal`-keyed dict that misses a key; the lookup raises `KeyError` only at request time. Add every new mapping keyed by `ThinkingEffort` to `test_every_effort_level_resolves` in `tests/test_thinking.py`, which runs each level through each mapping. When you add a value to another shared `Literal`, write the same kind of test for it.

Live-recorded **wire-contract cassette** asserting the exact outbound body (mode/field is sent for supported models, absent/ignored for unsupported), plus a test exercising the new setting. Prefer case-based parametrized VCR over mocks; a unit test is still right for asserting an internal request shape a cassette matcher wouldn't catch. This is also where a review bot's "this will fail" claim gets refuted with a recorded cassette.

## Step 5 — Docs & skills

The new public symbol's docstring lists **which providers support it** and how each interprets the value. Add/refresh the `docs/**.md` section and any sibling docstrings that now under-claim ("only OpenAI" → the full provider list). Describe the mechanism only as far as the provider documents it — don't assert a mechanism a provider's docs leave unstated. Update the relevant agent skill.

`ModelSettings`' `Supported by:` lists are enforced: `tests/models/test_model_settings_support.py` probes each model class's outgoing request and asserts every list names exactly the classes that send the field. Forwarding a new setting means editing its list, and a new `Model` class means adding a `Case` there. `tool_choice` and `thinking` are exempt via `HAND_MAINTAINED` and stay hand-maintained.

## Recurring maintainer principles (quoted)

1. **Reuse the cross-provider abstraction over a provider knob** — *"this maps to what the Anthropic and OpenAI APIs call `strict`… already represented on `ToolDefinition`… a more complete, consistent, 'doesn't require the user to do something special' implementation would be to automatically use this mode."* ([#5366](https://github.com/pydantic/pydantic-ai/issues/5366#issuecomment-4909047000))
2. **Promote to a shared setting once several providers have the concept; keep per-provider overrides underneath.** ([#4926](https://github.com/pydantic/pydantic-ai/pull/4926#discussion_r3028738692))
3. **Best-effort — silently ignore unsupported settings, don't error.** *"we typically do a 'best effort' so that as many requests as possible succeed."* ([#3438](https://github.com/pydantic/pydantic-ai/pull/3438#issuecomment-3553448605))
4. **Hide provider complexity** — the feature should be useful to people who don't want to become experts in that provider's limitations.
5. **Capability facts belong on `ModelProfile` flags, layered base + overlay.** ([#5934](https://github.com/pydantic/pydantic-ai/issues/5934#issuecomment-4858653237))
6. **Type safety over repetition; no untyped kwargs.** ([#3457](https://github.com/pydantic/pydantic-ai/pull/3457#discussion_r2547862149))
7. **Put shared fields on the base settings class; cover every API surface.** ([#3678](https://github.com/pydantic/pydantic-ai/pull/3678#discussion_r2602911025))
8. **Verify against the real API; defer validation to runtime.** *"If they're allowed by the SDK types and we can try it out and it doesn't fail, I'm fine with it."* ([#3678](https://github.com/pydantic/pydantic-ai/pull/3678#discussion_r2604746387))
9. **Name the eventual unification even when deferring it** — a `{provider}_*` knob today can note the future `Caching`/`Thinking`-style capability it should fold into. ([#4604](https://github.com/pydantic/pydantic-ai/pull/4604#discussion_r3277672350))

## Precedent map

| Capability | Reused abstraction | Default | Gating | PR |
|---|---|---|---|---|
| service tier (cross-provider) | promoted to `ModelSettings.service_tier` | opt-in, never silent upgrade | map-and-drop | [#4926](https://github.com/pydantic/pydantic-ai/pull/4926) |
| strict — OpenAI (origin) | `ToolDefinition.strict` | auto-promote per compatible schema | per-schema `is_strict_compatible` | [#1304](https://github.com/pydantic/pydantic-ai/pull/1304) |
| strict — Anthropic | reused `strict` | conservative opt-in | transformer + profile flag | [#3457](https://github.com/pydantic/pydantic-ai/pull/3457) |
| strict — Bedrock (+ fix) | reused `strict` | opt-in (auto-promote reverted) | transformer + profile + SDK probe | [#4237](https://github.com/pydantic/pydantic-ai/pull/4237), [#5580](https://github.com/pydantic/pydantic-ai/pull/5580) |
| strict — Gemini `VALIDATED` | reused `strict`; rejected raw `google_tool_config` | **default-on** (mode needs no schema rewrite) | profile flag; `is_strict_compatible = True` | [#6353](https://github.com/pydantic/pydantic-ai/pull/6353) |
| thinking (cross-provider) | `ModelSettings.thinking` + per-provider maps | opt-in, graceful degradation | `supports_thinking` flags | [#4640](https://github.com/pydantic/pydantic-ai/pull/4640) |
| reasoning effort — Groq | provider knob coexists with `thinking`, outranks it | opt-in | per-family profile flag | [#5797](https://github.com/pydantic/pydantic-ai/pull/5797), [#6231](https://github.com/pydantic/pydantic-ai/pull/6231) |
| prompt caching — Anthropic → Bedrock/OpenRouter | `CachePoint` marker + settings; breakpoint limits through `models/_prompt_cache.py` | opt-in | `{provider}_supports_prompt_caching` | [#3363](https://github.com/pydantic/pydantic-ai/pull/3363), [#3438](https://github.com/pydantic/pydantic-ai/pull/3438), [#4604](https://github.com/pydantic/pydantic-ai/pull/4604) |
