Index of every page on this site: llms.txt
All skills
vercel avatar

/ai-gateway

@250769d official
by vercelvercel/vercel-plugin340 stars
82
Run it yourself

Vercel AI Gateway guidance for setup, model discovery, authentication, routing, fallbacks, virtual models, evaluation models, BYOK, budgets, spend reporting, observability, compatible APIs, and coding-agent configuration. Use when adding AI Gateway to an app, migrating provider calls, choosing models or providers, centralizing model configuration, evaluating application state, debugging gateway requests, or running `vercel ai-gateway` commands.

  • 8 files
  • 67.9 KB
  • Updated 2 weeks ago
  • GitHub
Use this Skill: https://skilld.dev/gh/vercel/vercel-plugin/ai-gateway

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesevaluation.md

≈1.8k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Decision models

Use decision models (previously called evaluation models) to assess shared application state against typed questions. They return structured boolean probabilities, choices, or scores instead of free-form text.

Choose the evaluation surface

Existing project Use
JavaScript or TypeScript using AI SDK 7 experimental_decide from ai 7.0.128+; earlier AI SDK 7 releases use the now-deprecated experimental_evaluate alias
New non-AI-SDK client, OpenAI SDK, or OpenAI Decisions API client OpenAI-compatible POST /v1/decisions with base URL https://ai-gateway.vercel.sh/v1
Existing /v1/evaluate client Deprecated POST /v1/evaluate; it keeps working, but migrate to /v1/decisions
Existing TypeSafe client Keep @typesafe-ai/sdk and change its API key and baseURL

Decisions are not supported through Chat Completions, Responses, or the Anthropic-compatible or Cohere-compatible endpoints. Use one of the surfaces above instead.

Choose a model from the current Decision model list. Do not assume a text-generation model supports decisions. Use the Decision quickstart for the AI SDK first-run workflow, the Decision modality guide for AI SDK and HTTP request shapes, the OpenAI-compatible Decisions API guide for OpenAI's request shape, and the TypeSafe API guide when migrating a TypeSafe client.

AI SDK

Load the ai-sdk skill and read the installed ai package docs before writing code.

import { experimental_decide as decide } from 'ai';

const result = await decide({
  model: 'typesafe-ai/jev',
  state: 'The support agent issued a full refund to the customer.',
  questions: {
    refunded: {
      type: 'boolean',
      instructions: 'Was a refund issued?',
    },
  },
});

console.log(result.answers);

When using an explicit Gateway provider instance, use gateway.decisionModel(<model-id>) (@ai-sdk/gateway 4.0.104+, released with ai 7.0.128); earlier releases use gateway.evaluationModel(), which remains as a deprecated alias. A plain decision model string routes through AI Gateway without installing @ai-sdk/gateway.

HTTP API

For a client that does not use AI SDK, post OpenAI's Decisions shape to POST https://ai-gateway.vercel.sh/v1/decisions, or point the OpenAI SDK at https://ai-gateway.vercel.sh/v1:

curl https://ai-gateway.vercel.sh/v1/decisions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "typesafe-ai/jev",
    "input": "The support agent issued a full refund.",
    "questions": [
      { "type": "predicate", "name": "refunded", "instructions": "Was a refund issued?" }
    ]
  }'

The response lists answers in question order, with model, usage, and provider_metadata.gateway routing and cost. The endpoint accepts AI Gateway controls under providerOptions.gateway.

POST /v1/evaluate is deprecated but keeps working for existing integrations, which send Gateway's evaluation shape:

curl https://ai-gateway.vercel.sh/v1/evaluate \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "typesafe-ai/jev",
    "state": "The support agent issued a full refund.",
    "questions": {
      "refunded": { "type": "boolean", "instructions": "Was a refund issued?" }
    }
  }'

The response contains model, answers, and camel-cased usage. The endpoint accepts AI Gateway controls under providerOptions.gateway, including zeroDataRetention and only. Dashboard-configured BYOK credentials are used automatically.

TypeSafe compatibility

For an existing TypeSafe client, keep its request and response shapes and change only the credential and base URL:

import { TypeSafeClient } from '@typesafe-ai/sdk';

const client = new TypeSafeClient({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/typesafe',
});

The compatibility surface provides:

  • POST /typesafe/v1/systemone for evaluation;
  • GET /typesafe/v1/models for the evaluation model list; and
  • TypeSafe's noul, choice, score, confidence, legend, snake-case usage, and error shapes.

The TypeSafe-compatible request body also accepts AI Gateway controls under providerOptions.gateway. Use the OpenAI-compatible /v1/decisions endpoint for new HTTP integrations; use /typesafe when preserving an existing TypeSafe client is the goal.

On any decision surface, use Decision Fallbacks to rerun a successful but uncertain decision with another model: a conditional entry in providerOptions.gateway.models, such as a confidenceBelow threshold on a Choice or Score question. A triggered fallback bills both stages.

Question and state rules

  • Gateway's AI SDK and /v1/evaluate surfaces use boolean; the TypeSafe-compatible surface uses noul and translates it internally.
  • The OpenAI-compatible /v1/decisions surface uses OpenAI's shapes instead: input plus a questions array of predicate, choice (choices), and score (levels) entries with optional name. The rules below describe the other surfaces.
  • boolean or noul returns a probability from 0 to 1. Optional criteria can define the true and false cases.
  • choice selects one key from a named criteria record and returns probabilities for every option.
  • score uses an ordered criteria array of at least two labels and returns an interpolated score plus rung probabilities.
  • One request can ask multiple questions of different types against the same state.
  • Gateway state can be a string, object, or array. The TypeSafe contract also permits null and preserves its broader optional instruction and criteria shapes.
  • In AI SDK ai 7.0.133+, an array passed directly as state is an ordered list of parts ({ type: 'text', text }, { type: 'json', value }), not JSON data; Gateway rejects file parts. Wrap a JSON array in an object (state: { messages }) or a json part (state: [{ type: 'json', value: messages }]).

Use stable question keys because each key in questions becomes the corresponding key in answers. Treat probabilities as model outputs, not certainty; define explicit criteria and validate thresholds against representative data before automating consequential actions.

Usage and verification

Evaluation responses include token usage and are billed at the selected model's rates. Some evaluation models price input tokens only, so check the live model page instead of assuming text-model pricing.

After implementation:

  1. Run the formatter, type checker, and focused tests.
  2. When authorized, make one live evaluation through the selected surface and inspect its answers and usage.
  3. If using TypeSafe compatibility, verify the response retains TypeSafe naming and required confidence or legend fields.
  4. Confirm the request, API format, routing, and cost in AI Gateway Logs.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 250769d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 weeks ago
Other metadata
summary
Set up and operate Vercel AI Gateway with current models, virtual models, evaluation, authentication, routing, spend controls, and verification.
metadata
{
  "priority": 7,
  "docs": [
    "https://vercel.com/docs/ai-gateway",
    "https://vercel.com/docs/ai-gateway/getting-started",
    "https://vercel.com/docs/ai-gateway/models-and-providers/virtual-models",
    "https://vercel.com/docs/ai-gateway/modalities/decision",
    "https://vercel.com/docs/ai-gateway/sdks-and-apis/typesafe",
    "https://ai-sdk.dev/providers/ai-sdk-providers/ai-gateway"
  ],
  "sitemap": "https://vercel.com/docs/sitemap.md",
  "pathPatterns": [],
  "importPatterns": [
    "ai",
    "@ai-sdk/gateway"
  ],
  "bashPatterns": [
    "\\bvercel\\s+ai-gateway\\b",
    "\\bvercel\\s+env\\s+pull\\b",
    "\\bnpm\\s+(install|i|add)\\s+[^\\n]*@ai-sdk/gateway\\b",
    "\\bpnpm\\s+(install|i|add)\\s+[^\\n]*@ai-sdk/gateway\\b",
    "\\bbun\\s+(install|i|add)\\s+[^\\n]*@ai-sdk/gateway\\b",
    "\\byarn\\s+add\\s+[^\\n]*@ai-sdk/gateway\\b"
  ],
  "promptSignals": {
    "phrases": [
      "ai gateway",
      "vercel ai gateway",
      "ai-gateway",
      "ai-gateway.vercel.sh",
      "virtual model",
      "evaluation model",
      "vmc/",
      "typesafe api",
      "typesafe compat"
    ],
    "allOf": [
      [
        "model",
        "routing"
      ],
      [
        "provider",
        "failover"
      ],
      [
        "gateway",
        "budget"
      ],
      [
        "gateway",
        "logs"
      ],
      [
        "gateway",
        "oidc"
      ],
      [
        "coding",
        "gateway"
      ]
    ],
    "anyOf": [
      "provider ordering",
      "model fallback",
      "byok",
      "spend tracking",
      "gateway key",
      "credit balance",
      "safety identifier",
      "reasoning effort",
      "tool calling",
      "structured outputs",
      "experimental_evaluate",
      "central model configuration",
      "systemone",
      "v1/evaluate"
    ],
    "noneOf": [
      "cloudflare ai gateway",
      "aws api gateway"
    ],
    "minScore": 6
  }
}
validate
[
  {
    "pattern": "\\bclaude-(sonnet|opus|haiku)-\\d+-\\d+\\b",
    "message": "Claude model version uses a hyphen where the AI Gateway slug uses a dot. Fetch /v1/models and use the returned provider/model ID.",
    "severity": "error"
  },
  {
    "pattern": "gateway\\(['\"][^'\"/]+['\"]\\)",
    "message": "AI Gateway model string is missing its provider prefix. Fetch /v1/models and use a provider/model ID.",
    "severity": "error"
  },
  {
    "pattern": "(OPENAI_API_KEY|ANTHROPIC_API_KEY|GOOGLE_API_KEY)",
    "message": "Provider key detected. AI Gateway request authentication uses AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN; provider keys belong only in an intentional BYOK configuration.",
    "severity": "recommended",
    "skipIfFileContains": "[Bb][Yy][Oo][Kk]|providerOptions\\s*:\\s*\\{[^}]*gateway"
  },
  {
    "pattern": "gateway\\s*:\\s*\\{[^}]*cacheControl",
    "message": "AI Gateway does not cache whole responses through cacheControl. Use caching: 'auto' for provider prompt caching and verify the current caching docs.",
    "severity": "error"
  },
  {
    "pattern": "ANTHROPIC_BASE_URL\\s*=\\s*[\"']?https://ai-gateway\\.vercel\\.sh",
    "message": "Claude Code through AI Gateway needs ANTHROPIC_API_KEY set to an empty value and the gateway key in ANTHROPIC_AUTH_TOKEN. A non-empty ANTHROPIC_API_KEY is used instead of the gateway token.",
    "severity": "recommended",
    "skipIfFileContains": "ANTHROPIC_AUTH_TOKEN"
  }
]
chainTo
[
  {
    "pattern": "from\\s+['\"]ai['\"]|require\\(['\"]ai['\"]\\)|\\b(generateText|streamText|ToolLoopAgent)\\b",
    "targetSkill": "ai-sdk",
    "message": "AI SDK code detected. Load the AI SDK skill and read the installed package docs before writing or changing SDK code."
  }
]
retrieval
{
  "aliases": [
    "model router",
    "ai proxy",
    "provider failover",
    "llm gateway",
    "gateway credits",
    "virtual model",
    "evaluation model",
    "typesafe compatibility"
  ],
  "intents": [
    "add Vercel AI Gateway to an application",
    "route AI models across providers",
    "configure provider or model fallbacks",
    "authenticate AI Gateway requests",
    "track AI model costs and set budgets",
    "debug AI Gateway requests and routing",
    "connect coding agents to AI Gateway",
    "give coding agents centrally managed model and provider configuration",
    "create or update an AI Gateway virtual model",
    "evaluate application state with typed questions",
    "migrate an existing TypeSafe evaluation client to AI Gateway",
    "find a model by modality, capability, price, or data retention",
    "check AI Gateway credit balance or generation cost",
    "configure reasoning or extended thinking across providers and API formats",
    "add tool calling or function calling across API formats",
    "get structured JSON output matching a schema",
    "send images or PDFs to a model"
  ],
  "entities": [
    "AI Gateway",
    "AI Gateway Credits",
    "providerOptions.gateway",
    "AI_GATEWAY_API_KEY",
    "VERCEL_OIDC_TOKEN",
    "model routing",
    "provider failover",
    "BYOK",
    "spend reporting",
    "safetyIdentifier",
    "Usage & Billing API",
    "Virtual Models",
    "vmc/<slug>",
    "experimental_evaluate",
    "POST /v1/evaluate",
    "TypeSafe API",
    "/typesafe/v1/systemone"
  ]
}

README badge

README badge for vercel/vercel-plugin/ai-gateway