Index of every page on this site: llms.txt
All skills
vercel avatar

/ai-gateway

@250769d official
by vercelvercel/vercel-plugin340 stars
82
Run it yourself

Vercel AI Gateway guidance for setup, model discovery, authentication, routing, fallbacks, virtual models, evaluation models, BYOK, budgets, spend reporting, observability, compatible APIs, and coding-agent configuration. Use when adding AI Gateway to an app, migrating provider calls, choosing models or providers, centralizing model configuration, evaluating application state, debugging gateway requests, or running `vercel ai-gateway` commands.

  • 8 files
  • 67.9 KB
  • Updated 2 weeks ago
  • GitHub
Use this Skill: https://skilld.dev/gh/vercel/vercel-plugin/ai-gateway

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesvirtual-models.md

≈1.2k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Virtual Models

A Virtual Model is a team-scoped vmc/<slug> that resolves to a base model plus reusable routing, behavior, compliance, observability, and provider-specific configuration. Use one stable model ID and update its server-side configuration without redeploying each caller.

Docs: https://vercel.com/docs/ai-gateway/models-and-providers/virtual-models

When to use one

Use a Virtual Model when:

  • multiple applications or agents should share one model policy;
  • a coding agent accepts a model ID but cannot send providerOptions;
  • provider order, restrictions, fallbacks, service tier, caching, timeouts, compliance, or tags must be centrally managed;
  • provider-specific options should apply to every request through the slug; or
  • the underlying model or routing should change without editing callers.

Keep per-request providerOptions when policy genuinely varies by request and the client can send them. A Virtual Model is configuration, not a second inference request.

Create and inspect

Use the dashboard or the current CLI. Check help before scripting flags:

vercel ai-gateway virtual-models --help
vercel ai-gateway virtual-models create
vercel ai-gateway virtual-models inspect coding-agent
vercel ai-gateway models list --virtual

The CLI supports create, list, inspect, update, delete, and restore (update needs Vercel CLI 64.4+ and delete 64.6+; older CLIs use edit and remove). The slug is immutable after creation. Changes apply to new requests within a few minutes.

The Virtual Model's Provider Options field is a provider-keyed JSON map, for example:

{
  "anthropic": {
    "effort": "high"
  }
}

Keys must be known provider slugs, and each nested option must be valid for that provider.

Do not confuse that field with Gateway routing controls. Provider order, only, model fallbacks, sorting, service tier, caching, timeouts, capabilities, compliance, and tags are separate Virtual Model settings. In an AI SDK request, those routing controls normally live under providerOptions.gateway.

Call the Virtual Model

Use the vmc/ prefix anywhere AI Gateway accepts a model ID:

import { streamText } from 'ai';

const result = streamText({
  model: 'vmc/coding-agent',
  prompt: 'Review this change.',
  providerOptions: {
    gateway: {
      user: 'user-123',
    },
  },
});

The request may still supply settings the Virtual Model leaves unset. Saved settings resolve as follows:

  • Provider order, only, fallbacks, fast mode, sort, service tier, caching, timeouts, inference region, required capabilities, and tags win when the Virtual Model sets them.
  • Provider-specific options merge option by option; the Virtual Model wins collisions and request values fill unset options.
  • Compliance settings only tighten. Neither side can turn off a restriction enabled by the other.
  • A setting left unset by the Virtual Model remains request-configurable.
  • A vmc/<slug> can appear in a request fallback list; its settings apply only to that fallback attempt.

Read the current docs before relying on @default or cleared settings; only selected fields support the tri-state behavior.

Coding-agent pattern

Coding agents frequently expose a model picker or model config but no way to send AI SDK providerOptions. In that case:

  1. Connect the agent with vercel ai-gateway setup.
  2. Create a Virtual Model containing the shared routing and provider configuration.
  3. Select or configure vmc/<slug> as the agent's model. Use the agent's current AI Gateway page for exact config syntax. If its generated shortlist omits team Virtual Models, add the ID manually where the agent permits custom models.
  4. Verify in AI Gateway Logs that the slug resolved to the expected base model and provider.

Do not promise that every agent discovers team Virtual Models automatically. Model discovery and custom-model support vary by agent. The Virtual Model solves server-side configuration; it does not change the client's model-picker capabilities.

Operational cautions

  • Team Owners and Members can create Virtual Models; only the creator or a team Owner can edit, archive, remove, or restore one. Any team member can call one.
  • Archived or removed slugs stop serving requests. Know the callers before changing lifecycle state.
  • Virtual Model tags replace request tags when configured, so avoid unintentionally dropping caller attribution.
  • Required capabilities replace the request's capability list rather than merging with it.
  • Repointing a slug can change price, latency, retention, and model behavior for every caller. Verify Logs and cost after edits.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No third-party reports yet.

Signed by skilld at 250769d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 weeks ago
Other metadata
summary
Set up and operate Vercel AI Gateway with current models, virtual models, evaluation, authentication, routing, spend controls, and verification.
metadata
{
  "priority": 7,
  "docs": [
    "https://vercel.com/docs/ai-gateway",
    "https://vercel.com/docs/ai-gateway/getting-started",
    "https://vercel.com/docs/ai-gateway/models-and-providers/virtual-models",
    "https://vercel.com/docs/ai-gateway/modalities/decision",
    "https://vercel.com/docs/ai-gateway/sdks-and-apis/typesafe",
    "https://ai-sdk.dev/providers/ai-sdk-providers/ai-gateway"
  ],
  "sitemap": "https://vercel.com/docs/sitemap.md",
  "pathPatterns": [],
  "importPatterns": [
    "ai",
    "@ai-sdk/gateway"
  ],
  "bashPatterns": [
    "\\bvercel\\s+ai-gateway\\b",
    "\\bvercel\\s+env\\s+pull\\b",
    "\\bnpm\\s+(install|i|add)\\s+[^\\n]*@ai-sdk/gateway\\b",
    "\\bpnpm\\s+(install|i|add)\\s+[^\\n]*@ai-sdk/gateway\\b",
    "\\bbun\\s+(install|i|add)\\s+[^\\n]*@ai-sdk/gateway\\b",
    "\\byarn\\s+add\\s+[^\\n]*@ai-sdk/gateway\\b"
  ],
  "promptSignals": {
    "phrases": [
      "ai gateway",
      "vercel ai gateway",
      "ai-gateway",
      "ai-gateway.vercel.sh",
      "virtual model",
      "evaluation model",
      "vmc/",
      "typesafe api",
      "typesafe compat"
    ],
    "allOf": [
      [
        "model",
        "routing"
      ],
      [
        "provider",
        "failover"
      ],
      [
        "gateway",
        "budget"
      ],
      [
        "gateway",
        "logs"
      ],
      [
        "gateway",
        "oidc"
      ],
      [
        "coding",
        "gateway"
      ]
    ],
    "anyOf": [
      "provider ordering",
      "model fallback",
      "byok",
      "spend tracking",
      "gateway key",
      "credit balance",
      "safety identifier",
      "reasoning effort",
      "tool calling",
      "structured outputs",
      "experimental_evaluate",
      "central model configuration",
      "systemone",
      "v1/evaluate"
    ],
    "noneOf": [
      "cloudflare ai gateway",
      "aws api gateway"
    ],
    "minScore": 6
  }
}
validate
[
  {
    "pattern": "\\bclaude-(sonnet|opus|haiku)-\\d+-\\d+\\b",
    "message": "Claude model version uses a hyphen where the AI Gateway slug uses a dot. Fetch /v1/models and use the returned provider/model ID.",
    "severity": "error"
  },
  {
    "pattern": "gateway\\(['\"][^'\"/]+['\"]\\)",
    "message": "AI Gateway model string is missing its provider prefix. Fetch /v1/models and use a provider/model ID.",
    "severity": "error"
  },
  {
    "pattern": "(OPENAI_API_KEY|ANTHROPIC_API_KEY|GOOGLE_API_KEY)",
    "message": "Provider key detected. AI Gateway request authentication uses AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN; provider keys belong only in an intentional BYOK configuration.",
    "severity": "recommended",
    "skipIfFileContains": "[Bb][Yy][Oo][Kk]|providerOptions\\s*:\\s*\\{[^}]*gateway"
  },
  {
    "pattern": "gateway\\s*:\\s*\\{[^}]*cacheControl",
    "message": "AI Gateway does not cache whole responses through cacheControl. Use caching: 'auto' for provider prompt caching and verify the current caching docs.",
    "severity": "error"
  },
  {
    "pattern": "ANTHROPIC_BASE_URL\\s*=\\s*[\"']?https://ai-gateway\\.vercel\\.sh",
    "message": "Claude Code through AI Gateway needs ANTHROPIC_API_KEY set to an empty value and the gateway key in ANTHROPIC_AUTH_TOKEN. A non-empty ANTHROPIC_API_KEY is used instead of the gateway token.",
    "severity": "recommended",
    "skipIfFileContains": "ANTHROPIC_AUTH_TOKEN"
  }
]
chainTo
[
  {
    "pattern": "from\\s+['\"]ai['\"]|require\\(['\"]ai['\"]\\)|\\b(generateText|streamText|ToolLoopAgent)\\b",
    "targetSkill": "ai-sdk",
    "message": "AI SDK code detected. Load the AI SDK skill and read the installed package docs before writing or changing SDK code."
  }
]
retrieval
{
  "aliases": [
    "model router",
    "ai proxy",
    "provider failover",
    "llm gateway",
    "gateway credits",
    "virtual model",
    "evaluation model",
    "typesafe compatibility"
  ],
  "intents": [
    "add Vercel AI Gateway to an application",
    "route AI models across providers",
    "configure provider or model fallbacks",
    "authenticate AI Gateway requests",
    "track AI model costs and set budgets",
    "debug AI Gateway requests and routing",
    "connect coding agents to AI Gateway",
    "give coding agents centrally managed model and provider configuration",
    "create or update an AI Gateway virtual model",
    "evaluate application state with typed questions",
    "migrate an existing TypeSafe evaluation client to AI Gateway",
    "find a model by modality, capability, price, or data retention",
    "check AI Gateway credit balance or generation cost",
    "configure reasoning or extended thinking across providers and API formats",
    "add tool calling or function calling across API formats",
    "get structured JSON output matching a schema",
    "send images or PDFs to a model"
  ],
  "entities": [
    "AI Gateway",
    "AI Gateway Credits",
    "providerOptions.gateway",
    "AI_GATEWAY_API_KEY",
    "VERCEL_OIDC_TOKEN",
    "model routing",
    "provider failover",
    "BYOK",
    "spend reporting",
    "safetyIdentifier",
    "Usage & Billing API",
    "Virtual Models",
    "vmc/<slug>",
    "experimental_evaluate",
    "POST /v1/evaluate",
    "TypeSafe API",
    "/typesafe/v1/systemone"
  ]
}

README badge

README badge for vercel/vercel-plugin/ai-gateway