# Vercel AI Gateway tutorial

> Use Vercel AI Gateway with the AI SDK and OpenAI client, then add model routing, fallbacks, caching, budgets, BYOK, and observability.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-09-08 | Updated: 2026-08-21 | Topics: [AI](https://flaviocopes.com/tags/ai/) | Canonical: https://flaviocopes.com/vercel-ai-gateway/

[Vercel AI Gateway](https://vercel.com/ai-gateway) gives you one API for models from OpenAI, Anthropic, Google, xAI, and many other providers.

Your application calls Vercel. Vercel chooses an upstream provider, sends the request, and returns the response.

This means you can change models without installing another provider SDK or creating another integration. You also get usage logs, spend tracking, routing, and fallbacks in one place.

In this tutorial we'll make a request, stream a response, migrate an OpenAI client, and configure the production features that make a gateway useful.

## What is an AI gateway?

Without a gateway, your application talks to each provider directly:

```text
your app -> OpenAI
your app -> Anthropic
your app -> Google
```

Every provider has its own key, billing account, API details, and logs.

With Vercel AI Gateway, the shape changes:

```text
your app -> Vercel AI Gateway -> model provider
```

Your application uses one AI Gateway key. Model names follow the `creator/model` format:

```text
openai/gpt-5.4-mini
anthropic/claude-sonnet-4.6
google/gemini-3.1-flash-lite
```

Change that string, and you change the model.

Vercel AI Gateway is not limited to applications hosted on Vercel. You can call it from a VPS, a Cloudflare Worker, another serverless platform, or a script on your computer.

Keep the API key on the server. Never call the gateway directly from browser JavaScript with a secret key.

## Why use a gateway?

The main advantage is not shorter code. It is having one control point for every model call.

Vercel AI Gateway gives you:

- one key for many model providers
- a unified API
- provider routing and automatic retries
- model fallbacks
- usage, token, latency, and spend logs
- per-key spending budgets
- optional Bring Your Own Key (BYOK)
- OpenAI and Anthropic-compatible endpoints

There is a tradeoff. You add another service between your application and the model.

For a small script that always calls one provider, I would call that provider directly. For a production application using several models, the gateway becomes useful quickly.

## How pricing works

AI Gateway charges the upstream provider's list price with no token markup. Check the current price in the [model catalog](https://vercel.com/ai-gateway/models) before choosing a model.

Every Vercel team gets a monthly free AI Gateway credit. At the time of writing, it is $5. The free credit starts with the first request.

Once you purchase credits, the account moves to pay-as-you-go and the monthly free credit stops. Credits are prepaid and requests consume the balance.

BYOK also has no gateway markup. You pay the provider through your own account.

Pricing changes. Use the catalog as the source of truth instead of copying model prices into your code.

## Create an API key

Open your Vercel dashboard and go to **AI Gateway → API Keys**.

Click **Create key**, give it a clear name, and copy it immediately. Vercel does not show the value again.

For local development, create a `.env` file:

```text
AI_GATEWAY_API_KEY=your_ai_gateway_api_key
```

Add the file to `.gitignore`:

```text
.env
```

Create separate keys for separate applications. If one key leaks, you can revoke it without breaking every project.

## Set a budget before the first request

An AI key without a budget can spend the team's entire credit balance.

When you create the key, enable its [budget](https://vercel.com/docs/ai-gateway/observability-and-spend/api-key-budgets). Choose a dollar limit and a daily, weekly, monthly, or non-resetting period.

You can also create a budgeted key with the Vercel CLI:

```bash
vercel ai-gateway api-keys create \
  --name tutorial \
  --budget 5 \
  --refresh-period monthly
```

The minimum budget is $1.

The budget is a **soft cap**. AI Gateway checks it before each request. A request that starts below the limit can finish above it.

You still need application-level rate limits. A gateway budget protects the account, while a rate limit controls how often each user can call your API.

## Make the first request with the AI SDK

The [AI SDK](https://ai-sdk.dev) is the shortest path from TypeScript to AI Gateway.

Create a project:

```bash
mkdir vercel-ai-gateway-demo
cd vercel-ai-gateway-demo
npm init -y
```

Install the AI SDK, `dotenv`, and `tsx`:

```bash
npm install ai dotenv tsx
```

Create `index.ts`:

```ts
import 'dotenv/config'
import { generateText } from 'ai'

const { text, usage } = await generateText({
  model: 'openai/gpt-5.4-mini',
  prompt: 'Explain what an API gateway does in 3 sentences.',
})

console.log(text)
console.log(usage)
```

Run it:

```bash
npx tsx index.ts
```

The AI SDK sees the model string, uses AI Gateway as the provider, and reads `AI_GATEWAY_API_KEY` automatically.

The `usage` object tells you how many input and output tokens the request used.

## Stream a response

For chat and longer generations, show text as it arrives.

Replace `generateText()` with `streamText()`:

```ts
import 'dotenv/config'
import { streamText } from 'ai'

const result = streamText({
  model: 'openai/gpt-5.4-mini',
  prompt: 'Write a short story about a robot learning Git.',
})

for await (const textPart of result.textStream) {
  process.stdout.write(textPart)
}

console.log()
console.log('Usage:', await result.usage)
```

The model starts producing text before the full response is ready.

Streaming improves perceived speed. It does not reduce the number of tokens or their cost.

## Use the OpenAI client

AI Gateway implements an OpenAI-compatible Chat Completions API.

This is useful when an application already uses the OpenAI client. Change the API key, base URL, and model:

```bash
npm install openai
```

Then:

```ts
import 'dotenv/config'
import OpenAI from 'openai'

const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
})

const response = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-4.6',
  messages: [
    {
      role: 'user',
      content: 'Explain immutable deployments in 3 sentences.',
    },
  ],
})

console.log(response.choices[0].message.content)
```

Notice what happened. We are using the OpenAI client to call an Anthropic model.

The client speaks the OpenAI API format. AI Gateway translates and routes the request.

You can also call `https://ai-gateway.vercel.sh/v1/chat/completions` with `fetch()` or cURL. You do not need an SDK.

## Find available models

The public model endpoint does not require authentication:

```bash
curl https://ai-gateway.vercel.sh/v1/models
```

It returns model IDs, capabilities, context windows, and pricing.

In code:

```ts
const response = await fetch('https://ai-gateway.vercel.sh/v1/models')
const { data: models } = await response.json()

const textModels = models.filter((model) => model.type === 'language')

console.log(textModels.map((model) => model.id))
```

Do not let an untrusted browser user select any model in your catalog. An expensive model can burn through a budget very quickly.

Keep an allowlist of models your application supports.

## Control provider routing

A model and a provider are different things.

Anthropic creates Claude, but the same Claude model might be hosted by Anthropic, Amazon Bedrock, or Google Vertex AI.

By default, AI Gateway chooses between available providers using recent uptime and latency.

You can control the order with `providerOptions.gateway`:

```ts
const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4.6',
  prompt: 'Explain provider routing in one paragraph.',
  providerOptions: {
    gateway: {
      order: ['anthropic', 'bedrock'],
      only: ['anthropic', 'bedrock'],
    },
  },
})
```

`order` sets the preference. `only` prevents the gateway from using providers outside that list.

Only constrain routing when you have a reason. Compliance, data location, existing provider agreements, and predictable behavior are good reasons.

## Add model fallbacks

Provider routing can try another host for the same model.

A **model fallback** goes further. It tries a different model when the primary model fails or is unavailable.

```ts
const result = streamText({
  model: 'openai/gpt-5.4-mini',
  prompt: 'Write a short description for a coffee shop.',
  providerOptions: {
    gateway: {
      models: [
        'google/gemini-3.1-flash-lite',
        'anthropic/claude-sonnet-4.6',
      ],
    },
  },
})
```

AI Gateway tries the primary model first, then each fallback in order.

Choose fallbacks with compatible capabilities. A text-only model cannot replace a vision model when your prompt contains an image.

Also expect output differences. A fallback keeps the request running, but it does not guarantee identical wording or JSON behavior.

Validate structured output before your application trusts it.

## Use automatic prompt caching

Long system prompts and agent conversations repeat a lot of text.

Some providers cache prompt prefixes automatically. Others require explicit cache markers.

AI Gateway can handle this difference:

```ts
const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4.6',
  system: 'You are a patient JavaScript tutor.',
  prompt: 'Explain closures with a small example.',
  providerOptions: {
    gateway: {
      caching: 'auto',
    },
  },
})
```

Automatic caching adds markers for providers that need them. It leaves providers with implicit caching alone.

Caching can reduce cost and latency when requests share a stable prefix. It does not help much when every prompt is completely different.

## Bring your own provider keys

By default, Vercel pays the provider and deducts the cost from your AI Gateway credits.

With [**Bring Your Own Key**](https://vercel.com/docs/ai-gateway/authentication-and-byok/byok), AI Gateway uses credentials from your OpenAI, Anthropic, or other provider account.

Add provider credentials under **AI Gateway → Bring Your Own Key** in the Vercel dashboard.

The credentials are available across the Vercel team. Your application still uses its AI Gateway key, so provider keys do not need to live in the application.

Vercel tries BYOK credentials first. If they fail, AI Gateway may fall back to Vercel's system credentials to keep the request running.

This is important: the team still needs AI Gateway credits even when using BYOK.

Use BYOK when you have provider credits, negotiated pricing, or access to private provider features. If not, unified Vercel billing is easier.

## Use OIDC on Vercel deployments

An API key works everywhere, but it is a long-lived secret.

Applications deployed on Vercel can use an automatically generated [OIDC token](https://vercel.com/docs/ai-gateway/authentication-and-byok/oidc) instead. You do not need to store an AI Gateway API key in the project.

Link the local directory to a Vercel project:

```bash
vercel link
```

Pull the development environment:

```bash
vercel env pull
```

The AI SDK uses the OIDC token automatically.

Local OIDC tokens expire after 12 hours. Run `vercel env pull` again to refresh one.

For code running outside Vercel, keep using an API key.

## Read the request logs

After making a few requests, open **AI Gateway** in the Vercel dashboard.

The overview shows:

- requests by model
- time to first token
- input and output token counts
- spend
- requests grouped by project and API key

Open a generation to inspect its model, provider, latency, token usage, cost, and finish reason.

The AI SDK also returns a gateway generation ID in `providerMetadata`. Store that ID with your application's request ID when debugging production calls.

My advice is to check the dashboard before changing prompts blindly. First find out whether the problem is latency, a provider error, a token limit, a fallback, or the model's answer.

## Production checklist

Before exposing an AI feature to users:

1. Create one gateway key per application
2. Add a budget to every key
3. Keep keys out of browser code and Git
4. Rate-limit users before calling the model
5. Allow only the models your application needs
6. Add fallbacks only after testing their output
7. Validate structured responses
8. Log the gateway generation ID
9. Review spend and failed requests
10. Check provider data policies for sensitive prompts

The gateway protects the connection to model providers. It does not replace your application's authentication, authorization, rate limits, or input validation.

## When should you use Vercel AI Gateway?

Use it when you want to switch models, compare providers, centralize spend, or add fallbacks without maintaining several integrations.

Skip it when a small script calls one provider and you do not need shared observability or routing.

The simplest useful setup is one key, one inexpensive model, and one budget. Start there.

Add routing, BYOK, caching, and fallbacks when the application has a real reason for them.