Cloudflare Workers AI: run LLMs without API keys

By

Learn what Cloudflare Workers AI is, connect its AI binding to a Worker, and build a small text summarization API step by step.

~~~

You don’t need to rent a GPU to add an AI model to an application.

You don’t always need an OpenAI or Anthropic account either.

Cloudflare Workers AI runs models on Cloudflare’s infrastructure. A Worker can call those models through a binding, in the same way it talks to KV, D1, or R2.

In this tutorial we’ll understand what that means. Then we’ll build a small API that summarizes text.

What is Workers AI?

Workers AI is Cloudflare’s serverless AI inference platform.

Let’s unpack that sentence.

An AI model is a program trained to perform a task. Different models generate text, create images, transcribe audio, classify input, or turn text into embeddings.

Inference is what happens when we give a trained model some input and ask it for an output.

Serverless means Cloudflare manages the machines and GPUs that run the model. You send a request when you need an answer and pay for the work performed. You don’t keep a GPU server running yourself.

Workers AI combines those pieces. Cloudflare hosts a catalog of models and exposes them through its developer platform.

The basic mental model

A normal application might call an external AI provider over HTTP:

Worker -> external AI API -> model

Workers AI gives your Worker an AI binding:

Worker -> AI binding -> Workers AI model

In code, that binding appears as env.AI.

You select a model and call run():

const result = await env.AI.run(model, input)

There is no provider API key in your Worker. Cloudflare already knows which account owns the binding and adds the usage to that account.

What can Workers AI do?

Workers AI is larger than text chat.

Its model catalog includes models for:

  • text generation
  • text embeddings
  • image generation
  • image understanding
  • speech recognition
  • text-to-speech
  • translation
  • classification
  • reranking search results

Each model has its own input and output shape.

A text-generation model accepts a prompt or a list of messages. An embedding model accepts text and returns arrays of numbers. An image model may return binary image data.

The call is always env.AI.run(), but the data depends on the model.

Workers AI and AI Gateway are different

These two Cloudflare products are easy to mix up.

Workers AI runs models.

AI Gateway observes and controls model requests.

You can use Workers AI without AI Gateway. You can also put AI Gateway in front of Workers AI to add logs, caching, rate limits, retries, and routing.

AI Gateway can also sit in front of models from OpenAI, Anthropic, Google, and other providers.

Think of Workers AI as the engine and AI Gateway as the control point in front of it.

Choose a current model

Model names are not permanent.

Providers add, replace, and retire models. A name copied from an old tutorial may no longer work.

List the current catalog with Wrangler:

npx wrangler ai models list

You can also browse the Workers AI model catalog.

For this tutorial we’ll use:

@cf/meta/llama-3.2-3b-instruct

This is a small instruction-tuned text model. It is a good fit for short summaries and other focused tasks.

Always open the model page before using a model in production. Check its task, input schema, context window, price, and current availability.

Create a Worker project

Let’s build a small text summarization API.

Create a Worker project:

npm create cloudflare@2 workers-ai-summary
cd workers-ai-summary

This uses the latest 2.x release of the project generator. Choose a basic Hello World Worker and JavaScript when prompted.

If you already have a Worker project, use that instead.

Add the AI binding

Open wrangler.jsonc and add the ai property:

{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "name": "workers-ai-summary",
  "main": "src/index.js",
  "compatibility_date": "2026-08-15",
  "compatibility_flags": ["nodejs_compat"],
  "ai": {
    "binding": "AI",
    "remote": true
  }
}

The name AI is our choice. It becomes env.AI in the Worker.

AI inference always happens on Cloudflare’s GPUs and counts toward your usage, even during local development. The remote setting just makes that explicit in the configuration. Other bindings, like KV, can run in a local simulation. Workers AI cannot.

If your project uses TypeScript, generate types from the configuration:

npx wrangler types

Run that command again whenever you add or rename a binding.

Run your first model

Before building the API, let’s make the smallest possible call.

Replace the Worker code with this:

export default {
  async fetch(request, env) {
    const result = await env.AI.run(
      '@cf/meta/llama-3.2-3b-instruct',
      {
        messages: [
          {
            role: 'user',
            content: 'Explain what DNS does in one short sentence',
          },
        ],
        max_tokens: 80,
      },
    )

    return Response.json(result)
  },
}

Start the development server:

npx wrangler dev

Wrangler normally opens the Worker at http://localhost:8787.

Call it from another terminal:

curl http://localhost:8787

The response has this shape:

{
  "response": "DNS translates human-readable domain names into IP addresses that computers use to find each other.",
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 18
  }
}

The wording and token counts will vary.

Understand the model call

The call has two required parts.

The first argument is the model ID:

'@cf/meta/llama-3.2-3b-instruct'

The second argument is the model input:

{
  messages: [
    {
      role: 'user',
      content: 'Explain what DNS does in one short sentence',
    },
  ],
  max_tokens: 80,
}

The messages array describes the conversation. We only need one user message here.

max_tokens caps the generated output. My advice is to set it instead of relying on the default. A small limit controls cost and stops a short feature from producing a long answer.

Turn it into a summarization API

Now let’s accept text from a client.

Replace the Worker code with this:

export default {
  async fetch(request, env) {
    if (request.method !== 'POST') {
      return new Response('Send a POST request', { status: 405 })
    }

    const body = await request.json()

    if (typeof body.text !== 'string' || body.text.length > 5000) {
      return Response.json(
        { error: 'Send a text value no longer than 5000 characters' },
        { status: 400 },
      )
    }

    const result = await env.AI.run(
      '@cf/meta/llama-3.2-3b-instruct',
      {
        messages: [
          {
            role: 'system',
            content: 'Summarize the text in one clear sentence.',
          },
          {
            role: 'user',
            content: body.text,
          },
        ],
        max_tokens: 100,
      },
    )

    return Response.json({ summary: result.response })
  },
}

The endpoint only accepts POST requests.

It also rejects missing or very large input before calling the model. Validation matters because every accepted request can consume AI usage.

The system message gives the model one focused job. The user message contains the text to summarize.

Test the API

Keep wrangler dev running and send a request:

curl http://localhost:8787 \
  -H 'Content-Type: application/json' \
  -d '{"text":"Cloudflare Workers run JavaScript on Cloudflare infrastructure close to users. They can access services such as KV, D1, R2, and Workers AI through bindings."}'

You should receive JSON like this:

{
  "summary": "Cloudflare Workers run JavaScript near users and connect to Cloudflare services through bindings."
}

We now have a small but complete AI API.

Deploy it when you are ready:

npx wrangler deploy

Prompts are part of the program

The model does not know what your feature is trying to accomplish unless you tell it.

Compare these instructions:

Summarize this text.

and:

Summarize the text in one clear sentence for a beginner.

The second prompt defines length, style, and audience. It produces more predictable output.

Treat prompts like code. Keep them in version control, test them with real inputs, and check the output when you change models.

Models are nondeterministic. A prompt that worked once is not enough evidence.

Stream long responses

Our summary is short, so waiting for the complete result is fine.

For longer text, you can stream output as the model generates it. Add stream: true:

const stream = await env.AI.run(
  '@cf/meta/llama-3.2-3b-instruct',
  {
    messages: [
      { role: 'user', content: 'Explain how HTTP caching works' },
    ],
    stream: true,
  },
)

return new Response(stream, {
  headers: {
    'Content-Type': 'text/event-stream',
  },
})

The binding returns a ReadableStream. Returning that stream directly lets the browser receive chunks without buffering the entire response in the Worker.

For a POST chat endpoint, call the Worker with fetch() and read response.body. Browser EventSource only sends GET requests.

Run other kinds of models

The same binding can run a completely different task.

For example, an embedding model turns text into a vector:

const result = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: [
    'Cloudflare Workers run code',
    'R2 stores files',
  ],
})

const vectors = result.data

Embeddings are useful for semantic search and retrieval. They are not generated prose.

The output shape changes because the task changed. Read the selected model’s schema instead of assuming every model returns result.response.

How Workers AI pricing works

Workers AI measures usage in neurons.

Neurons represent the GPU work used by an inference request. Different model types consume work differently, so this gives Cloudflare one billing unit for text, images, and audio.

At the time of writing, both the Workers Free and Workers Paid plans include 10,000 free neurons per day. On Workers Paid, usage above that allocation costs $0.011 per 1,000 neurons. On the Free plan, requests stop until the daily reset unless you upgrade.

Model pages also show unit prices in familiar terms, such as input and output tokens. Check the current Workers AI pricing page before estimating production cost.

Usage resets daily at midnight UTC.

A smaller model is often enough

The largest model is not automatically the right model.

A small model may be faster and cheaper for classification, extraction, short summaries, or rewriting a sentence. A larger model may be worth the cost for harder reasoning.

Start with the smallest current model that can pass your real tests.

Build a small set of representative inputs and expected qualities. Run every candidate model against the same set. Then compare output, latency, and cost.

Model choice is an application decision, not a popularity contest.

Protect a public AI endpoint

Our tutorial endpoint is public. Anyone who knows the URL can call it.

Before using this pattern in production, add limits before env.AI.run():

  1. authenticate the user or verify a Turnstile token
  2. validate and limit the input size
  3. enforce a per-user or per-IP allowance
  4. enforce a global daily cap
  5. call the model only after those checks pass

The order matters. A rate limit checked after inference does not protect your budget.

Also handle model errors. A model can be unavailable, rate limited, or retired. Decide whether your feature should return an error, try another model, or provide a non-AI fallback.

Add AI Gateway when you need control

Workers AI works without AI Gateway.

When the feature becomes important, route the call through a gateway:

const result = await env.AI.run(
  '@cf/meta/llama-3.2-3b-instruct',
  input,
  {
    gateway: {
      id: 'production-ai',
      metadata: {
        feature: 'summary',
      },
    },
  },
)

This keeps the same model call and adds gateway logs and controls.

Read the complete setup in my Cloudflare AI Gateway tutorial.

How I use Workers AI

I use Workers AI for the app idea generator on this site.

That feature has a narrow job and short output. A small model fits it well.

The endpoint verifies Turnstile, checks visitor and global limits, and only then runs the model. If inference fails, it returns an idea from a curated list instead.

This is how I would approach similar features. Give the model a small task, put a hard boundary around usage, and define what happens when AI is unavailable.

I would not use a small Workers AI model for every problem. If an application needs the quality of a specific frontier model, I would call that provider and put AI Gateway in front of it.

When to use Workers AI

Workers AI is a good fit when:

  • your application already runs on Workers or Pages
  • a current catalog model handles the task well
  • you want inference without managing GPUs
  • you want to avoid a separate provider API key
  • the feature is focused enough to test and limit

It is a poor fit when no available model meets your quality, context, latency, or modality requirements.

The important part is not that the model runs on Cloudflare. The important part is that it solves the feature reliably.

Workers AI makes the infrastructure part very small: one binding and one method call. That leaves you more time for the real work — choosing the right task, testing the output, and protecting the endpoint.

Tagged: Cloudflare · All topics

Want me to talk about your product? You can sponsor this site.

~~~

Related posts about cloudflare: