An LLM adapter pattern for Cloudflare Workers

Your app calls OpenAI today. Tomorrow you want Groq, OpenRouter, or a local bridge in dev. If every route imports a vendor SDK, you’re stuck.

StackPlan runs on Cloudflare Workers. Heavy SDKs often don’t run in workerd. I used a tiny adapter instead.

One interface

Define what your app actually needs. For StackPlan that’s two methods:

/** @typedef {object} LlmAdapter
 * @property {(input: RationaleInput) => Promise<Rationale>} generateRationale
 * @property {(prompt: string) => Promise<string>} completeText
 */

Every provider implements the same shape. Routes call adapter.completeText(prompt) and parse JSON from the string. They never import openai or @cursor/sdk directly.

Fetch-based OpenAI adapter

The OpenAI SDK is fine in Node. On Workers, use fetch:

function openAiAdapter({ apiKey, model, baseUrl }) {
  async function completeText(prompt) {
    const res = await fetch(`${baseUrl}/chat/completions`, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model,
        messages: [{ role: 'user', content: prompt }],
        temperature: 0.7,
      }),
    })

    if (!res.ok) throw new Error(`LLM failed (${res.status})`)

    const data = await res.json()
    const text = data.choices?.[0]?.message?.content
    if (!text) throw new Error('Empty LLM response')
    return text
  }

  return {
    async generateRationale(input) {
      const prompt = buildPrompt(input)
      return parseRationaleResponse(await completeText(prompt))
    },
    completeText,
  }
}

Any OpenAI-compatible endpoint works. Change baseUrl to OpenRouter, Groq, or Cloudflare AI Gateway. Same JSON shape.

Swap providers with an env var

Centralize selection:

function getAdapter(env) {
  if (env.LLM_PROVIDER === 'openai' && env.OPENAI_API_KEY) {
    return openAiAdapter({
      apiKey: env.OPENAI_API_KEY,
      model: env.OPENAI_MODEL || 'gpt-4o-mini',
      baseUrl: env.OPENAI_BASE_URL || 'https://api.openai.com/v1',
    })
  }
  if (env.LLM_PROVIDER === 'cursor' && env.CURSOR_API_KEY) {
    return cursorAdapter(env.CURSOR_API_KEY, env.CURSOR_BRIDGE_URL)
  }
  return null
}

Production sets LLM_PROVIDER=openai. Local astro dev can set LLM_PROVIDER=cursor and POST to a Node middleware that runs the Cursor SDK — because that SDK can’t execute inside workerd even during dev.

Degrade gracefully

When getAdapter() returns null, features should hide, not crash.

const adapter = getAdapter(env)
if (!adapter) return null

try {
  return await adapter.generateRationale(input)
} catch (err) {
  console.error('LLM rationale failed:', err)
  return null
}

The report page still shows stacks and costs from the rules engine. The prose block simply doesn’t render. Prefill from a GitHub URL still works from manifest scanning alone.

Cache successful responses in KV keyed by a hash of the input. Skip the network on repeat views.

The rules engine still picks providers and prices. The LLM only writes prose on top of numbers you already computed. That boundary keeps bad model output from breaking billing logic.

Parse defensively

Models wrap JSON in markdown fences. Strip fences, find the outermost {…}, validate with Zod, retry once on failure.

Keep prompt builders and parsers in the same module as the adapters. The interface stays thin. The messy vendor details stay in one file.

buildPrompt() and buildExtractionPrompt() live next to openAiAdapter(). Routes import getAdapter and the parse helpers. Nothing else touches vendor URLs.

Why not the SDK everywhere

Node apps can use @anthropic-ai/sdk or openai and call it a day. Workers are different. The runtime has fetch, crypto, and TextEncoder — not the full Node API surface vendor packages expect.

A twenty-line fetch wrapper buys you portability. You test it once. It runs the same in wrangler dev, production, and any OpenAI-compatible host.

That’s the pattern: one interface, fetch for Workers, env-driven provider swap, null when unconfigured. Your product keeps working when the LLM is off.

Related posts about js: