# Muse Ember vs Claude Haiku 5.5 vs GPT-6 Luna on a real app job > Three small AI models at the same price, $0.10 and $0.50 per million tokens, tagging 200 blog posts. Accuracy, latency and cost compared, with the script. Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-11-12 | Topics: [AI](https://flaviocopes.com/tags/ai/) | Canonical: https://flaviocopes.com/small-ai-models/ Most AI features in an app are small jobs. Pick a category, pull three fields out of an email, decide which queue a message goes to. You don't need a frontier model for that. You need a small one that's fast, cheap, and right most of the time. There are now three of them at the same price. **Muse Ember**, **Claude Haiku 5.5** and **GPT-6 Luna** all cost $0.10 per million input tokens and $0.50 per million output tokens. So the price list doesn't decide. I gave all three the same job from this site and measured how often they're right, how long they take, and what the bill says. They got between 74% and 79% of the answers right, very close to each other. The bill was not close: for the same 200 requests one model cost four times more than another. ## The three models | | Muse Ember | Claude Haiku 5.5 | GPT-6 Luna | | --- | --- | --- | --- | | Company | Meta | Anthropic | OpenAI | | Released | Nov 9, 2026 | Oct 7, 2026 | Sep 22, 2026 | | Input / output per 1M tokens | $0.10 / $0.50 | $0.10 / $0.50 | $0.10 / $0.50 | | Price goes up after | 200k tokens | 100k tokens | 272k tokens | | Context window | 1M | 1M | 1M | | Reasoning can be turned off | No | Yes | Yes | | OpenRouter id | `meta/muse-ember` | `anthropic/claude-haiku-5.5` | `openai/gpt-6-luna` | [Muse Ember](https://research.meta.ai/blog/muse-ember) is the newest. Meta built it for repeated agent work: tool calls, routing, extraction, parsing long documents. It takes text, images, audio and video. [Claude Haiku 5.5](https://www.anthropic.com/claude-haiku-5-5) is the small model of the Claude 5.5 family, and the first Haiku with an effort setting. Be careful with long prompts: past 100k tokens it costs $0.50 / $2.50, five times more. [GPT-6 Luna](https://openai.com/index/introducing-gpt-6-sol-and-luna/) is OpenAI's smallest GPT-6 model, made for simple tasks you run a lot. I cover the bigger models, the ones you'd use to write code, in [which AI model is best for coding](https://flaviocopes.com/ai-coding-models/). ## The job: tagging my posts This site has more than 2,000 posts, and each one has a tag picked from a list of 51: `js`, `node`, `css`, `ai`, `swift` and so on. I assigned those tags by hand over the years. That makes them a good test, because I already know the right answer for every post. The job is: read the title and the description of a post, and pick one tag from the list. It's the kind of feature you'd add to a CMS, a help desk or a notes app. 2,085 published posts have exactly one tag and a description. I sampled 200 of them, evenly spread across the alphabet, into a `dataset.json` file: ```json { "tags": ["ai", "alpine", "astro", "blogging", "business", "clang"], "posts": [ { "slug": "c-string-length", "title": "How to find the length of a string in C", "description": "Learn how to find the length of a string in C using the strlen() function from the string.h standard library, which returns the length as an integer.", "tag": "clang" } ] } ``` I cut the tags list here, the real one has all 51. The sample has 42 different tags in it. `js` is the most common with 38 posts, then `ai` and `node` with 19 each. A model that answers `js` every time would get 19% right, so that's the number to beat. ```mermaid flowchart LR accTitle: How the tagging test runs accDescr: 200 posts with a hand-picked tag go to Muse Ember, Claude Haiku 5.5 and GPT-6 Luna through OpenRouter. Each model returns one tag per post as JSON, and the script compares it with the tag picked by hand, then reports accuracy, latency and cost. A[200 posts: title + description] --> B[OpenRouter] T[51 allowed tags] --> B B --> C[Muse Ember] B --> D[Claude Haiku 5.5] B --> E[GPT-6 Luna] C --> F[One tag as JSON] D --> F E --> F F --> G{Same as my tag?} G --> H[Accuracy, latency, cost] ``` ## One API for the three models Each company has its own API. To keep the code the same for the three models I used [OpenRouter](https://openrouter.ai/), which gives you one endpoint and one key for all of them. Create a key on the [API keys page](https://openrouter.ai/settings/keys), set a spending limit of $1 on it, and put it in a `.env` file: ```dotenv OPENROUTER_API_KEY=your-openrouter-key ``` The script uses Node's built-in `fetch()`, so there's nothing to install. ## Check the prices first Create `tag-bench.mjs`. The first part loads the dataset and reads the live prices from OpenRouter's public catalog, which needs no key: ```javascript import { existsSync, readFileSync, writeFileSync } from 'node:fs' const models = ['meta/muse-ember', 'anthropic/claude-haiku-5.5', 'openai/gpt-6-luna'] const runs = [ ['anthropic/claude-haiku-5.5', 'off'], ['openai/gpt-6-luna', 'off'], ['meta/muse-ember', 'low'], ['anthropic/claude-haiku-5.5', 'low'], ['openai/gpt-6-luna', 'low'] ] const key = process.env.OPENROUTER_API_KEY const { tags, posts } = JSON.parse(readFileSync('dataset.json', 'utf8')) const resultsFile = 'results.json' const results = existsSync(resultsFile) ? JSON.parse(readFileSync(resultsFile, 'utf8')) : {} async function readJson(url, options = {}) { const response = await fetch(url, { ...options, signal: AbortSignal.timeout(120_000) }) const result = await response.json() if (!response.ok || result.error) { throw new Error(`${response.status} ${result.error?.message || ''}`) } return result } const catalog = await readJson('https://openrouter.ai/api/v1/models') const prices = {} for (const id of models) { const model = catalog.data.find(item => item.id === id) if (!model) throw new Error(`Model is unavailable: ${id}`) prices[id] = { input: Number(model.pricing.prompt), output: Number(model.pricing.completion) } console.log( id.padEnd(28), `$${(prices[id].input * 1e6).toFixed(2)} in`, `$${(prices[id].output * 1e6).toFixed(2)} out` ) } if (!process.argv.includes('--run')) { console.log(`${posts.length} posts, ${tags.length} tags. Add --run to make paid requests.`) process.exit(0) } if (!key) throw new Error('Set OPENROUTER_API_KEY in .env') ``` `runs` is the list of tests: a model and a reasoning setting. I'll explain why there are five in a moment. The catalog gives prices per token, so we multiply by a million to read them. Run it without `--run` and it sends nothing to the models: ```bash node tag-bench.mjs ``` ```text meta/muse-ember $0.10 in $0.50 out anthropic/claude-haiku-5.5 $0.10 in $0.50 out openai/gpt-6-luna $0.10 in $0.50 out 200 posts, 51 tags. Add --run to make paid requests. ``` ## Ask for one tag Now the request. The system prompt lists the allowed tags, and the user message is the title and the description of one post. The important part is `response_format`. We give the model a JSON schema where `tag` is an `enum` of our 51 tags, so it can't invent a new one or wrap the answer in a sentence. All three models support this. Add this to the file: ```javascript const system = `You tag blog posts for a programming blog. Pick the one tag that fits the post best. Allowed tags: ${tags.join(', ')}` async function tagPost(model, reasoning, post) { const started = performance.now() const result = await readJson('https://openrouter.ai/api/v1/chat/completions', { method: 'POST', headers: { Authorization: `Bearer ${key}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ model, messages: [ { role: 'system', content: system }, { role: 'user', content: `Title: ${post.title}\nDescription: ${post.description}` } ], response_format: { type: 'json_schema', json_schema: { name: 'tag', strict: true, schema: { type: 'object', properties: { tag: { type: 'string', enum: tags } }, required: ['tag'], additionalProperties: false } } }, reasoning: reasoning === 'off' ? { enabled: false } : { effort: reasoning }, max_tokens: 2000 }) }) let tag = null try { tag = JSON.parse(result.choices[0].message.content).tag } catch {} return { slug: post.slug, expected: post.tag, tag, ms: Math.round(performance.now() - started), inputTokens: result.usage.prompt_tokens, outputTokens: result.usage.completion_tokens } } ``` We keep the time and the token counts of every request, next to the answer. ## Reasoning on or off A tagging feature runs on every save, so I wanted the fast path: `reasoning: { enabled: false }`, which tells the model to answer without thinking first. Claude Haiku 5.5 and GPT-6 Luna accept that. Muse Ember doesn't, and every request came back with this error: ```text meta/muse-ember 400 Reasoning is mandatory for this endpoint and cannot be disabled. ``` So Ember runs with `reasoning: { effort: 'low' }`, the lowest setting I could send. To compare it fairly I also ran the other two with low reasoning. That's the five entries in `runs`. ## Run the 1,000 requests The loop sends five posts at a time for one entry of `runs`, then moves to the next. It saves `results.json` after every batch, so if a request fails you run the script again and it only does what's missing. ```javascript for (const [model, reasoning] of runs) { const name = `${model} ${reasoning}` results[name] ??= [] const done = new Set(results[name].map(row => row.slug)) const todo = posts.filter(post => !done.has(post.slug)) for (let i = 0; i < todo.length; i += 5) { const rows = await Promise.allSettled(todo.slice(i, i + 5).map(post => tagPost(model, reasoning, post))) for (const row of rows) { if (row.status === 'fulfilled') results[name].push(row.value) else console.error(name, row.reason.message) } writeFileSync(resultsFile, JSON.stringify(results, null, 2) + '\n') } } ``` The last part prints one line per run: how many tags match mine, how many answers were not a valid tag, the median and 95th percentile time, and the cost computed from the token counts. ```javascript for (const [model, reasoning] of runs) { const name = `${model} ${reasoning}` const rows = results[name] const correct = rows.filter(row => row.tag === row.expected).length const invalid = rows.filter(row => !tags.includes(row.tag)).length const times = rows.map(row => row.ms).sort((a, b) => a - b) const cost = rows.reduce((sum, row) => sum + row.inputTokens * prices[model].input + row.outputTokens * prices[model].output, 0) console.log( name.padEnd(32), `${correct}/${rows.length} correct`, `${invalid} invalid`, `median ${times[Math.floor(times.length / 2)]}ms`, `p95 ${times[Math.floor(times.length * 0.95)]}ms`, `$${cost.toFixed(4)}` ) } ``` Run it: ```bash node --env-file=.env tag-bench.mjs --run ``` ## The results Here's the output, from November 12, 2026: ```text anthropic/claude-haiku-5.5 off 148/200 correct 0 invalid median 808ms p95 1089ms $0.0166 openai/gpt-6-luna off 158/200 correct 0 invalid median 999ms p95 1681ms $0.0078 meta/muse-ember low 149/200 correct 0 invalid median 1278ms p95 2378ms $0.0342 anthropic/claude-haiku-5.5 low 151/200 correct 0 invalid median 877ms p95 1533ms $0.0171 openai/gpt-6-luna low 150/200 correct 0 invalid median 1868ms p95 3665ms $0.0093 ``` And the same numbers as a table, with each model on its fastest setting first: | Model | Reasoning | Correct out of 200 | Median time | 95th percentile | Cost for 200 posts | | --- | --- | --- | --- | --- | --- | | GPT-6 Luna | off | 158 (79%) | 999ms | 1,681ms | $0.0078 | | Claude Haiku 5.5 | off | 148 (74%) | 808ms | 1,089ms | $0.0166 | | Muse Ember | low | 149 (74.5%) | 1,278ms | 2,378ms | $0.0342 | | Claude Haiku 5.5 | low | 151 (75.5%) | 877ms | 1,533ms | $0.0171 | | GPT-6 Luna | low | 150 (75%) | 1,868ms | 3,665ms | $0.0093 | The whole test, 1,000 requests, cost less than 9 cents. No model returned an invalid tag in 1,000 answers. The JSON schema with the `enum` did its job, and I didn't have to write any code to clean up the output. ### Accuracy is a tie Every run landed between 148 and 158 correct answers. Ten posts out of 200 is a small gap for a single run, and I wouldn't pick a model on it. Low reasoning didn't help. Haiku got 3 more right with it, Luna got 8 fewer and took almost twice as long. For a job this small, turn it off when you can. ### Same price, different bill This is the part the price list doesn't tell you. Here's the average number of tokens each model billed for one request, with the same prompt: | Model | Input tokens | Output tokens | | --- | --- | --- | | Muse Ember (low) | 319 | 279 | | Claude Haiku 5.5 (off) | 778 | 11 | | GPT-6 Luna (off) | 329 | 12 | Haiku counts 778 input tokens where the other two count about 320. The text is the same, but each company counts tokens in its own way. So Haiku costs twice as much as Luna for this job. Ember's answer is one word, like the others, but it bills 279 output tokens for it, because the reasoning we can't turn off is billed as output. Output costs five times more than input, and that makes Ember the most expensive of the three here, more than four times Luna. ### Haiku is the fastest Claude Haiku 5.5 answered in 808ms at the median, and 95% of its answers arrived within 1.1 seconds. Luna is a little slower, with a longer tail. Ember takes about half a second more than Haiku because it thinks first. ## Where they disagree with me Take each model on its fastest setting. On 141 of the 200 posts, all three picked my tag. On 38 posts, all three picked a different one. In 25 of those 38 the three models agree with each other, and most of the time they have a point: | Post | My tag | All three said | | --- | --- | --- | | How to replace a DOM element | `platform` | `js` | | Fix Homebrew 'Permission denied @ apply2files' error | `node` | `mac` | | Tailscale tutorial | `security` | `network` | | Using Tailwind CSS with Vue 3 | `vue` | `tailwind` | | Johnny Five, how to work with an LCD Screen | `js` | `electronics` | | Why I think more clearly when I type | `blogging` | `essay` | Some of these are my habits. I use `platform` for browser APIs, and nothing in the prompt says so. That's the most common miss for every model, and one line in the system prompt would fix it. Others are my mistakes. A Homebrew permission fix has little to do with `node`. So the 74% to 79% says as much about my tags as about the models. If you run this on your own data, read the misses before you trust the score. ## How I would use it For a tagging feature on this site I would use GPT-6 Luna with reasoning off. It was the cheapest and it tied for the most accurate. At the cost I measured, tagging all 2,085 posts would be about 8 cents. If the tag had to appear while someone is typing, I would use Claude Haiku 5.5 for its speed and pay double. Double of very little is still very little. I wouldn't use Muse Ember for this. It's as accurate as the others, but slower and four times the cost on a one-word answer. Meta built it for longer agent work, and this test doesn't measure that. What I'd do first is different, though. I'd run the three models on every post and look only at the ones where all three disagree with me. In this sample that was 25 posts out of 200, a short list of tags to fix by hand. For new posts I don't need a model at all. I pick the tag when I write the post. ## What this test doesn't tell you This is one job, with short inputs and a one-word answer. It says nothing about long documents, tool calls or image input, where these models are also meant to be used. The times include OpenRouter in the middle. If latency is what you care about, run the winner again on the company's own API before you decide. To try it on your own app, replace `dataset.json` with 200 examples where you already know the right answer.