What is temperature in a language model?

By

Temperature changes how a language model samples its next token. Learn what it affects, when to adjust it, and why lower values do not ensure correct code.

Temperature is a generation setting that changes how strongly a language model favors likely next tokens. Lower values concentrate sampling on more likely choices. Higher values allow more variation.

Build a language model from scratch explains temperature alongside sampling and generation. This entry covers what the setting means when you use a model for coding.

How temperature works

A model produces scores for possible next tokens. Temperature changes those scores before the runtime samples a token. It affects inference, while the learned weights stay the same.

Temperature changes token samplingA model produces next-token scores, temperature adjusts the distribution, and the runtime samples a token before continuing generation.Model token scoresApply temperatureSample next tokenContinue generation

Ollama’s parameter reference documents temperature and a separate seed setting for repeatable generation. Keep the model and prompt fixed when comparing settings.

In coding tools

The Claude Code model configuration guide documents model and effort controls. Reasoning effort controls thinking and is a different setting from sampling temperature.

Cursor’s model documentation describes model selection. For Codex, see OpenAI’s working with agents guide. For Grok Build, see its overview. These pages do not document a shared temperature control across the four products. Use the specific client’s reference before adding a configuration key.

Details checked against each vendor’s documentation in November 2026.

A concrete example

When building a model client, give your coding agent this prompt:

Inspect our model request builder and the installed SDK types.
Find whether this endpoint and model accept temperature.
If supported, show the current value and where it is set.
Do not add a parameter based only on another provider's API.

If you experiment with generation settings, save the model identifier and exact input with each result. Compare the generated code using the same tests. A different answer alone tells you little about correctness.

When to adjust temperature

Adjust it when your model interface supports it and you want to explore output variation. A naming or explanation task can benefit from trying several alternatives. For a bug fix, evaluate whether each proposed patch passes the regression test.

Skip the adjustment when the client does not expose it. Express the required behavior in the prompt and verify the result.

Pitfalls

Lower temperature does not fix a false premise or supply missing documentation. A model can favor an incorrect answer strongly. Check unfamiliar APIs against installed code, as described in hallucination.

Do not change the prompt, model, and sampling settings together and attribute the result to temperature. Keep the other inputs fixed when testing a setting.

  • Inference: Inference is running a trained model on new input. Learn where coding models run, how inference differs from training, and what a local agent needs.
  • Hallucination: An AI hallucination is a plausible but false answer. Learn how to catch invented APIs, verify coding-agent claims, and separate output from evidence.
  • Reasoning effort: Reasoning effort sets how long a model thinks before it answers. The levels in Claude Code and Codex, what Cursor's plans fix, and how Grok sets effort.

All AI glossary entries →

Want me to talk about your product? You can sponsor this site.