Build a language model on your Mac with Language Model Builder

Language Model Builder is a free macOS app that teaches you how language models work, then lets you train one yourself.

Everything happens on your Mac. You choose the data, watch training progress, fine-tune the result, and chat with the model you made.

You don’t need Python, a cloud GPU, or a machine learning background.

I really like this idea. Language models feel less magical once you see each part working.

What is Language Model Builder?

Language Model Builder is an educational app created by Felix Rieseberg.

It has two parts:

• an interactive introduction to language models
• a native workbench for training a small GPT

The introduction takes about 90 minutes. It covers next-token prediction, tokenization, embeddings, attention, transformers, training data, gradient descent, and fine-tuning.

The workbench lets you use those ideas immediately. You can build a tokenizer, select a dataset, start a training run, inspect samples, and talk to the finished model.

If you follow the default path, you are not downloading a finished chatbot and changing its name. You are creating a small model and training its weights.

What do you need?

The app currently requires:

• an Apple Silicon Mac
• macOS 15 or later

It uses MLX, Apple’s machine learning framework for Apple Silicon.

The app is free. It does not require an account or subscription. You can download the current build from the official website or its GitHub releases.

Windows, Linux, and Intel Macs are not supported at the moment.

Start with the interactive textbook

My advice is to resist the temptation to start training immediately.

Go through the introduction first. The lessons build the mental model you need to understand the training screens.

You start with a simple idea: a language model predicts what comes next.

The app then walks through the pieces that make this possible.

Tokenization

A model does not read text as words. It receives a list of numbers called tokens.

Tokenization turns text into those numbers. Different tokenizers can split the same sentence in different ways, which changes what the model sees and how efficiently it learns.

Language Model Builder lets you build and inspect a tokenizer instead of treating it as a hidden preprocessing step.

Embeddings and attention

An embedding represents each token as a position in a large numerical space. Tokens used in similar ways can develop similar representations.

Attention lets each token consider other tokens in its context. This is how the model connects a word to the earlier parts of a sentence.

The app includes interactive playgrounds for both ideas. You can move through embedding space and inspect attention rather than only reading definitions.

The transformer

The transformer combines embeddings, attention, and other layers into the architecture behind modern language models.

You do not need to memorize every equation. First understand how information moves through the model and how a prediction becomes a training signal.

That prepares you for the second half of the app.

Build your model

The complete from-scratch workflow looks like this:

1. configure the tokenizer and model
2. choose or import training data
3. pre-train the model
4. inspect samples and checkpoints
5. run supervised fine-tuning
6. optionally run direct preference optimization
7. chat with the result

Let’s look at those stages.

The Starter blueprint uses a 10,000-token BPE vocabulary, a 512-token context, 256-wide embeddings, eight transformer blocks, and four attention heads. On my M4 Pro, the app estimated roughly one to two hours for its standard 20,000-step run.

You can also import a base model

The app also offers a shortcut. You can import an open base model and skip project setup and pre-training.

The catalog includes SmolLM2, Supra, and Qwen2.5 models. The smallest models are easier to fine-tune, while the larger ones need much more unified memory.

Importing locks the project’s architecture and tokenizer to the selected model. You can still run SFT, DPO, sampling, and chat.

This is how most real fine-tuning projects start. But if your goal is to understand the entire process, build the small model first.

Pre-training

During pre-training, the model learns to predict the next token from a large text collection.

At first, its output is mostly noise. Each training step compares its prediction with the correct token and adjusts the model’s weights.

Language Model Builder shows the training and validation loss as the run progresses. A falling loss means the model is getting better at predicting the data.

You can also watch throughput, saved samples, and checkpoint history. This turns training from a script that prints numbers into something you can follow.

The app includes a catalog of explained datasets. You can also provide your own.

The catalog includes TinyStories, WikiText, Simple English Wikipedia, arXiv abstracts, poetry, and other text collections. Each entry shows its size, source, license, and a preview.

Be careful with data you did not create. A local training run does not remove copyright, privacy, or licensing concerns.

Checkpoints and sampling

Training can take hours or days. You do not have to finish it in one session.

The app saves checkpoints, which are snapshots of the model’s weights. You can stop a run, close the app, and continue from a checkpoint later.

You can also sample text while training. This is where you start seeing the model move from random output toward recognizable language.

The sampling view exposes controls such as temperature, top-k, top-p, and min-p. It also shows token probabilities, so you can see why different settings produce different text.

Supervised fine-tuning

Pre-training teaches a model to continue text. It does not automatically teach it to answer a question like an assistant.

Supervised fine-tuning, or SFT, trains the model on examples of prompts and good responses.

The app includes fine-tuning datasets with conversations, instructions, and math problems. You can also bring your own data.

This stage changes the model’s behavior. It teaches a base model what kind of response you want.

Direct preference optimization

Direct preference optimization, or DPO, teaches the model from comparisons.

You give it a preferred answer and a rejected answer. Training pushes the model toward the preferred pattern.

Language Model Builder presents this as a ballot with two candidate answers. You choose the better one, turning an abstract alignment technique into a concrete action.

For a first experiment, SFT is enough. Add DPO when you want to understand how preference data changes the result.

Chat with the model

The final step is chatting with your creation.

The chat interface includes an X-ray view. It shows generated tokens, their probabilities, and alternatives the model did not choose.

This is more useful than a polished chat window. It reminds you that the model generates one token at a time from a probability distribution.

You can export models using the safetensors format. This makes the weights portable to other compatible tools.

What kind of model can you build?

Set your expectations correctly.

Language Model Builder trains small educational models. Its website says the default settings can produce coherent, grammatical text in about a day.

It also gives a higher-end example: a MacBook Pro with an M5 Max can train a GPT-2-small-class model of roughly 100 to 150 million parameters on a few billion tokens in about a week.

That is tiny compared with current frontier models.

The goal is not to replace ChatGPT or Claude. The goal is to understand what training a model actually involves.

Small models are perfect for that. Their mistakes are easier to inspect, and experiments finish on hardware you already own.

Memory still matters. Model size is only part of the calculation because training needs memory for gradients, optimizer state, and intermediate values. If you want to understand the basic model-memory math, read how much VRAM an LLM needs.

Does everything stay local?

Your projects, imported data, weights, prompts, preferences, and training results stay on your Mac.

There is no account and no cloud training service.

There are a few normal network connections worth knowing about. Dataset downloads connect to their hosting service, such as Hugging Face. The direct-download app checks for software updates. Crash reports are only sent with your permission, and you can inspect them first.

The privacy policy explains these details clearly.

Who is this for?

Language Model Builder is a good fit if you:

• use language models but don’t understand their internals
• learn better by changing things and watching the result
• want to experiment without assembling a Python toolchain
• want a local project with no API bill
• teach AI concepts to other people

It is not the right tool for training a production frontier model. It is also limited to recent Apple Silicon Macs.

But as a learning environment, that narrow focus is a strength. The app removes setup work while keeping the important parts visible.

For a longer software-engineer walkthrough of training a small GPT on a Mac with Language Model Builder, see How to build an LLM from scratch.

Download Language Model Builder, go through the introduction, and train the smallest model first.

Watching a model go from nonsense to recognizable text will teach you more than another hour of talking about AI.

Want me to talk about your product? You can sponsor this site.