Parameters, memory, and training estimates

Parameters are the learned weights changed during training. More parameters give a model more capacity, but they also increase compute and memory use.

Estimate the weight storage first:

weight bytes ≈ parameter count × bytes per stored parameter

A 9-million-parameter model stored with two bytes per value needs roughly 18 MB for weights. File metadata and tensor packaging add some overhead.

The saved weight file is not the full training-memory requirement. Training can also keep gradients, optimizer state, temporary buffers, and activations for every batch and layer.

Activations grow with batch size, sequence length, width, and depth. This is why halving context length or batch size can rescue a run even when the parameter count stays fixed.

Record the app’s parameter count, weight size, and standard training estimate. Note your Mac chip and unified memory beside them.

Treat the time as an estimate. Thermal conditions, background activity, dataset work, and configuration can change throughput.

Turn throughput into a second estimate when the app reports tokens per second:

training time ≈ total training tokens / tokens per second

If 10 million training tokens run at 2,000 tokens per second, the compute portion is about 5,000 seconds, or 83 minutes. Validation, checkpoint writes, startup, and changing throughput add time.

If the estimate is longer than you can leave the Mac available, reduce the run before starting. A completed small experiment is more useful than an abandoned oversized one.

Keep memory headroom for macOS and the application. A configuration that barely fits may slow dramatically or fail when another process uses unified memory.