What does the count actually tell me?

How much memory you need and roughly what it will cost to serve. That is genuinely useful if you are running the model yourself and close to irrelevant if you are using somebody's hosted service, where the figure has no bearing on your bill and no reliable bearing on the quality you experience.

Should we prefer a bigger model?

Only if the work needs it, and most work does not. The gap between sizes shows up on ambiguous, multi-step problems and closes to almost nothing on summarising, classifying and drafting, so paying for headroom that the task never reaches is the usual mistake rather than the safe choice.

How does a large model run on a laptop?

By storing the learnt values less precisely, which shrinks what has to be held in memory at some cost to quality. That trade is what local runners are doing when they offer several versions of the same model, and it is why the same name can appear in sizes that behave noticeably differently.

Glossary

Parameters

Parameters are the learnt numbers inside a model, and the count is quoted as though it measured quality when what it actually measures is the hardware you will need.

In plain terms

The learnt numbers inside, the ones adjusted during training until the thing behaved. The count gets quoted everywhere as though bigger meant better. What it reliably tells you is how much machine you need to run it and how much it will cost to serve, which is useful and is not the same claim at all.

Why it matters

Because it is the figure buyers are given and the wrong one to compare on. Two systems with similar counts can differ enormously in usefulness depending on what they were trained on and how they were shaped afterwards, while the count predicts your hardware bill precisely, which is why it survives as a specification long after it stopped predicting quality.

How it works

The count sets the memory requirement almost directly, which is the one thing it genuinely determines. Every learnt number has to be held while the system runs, so the size of the file and the size of the machine follow from the count, and this is why the figure is indispensable to anybody serving a model themselves and largely decorative to anybody using a hosted one.

Quality stopped tracking it some time ago. How much material something was trained on, how well that material was chosen and how carefully the behaviour was shaped afterwards now account for more of the difference than raw size, which is why smaller recent models routinely outperform larger older ones on the work people actually do.

Counting is not consistent across designs, so comparisons mislead. Some arrangements hold a very large number of learnt values and use only a fraction of them for any given request, which means a headline figure and a running cost can diverge sharply, and two systems described with similar numbers may behave nothing alike.

It can be traded against precision after training. Storing the same learnt values less precisely shrinks what has to be held in memory, letting a larger system run on a smaller machine at some cost to quality, which is how a model that nominally needs a server ends up running on a laptop at all.

What the number predicts

There is a familiar pattern behind this, which is that a measurement in wide use is usually the one that was easy to obtain rather than the one that answers the question. Counting learnt values is trivial and reporting it is unambiguous, so it became the number on the box. Establishing whether a model is good at your work requires running it on your work, which is effortful, unglamorous and produces a result that does not generalise to anybody else, so nobody puts it on a box. The consequence is that a figure with genuine engineering meaning gets pressed into service as a proxy for quality, and it was a passable proxy for a while, back when everything was trained similarly and the only real variable was scale. That stopped being true some years ago. What has not changed is that it remains the easiest number to quote, which is why it is still the first thing anybody is told about a model and among the last things worth deciding on.

Seen in the wild

A local runner where the practical limit is the hardware in front of you, because size and speed follow the machine rather than the ambition.
Ollama
A desktop application for running open models where memory settings sit on visible controls, so the trade is made deliberately rather than discovered.
LM Studio
An open-weight family whose members are published in several sizes, so the same line can be served on very different hardware.
Llama (Meta)

Common misconceptions

People assume

More is better.

In fact

More is more expensive to run, and quality now depends far more on the training material and the shaping afterwards. Smaller recent models routinely beat larger older ones on real work, which is why the count persists as a specification rather than as a reliable guide to anything you care about.

People assume

Two models with similar counts are comparable.

In fact

Not across different designs. Some arrangements hold an enormous number of learnt values and use only a portion for any request, so the headline figure and the actual cost of running can diverge sharply while the specification sheets look alike.

Telling them apart

Parameters vs Small language model

Parameters

The count, which sets the hardware.

Small language model

A deliberate choice to stay compact for speed and cost.

One is a measurement and the other is a design intent, which is why a small model is not simply a large one with a lower number.

Questions

What does the count actually tell me?: How much memory you need and roughly what it will cost to serve. That is genuinely useful if you are running the model yourself and close to irrelevant if you are using somebody's hosted service, where the figure has no bearing on your bill and no reliable bearing on the quality you experience.
Should we prefer a bigger model?: Only if the work needs it, and most work does not. The gap between sizes shows up on ambiguous, multi-step problems and closes to almost nothing on summarising, classifying and drafting, so paying for headroom that the task never reaches is the usual mistake rather than the safe choice.
How does a large model run on a laptop?: By storing the learnt values less precisely, which shrinks what has to be held in memory at some cost to quality. That trade is what local runners are doing when they offer several versions of the same model, and it is why the same name can appear in sizes that behave noticeably differently.

Key takeaways

The count sets memory and serving cost, almost directly.
It stopped predicting quality; training material and shaping matter more.
Counting differs between designs, so similar figures can mean different things.
Precision can be traded after training, which is how big models reach small machines.