Does it need less hardware?

No, and this is the most common misunderstanding. The whole model must be held ready because any section might be required next, so the memory requirement follows the total size. What falls is the work done per answer, which shows up in running cost and in speed rather than in what you must buy.

Why does this make comparison harder?

Because a stated figure may describe everything held or only what runs, and specifications frequently do not say which. Two models presented as comparable can therefore differ several-fold in operating cost, so the useful question when self-hosting is what has to be resident and what is active per request.

Should this change what we buy?

It should change what you ask. If somebody else hosts the model, the arrangement is invisible and irrelevant to you. If you are serving it yourself, it decides your hardware bill and your throughput, and the two numbers you need are the total that must be resident and the portion that runs.

Glossary

Mixture of experts (MoE)

A mixture of experts splits a model into sections and uses only a few of them for any given request, which separates the size you must keep loaded from the work each answer actually costs.

In plain terms

One model built as many sections, with only a few switched on for any given request. The whole thing still has to be kept loaded, and only a fraction of it does the work each time. That splits two things that used to move together: how big it is, and how much each answer costs.

Why it matters

Because it broke the rule of thumb everybody was using. Stated size used to predict running cost closely enough to compare products by it, and this arrangement severs that link, so two models described with similar figures can cost very different amounts to run and a much larger one can be cheaper per answer than a smaller conventional one.

How it works

The model is divided into sections and something decides which few to use for each request, so most of it stays idle. That routing decision is made per request rather than per deployment, which is why the same model can behave economically on ordinary work and reach for more of itself on something harder.

Memory and compute stop tracking each other, and this is the property with commercial consequences. Everything must still be held ready, so the hardware requirement follows the total size, while the work done per answer follows only the active portion, and a specification sheet quoting one figure tells you almost nothing about the other.

It is how large models became affordable to serve at volume. Getting the quality associated with an enormous system without paying enormous costs on every request is the entire commercial argument, and it is why several of the open families in this guide adopted the arrangement rather than simply building bigger.

Comparing two models by their headline figure became misleading as a result. One number may describe everything held and another may describe what runs, and unless a specification says which, two systems that look comparable can differ several-fold in what they cost to operate.

Two numbers that used to be one

This is the sort of detail that would be safely ignorable if it had not quietly invalidated a comparison everybody was making. For years the stated size of a model was a decent proxy for everything a buyer cared about: bigger meant more capable, more expensive to run and heavier to host, and the three moved together closely enough that one figure served. This arrangement pulls them apart. A model whose headline number is very large may activate a modest portion for each request, so it hosts like a big model and bills like a small one, while a conventional model of similar stated size does neither. For anyone using a hosted service none of this is visible or relevant, because the provider absorbs it and prices however they price. For anyone serving models themselves it is the difference between a hardware estimate that works and one that is wrong by a factor, which is why the question worth asking is always two questions.

Seen in the wild

A price disruptor whose open-weight releases pair genuinely competitive reasoning with an interface priced far below the western frontier labs.
DeepSeek
An open-weight line published for self-hosting, where what must be held in memory and what runs per request are separate practical questions.
Qwen
A production inference engine that squeezes throughput and concurrency out of hardware when open models go behind real applications.
vLLM

Common misconceptions

People assume

Only part of it needs to be loaded.

In fact

All of it has to be held ready, because any section might be needed for the next request. The saving is in the work done per answer rather than in the memory required, which is why this arrangement lowers running cost without lowering the hardware you must own to serve it.

People assume

The sections are experts in recognisable subjects.

In fact

The specialisation emerges from training and rarely corresponds to anything a person would name. The word suggests a panel of consultants and the reality is statistical division of labour, which is why nobody can tell you which section handles legal questions.

Telling them apart

Mixture of experts vs Parameters

Mixture of experts

An arrangement that separates size from work per request.

Parameters

The count, which used to predict both.

This arrangement is precisely why the count stopped being a reliable guide to running cost, so the two terms explain each other.

Questions

Does it need less hardware?: No, and this is the most common misunderstanding. The whole model must be held ready because any section might be required next, so the memory requirement follows the total size. What falls is the work done per answer, which shows up in running cost and in speed rather than in what you must buy.
Why does this make comparison harder?: Because a stated figure may describe everything held or only what runs, and specifications frequently do not say which. Two models presented as comparable can therefore differ several-fold in operating cost, so the useful question when self-hosting is what has to be resident and what is active per request.
Should this change what we buy?: It should change what you ask. If somebody else hosts the model, the arrangement is invisible and irrelevant to you. If you are serving it yourself, it decides your hardware bill and your throughput, and the two numbers you need are the total that must be resident and the portion that runs.

Key takeaways

Only a few sections run per request; all of them must still be held.
Memory and compute stopped tracking each other, which broke the old rule of thumb.
It is how large-model quality became affordable to serve at volume.
The specialisations emerge from training and match no human category.

Tools that use this

DeepSeek
Open-weight releases pairing competitive reasoning with low pricing.
Qwen
Self-hosted, where resident size and active work are separate questions.
vLLM
Serving infrastructure built for throughput and concurrency.

Last checked July 2026

All glossary terms