How AI is built

The mechanism that lets a model weigh which parts of an input matter most for each word it produces, and the idea the whole transformer design is built around.

A model adapted to one organisation's data or tone, which vendors sell as a premium tier and which needs upkeep as the underlying model moves.

Marking up examples so a model can learn from them, such as tagging which emails are complaints, usually slow and done by people.

Training a smaller model to imitate a larger one, to get most of the quality at a fraction of the cost and wait.

Taking a trained model and training it further on a smaller, specific set of examples: your firm's writing style, a legal domain, a support script.

A fixed set of examples with known good answers, kept unchanged so that any drop in quality shows up as a difference rather than a feeling.

The kind of chip AI runs on, built to do enormous numbers of simple calculations at once, which is exactly the shape of the work models do.

A short standard summary of one model's intended uses, training approach and limitations, published so buyers can compare like with like.

A vendor retiring a model version on a published date, after which anything built on it must move or stop working.

Naming and dating the exact model behind a feature, so results can be reproduced and a change can be traced to a cause.

When a model learns its training examples too closely and performs worse on anything new, the commonest way a fine-tuning project quietly fails.

Everything done to shape a model after its main training run, which is where most of its manners and refusals come from.

The long first stage of building a model, where it learns general patterns from an enormous body of material before anyone teaches it to be helpful.

Storing a model's numbers at lower precision so it needs less memory and can run on smaller hardware, giving up a little accuracy.

Training by trial and reward rather than by copying examples: the system tries something, gets scored, and shifts towards what scored well.

The stage where people rank a model's competing answers and it is trained towards the ones they preferred, which is much of why assistants feel helpful.

Training where the material supplies its own answers, such as hiding the next word and scoring the guess, which is what made training on the open web possible.

Training on examples where the right answer is already attached, which is how most classification and prediction systems are built.

Training material generated rather than collected, used where real examples are scarce, sensitive or expensive to label.

A vendor's published account of how a whole AI system behaves, including its known weaknesses and the testing it was put through.

Cutting text into the chunks a model actually reads, which is why word counts and token counts never quite match and why some languages cost more.

The date a model's training data stops, after which it knows nothing unless the information is searched for or supplied in the request.

The text, images and code a model learnt from: much of the public web, books, licensed sources, and sometimes users' conversations.

Reusing what a model learnt on one large task as the starting point for a different, usually smaller one, instead of starting from nothing.