What kind of machine do I need?

Enough memory to hold the model while it runs, which is the constraint that bites first, and a reasonably recent processor of the sort designed for this kind of work. Smaller and reduced-precision models widen what an ordinary laptop can manage. The honest test is downloading one and seeing.

There is no bill, and it is not costless. You are spending your own hardware, your own electricity and your own time keeping it current, and the machine is slower while it works. For one person that adds up to very little, which is exactly why it is worth trying before assuming anything.

Can it work without an internet connection?

Yes, once the model is downloaded, and this is a genuinely useful property rather than a novelty. It also means the arrangement survives a provider changing its terms, raising its prices or withdrawing the model, because the file you have is the file you keep.

Glossary

Local AI

Local AI means a model running on the machine in front of you, so what you type is processed by your own hardware and never travels anywhere, with no account, no connection and no provider involved.

In plain terms

You download a file and run it. The model works on your own processor, your own hardware and your own electricity. Turn off the internet and it carries on. That is the whole appeal, and the price of it is that your laptop is not a data centre, so what you can run is smaller than what a provider would hand you.

Why it matters

For one person's own work, it is the arrangement where the question of what happens to your material has a one-word answer. For a lawyer reading a confidential draft, a clinician with notes, or anybody working under a rule about material leaving a boundary, that is not a preference but the whole decision. It is also unusually easy to try: an afternoon and a reasonably modern machine will tell you whether the smaller models clear your bar, which is a cheaper way to find out than any amount of discussion.

How it works

You download a published model file and a program that runs it. The file is large and the program is ordinary software. Once it is on the disk nothing else is required, which is why these setups keep working on an aeroplane and keep working if the company that published the model loses interest.

What you can run is decided by memory rather than by ambition. The model has to fit while it works, so a machine with modest memory runs modest models, and reduced-precision versions exist that fit more into less at some cost to quality. Choosing the smallest one that clears your own tests is the practical approach.

Speed depends on your hardware and is usually the first disappointment. Machines with the right sort of processor handle this comfortably and older ones produce answers at a pace that makes long tasks tiresome. It is worth measuring on the actual machine rather than reading about the general case.

Everything is yours, including the parts nobody wants. No account, no terms to read and no bill, and also no updates arriving on their own, no support and nobody else responsible when a new version behaves differently. For one person's own use that is a fair trade and it is not the same trade at organisational scale.

The trade, stated once

The trade here is unusually clean, which is why it is worth stating in full rather than arguing about. Everything on the left follows from the file being on your disk: the text has nowhere to travel, the network is optional, no terms apply, and nothing anybody decides later can take the model away from you. Everything on the right follows from exactly the same fact. Your machine is smaller than a provider's, its speed is now your problem, nothing updates itself, and there is no support desk. For one person working on confidential material the left column frequently settles the matter on its own. The moment somebody else starts depending on it, the right column stops being a list of inconveniences and becomes a job.

Seen in the wild

Download a published model and put a confidential draft to it with the network switched off, which settles the question of where the text went.
Ollama
Load two models into a local application and compare them on the same question, where the only cost is the time your own machine takes.
LM Studio
Point a self-hosted assistant at a folder on the same machine, so both the documents and the answers stay on one disk.
AnythingLLM

Common misconceptions

People assume

It is as capable as the hosted assistant we use.

In fact

Usually less so, because the models that fit on a laptop are smaller than the ones a provider runs on a rack. The gap has narrowed considerably and for summarising, drafting, classifying and tidying it is often invisible. For the hardest reasoning it is still there, which is why testing on your own work beats assuming in either direction.

People assume

Local means secure.

In fact

It means the text did not travel, which is a real and narrow claim. The machine itself still needs a password, disk encryption and the same care as any laptop holding confidential material, and a stolen machine is a bigger problem than a provider's retention policy ever was.

Telling them apart

Local AI vs Self-hosted

Local AI

One person's own device, serving them alone. Nothing to administer, nothing shared, and no colleague depending on it.

Self-hosted

The organisation's servers, shared and administered centrally, sized for several people at once.

Both keep material off a provider's machines and only one of them has a rota. If somebody would have to be called when it stops, it is the second.

Questions

What kind of machine do I need?: Enough memory to hold the model while it runs, which is the constraint that bites first, and a reasonably recent processor of the sort designed for this kind of work. Smaller and reduced-precision models widen what an ordinary laptop can manage. The honest test is downloading one and seeing.
Is it free?: There is no bill, and it is not costless. You are spending your own hardware, your own electricity and your own time keeping it current, and the machine is slower while it works. For one person that adds up to very little, which is exactly why it is worth trying before assuming anything.
Can it work without an internet connection?: Yes, once the model is downloaded, and this is a genuinely useful property rather than a novelty. It also means the arrangement survives a provider changing its terms, raising its prices or withdrawing the model, because the file you have is the file you keep.

Key takeaways

The model runs on your own machine, so what you type never leaves it.
Memory decides what you can run, and smaller or reduced-precision models widen the options.
It keeps working offline and survives a provider changing its mind.
The models that fit a laptop are smaller, and for everyday drafting work the gap is often invisible.
Local means the text did not travel, which is narrower than secure.

Tools that use this

Ollama
Downloads and runs published models on your own machine.
LM Studio
A local application for comparing models on the same question.
AnythingLLM
Documents and answers on one disk, with nothing sent out.

Last checked July 2026

All glossary terms