By what it does
Private, local & self-hosted
For private, local AI, the strongest pick is LM Studio: a desktop app that downloads and runs open models entirely on your machine, free to use. Ollama suits developers happier in a terminal; vLLM serves models at production scale; Open WebUI puts a chat interface over whichever engine you choose.
Local/self-hosted AI matters when data cannot leave your environment — NDA code, client data, regulated material — or when you want offline use and no API bills. Pick a runner by interface (CLI vs GUI vs web), then a model by task. The key constraint is hardware: memory first, GPU second. All the runners are free; cost shifts to hardware.
Capability is capped by your hardware (RAM/VRAM first); some open models are Chinese-origin — self-hosting the open weights mitigates governance concerns for regulated data.
13 tools fit Private, local & self-hosted · 13 have a free tier · 12 can run privately
Pick by what you need
The same task, a different best tool depending on the output you want.
CLI runner
Web/multi-user UI
Document RAG
Production serving
Model hub
Open-weight models
Self-hosted agents
What you can do here
Common workflows people run here.
Recommended tools
Tools in this category, easiest to start with first.
| Tool | Why it fits here | Cost | Ease | Official site |
|---|---|---|---|---|
| AnythingLLM | AnythingLLM is private document chat done properly | Free | Beginner-friendly | Site |
| DeepSeek | DeepSeek matters here for its open weights | Free tier + paid plans | Beginner-friendly | Site |
| Jan | Jan is the local assistant for people who want privacy without ceremony | Free | Beginner-friendly | Site |
| LM Studio | LM Studio makes local AI feel like installing an app | Free | Beginner-friendly | Site |
| GLM (Z.ai) | GLM belongs in this category because the MIT licence converts a jurisdiction question int… | Free tier + paid plans | Intermediate | Site |
| Ollama | Ollama is the default way to run open models on your own hardware | Free | Intermediate | Site |
| Open WebUI | Open WebUI turns a local model into a team product | Free | Intermediate | Site |
| Qwen | This category is Qwen's home ground | Free tier + paid plans | Intermediate | Site |
| Tabnine | Tabnine exists for the environments the cloud assistants cannot serve | Free tier + paid plans | Intermediate | Site |
| Hugging Face | Hugging Face is where the self-hosting journey starts | Free tier + paid plans | Advanced | Site |
| Llama (Meta) | Private and local is the posture Llama serves best: downloadable weights, deployed on har… | Free | Advanced | Site |
| OpenClaw | In the private and local category OpenClaw fills the gap the model runners leave: they ru… | Free | Advanced | Site |
| vLLM | vLLM is what self-hosting looks like at production scale | Free | Advanced | Site |
Not sure which of these fits? 60 seconds, five questions. Take the quiz
Tools that work together
Stacks for this work
The desktop private stack
For individuals who want private AI on their own machine, through apps rather than terminals.
CompetitiveThe private and local stack
For teams and individuals whose data cannot leave the building.
World-ClassThe production private AI stack
For organisations serving private AI to many users: regulated, air-gapped or simply sovereign by policy.
Where to start
Not sure what to adopt first?
Five quick questions about your job, task and constraints. We'll suggest your top three tools, plus the one to try first.
Common questions
Easiest way to run a local model?
LM Studio for a friendly desktop GUI; Jan if offline-first design matters; Ollama when you want a clean command line and an API other tools build on. All three get a capable open model running on ordinary hardware in minutes. Start with LM Studio to learn, graduate to Ollama when you want to wire it into things.
How do I chat with my documents privately?
AnythingLLM: point it at your documents and chat against them with everything, including the model, running on your own hardware. Nothing leaves the machine, which is the entire point for sensitive material. Answer quality depends on the model you pair it with; larger local models cost hardware but improve grounding noticeably.
What gives a local AI a multi-user interface?
Open WebUI layered on Ollama: a team-grade chat interface with accounts, access control and document features, backed by models served locally. That combination is the standard pattern for a private, shared AI capability without any cloud dependency. Operating it is real, if modest, sysadmin work; someone must own updates and backups.
What should I use for production model serving?
vLLM: the reference open-source engine for serving models at production throughput, with the batching and memory efficiency that hobbyist runtimes lack. It is the step you take when local AI graduates from a laptop tool to infrastructure other systems call. Ollama serves prototypes fine; sustained multi-user load is vLLM territory.
Which open models?
Llama, Mistral and Qwen are the established open-model families, each with sizes spanning laptop-runnable to server-class, and Hugging Face is the hub where models, variants and fine-tunes are published. Which family wins shifts with releases; check current community benchmarks for your size class rather than loyalty to a name.
Tool facts last checked July 2026