Self-hosted AI used to mean either a beefy Mac running Ollama for personal experiments or a homelab full of consumer GPUs being coaxed into running 70B models with too much swap. By 2026 the picture is cleaner. The model landscape has stratified: small models (3–8B) run on any laptop with one-click installers; mid-sized models (13–32B) want a single consumer GPU or Apple Silicon with enough unified memory; and frontier-quality open weights (70B+) still demand multi-GPU rigs. The tooling has settled too — llama.cpp and its ports have become the universal substrate; Ollama is the friendly default for individuals; vLLM and SGLang are what you reach for when serving matters; and the surrounding ecosystem of UIs, agent frameworks, and fine-tune managers has consolidated around half a dozen tools that actually work in production. This page is a pragmatic field guide for someone setting up self-hosted AI in 2026: what to install, what hardware to buy, and what models to run on it. One deliberate omission: the cloud-API-bridged "hybrid" approach (LiteLLM proxies pointing at OpenAI under the hood) is not covered — that's a different product category and would dilute the focus on truly local inference.
A pragmatic 2026 field guide to running language models on your own hardware — runtimes, UIs, models, and the homelab itself.