The shelf

What I actually run.

No 40-line software lists. This is the small number of tools I'd recommend if you sat next to me, with the commands that get you from zero to chatting in an evening.

⚙️ Runtimes — pick one

Tool	Best for	Why I like it
Ollama	Fastest start, day-to-day chat, scripting	One install, one line per model, OpenAI-compatible API out of the box. My default.
LM Studio	GUI users, discovering models	Beautiful browser for the model ecosystem, built-in chat with tunable settings, GPU detection.
llama.cpp	Tinkering, maximum performance	The engine under most of the rest. Every knob: context, threads, offloading, sampling.
vLLM	Serious serving, multi-user	Production-grade throughput if you're hosting a model for a team or app.

The whole evening, start to finish:

# 1. Install Ollama (macOS/Linux; there's a Windows build too)
curl -fsSL https://ollama.com/install.sh | sh

# 2. Pull a solid generalist (8B-class, 4-bit)
ollama pull llama3.1:8b

# 3. Chat
ollama run llama3.1:8b

# 4. Call it from code via the local OpenAI-compatible API
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3.1:8b",
       "messages": [{"role": "user", "content": "Explain KV-cache in two lines."}],
       "temperature": 0.2}'

That last command is the doorway to everything: the same /v1/chat/completions shape the cloud uses, pointed at your desk.

🏛️ Where models come from

Source	What you find there	Hint
Hugging Face	Everything, in every format (GGUF, safetensors…)	Filter by `gguf` and sort by downloads; check the "used by" / quant size before downloading.
Ollama library	Curated, correctly-tagged builds of popular models	Right answer for 80% of people; fewer surprises.
Official model repos	Original weights + the system prompt / chat template	Read the model card first — that tip #8 material lives there.

My standing shelf (names generic on purpose — the families matter more than any single release, and they all age fast): a generalist 8B–14B for chat and writing, a coding-tuned model in the 7B–14B range, and a 3B-class model for fast background jobs. Each family's reputation shifts every couple of months; that's half the fun.

🖥️ My actual setup, plainly stated

Layer	What I run	Notes
Hardware	Apple Silicon laptop (unified memory) + one older GPU box for the big models	Unified memory on Apple Silicon is a local-LLM cheat code — no "VRAM" ceiling to hit.
Runtime	Ollama (daily) · LM Studio (discovery) · llama.cpp (benchmarks)	Ollama runs overnight as a service; the others are for when I'm experimenting.
Chat UIs	A couple of local web UIs pointed at the Ollama API	Anything that speaks the OpenAI API works; I care more about UI comfort than the brand.
RAG	Small local embedding model + a vector store of my markdown notes	Chunked on headings, re-ranked, paths stored with chunks — the whole lesson from the June entry.
Evals	A 60-prompt harness + a judge model, run on every release I care about	Boring, repeatable, and it has saved me from three bad default choices.
Routing	A ~100-line script: local by default, cloud escalation for deep reasoning	The hybrid setup from the October entry — and the reason my API bill collapsed.

One honest warning: this field moves fast. Model names and recommended sizes on this page have a shelf life of a few months; the principles (quant math, per-task model matching, eval before tune, route by task) are the durable part. Re-check current model recommendations on the day you start.