Recently I wrote about my homelab "substrate" — a board of AI agents running on my own hardware, and why…

Recently I wrote about my homelab "substrate" — a board of AI agents running on my own hardware, and why the system around the models matters more than any single model. I mentioned, almost in passing, that I'd swapped the model class underneath them. This post is the why, because it's the part I get asked about most: how you actually choose the model.

Here's the thing nobody tells you. Choosing a model is an architecture decision, and you make it BEFORE you go chasing whatever topped the leaderboard this week. Decide the model class first. Then go shopping.

Let me show you why, the hard way.

My agents were running on hybrid reasoning models — the kind that think out loud before they answer. Great on paper. But on my agentic tool-calling path, that internal monologue leaked straight into the output and back into the context window.

So instead of a clean tool call, my agents were posting their own thought process. "Let me check the constraints... I will now post the comment..." The reasoning bloated the context, polluted the structured output, and added latency and token cost on every single hop.

I switched to an instruction-tuned, non-reasoning model — no narration, just the answer. Clean, direct, reliable tool calls. For agentic work, the model that just does the job beat the "smarter" one that narrates it.

Think of it like GDP. Productivity is useful output per unit of input — compute, tokens, latency, and the human cleanup tax. By that measure a reasoning model can win the benchmark and still deliver lower real productivity for your job, because it burns resources thinking aloud and then you pay again to clean it up.

So the decision driver isn't rank. It's three things: the use case, the output shape you actually need (tool-use wants clean deterministic calls, not an essay), and the resources you'll spend — a 16GB Mac is a very different budget to a GPU rig or paid cloud tokens.

Smartest model isn't the same as most productive model for this job on these resources.

How are you choosing: leaderboard rank, or fit for the output you actually need?

#HomeLab #LLMs #AIEngineering #LocalLLM #AIEfficiency