Published on

Model + Harness = Agent

Model + Harness = Agent

Everyone is watching the models. The release notes, the Hugging Face leaderboard and the dozen others chasing it, the breathless threads about who edged ahead this week. But sit down to build something that actually works and you notice a strange thing: you spend almost none of your time on the model and almost all of it on everything around the model. The orchestrator. The tools. The memory. The context. The prompts. The part you didn't build and can't change is the model; the part you sweat over is the rest. An agent is a model plus a harness, and there's a line I keep coming back to: if you're not the model, you're the harness.

The model, on its own, does exactly one thing: it turns tokens into tokens. It has no memory of yesterday, no way to run its own code or check its own work. Everything you think of as "the agent" is the harness supplying what the model lacks — a filesystem so it can remember, a sandbox so it can act without breaking things, a loop that lets it try, look, and try again. Strip the harness away and you don't have a worse agent. You have a chatbot.

Which leads somewhere uncomfortable for the model-obsessed: the harness increasingly matters more than the model. A strong harness around a middling model beats a weak harness around a great one. The people at Databricks took a single fixed model and moved a hard document task from thirty-six percent to fifty-three just by improving the scaffolding around it. The intelligence was sitting there the whole time. The harness is what turned it into reliable work.

That should be encouraging, because the model is the one thing you can't win. Everyone rents the same frontier brain; you will not out-model the labs. But the harness is entirely yours — your context, your tools, your evals, your judgment about when the thing is actually done. It's the same reason the recipe is public but the cooking is proprietary: the commodity is the part everyone shares, and the craft is everything you build around it. Even orchestration is just harness at a larger scale — when you spawn a team of subagents, you aren't buying a smarter model, you're building a bigger harness.

There is one honest catch. Harnesses co-evolve with models, and they rot. Today's clever trick — compacting context, a planning loop, an elaborate permission dance — is tomorrow's built-in feature, absorbed into the model and quietly deleted from your code. Some of what you build now is scaffolding, and scaffolding comes down. But that's not a reason to skip it. Prompt engineering didn't die when models got better; it moved up a level. Harness engineering will do the same.