The night before I took my new multi-agent harness public, I did something a little obsessive: I pointed ten read-only agents at ten of my own repositories and asked each one the same question. What mechanisms live here that the harness should steal?
I expected to find a few utilities worth vendoring. A regex here, a retry loop there.
What I found instead was embarrassing, in the useful way. The same five safety mechanisms appeared in almost every repo, rebuilt from scratch each time, under different names, by past versions of me who apparently never compared notes. A PII scrubber in a commercial backend. A staleness tracker in a data reference site. A fail-closed overnight pipeline that refuses to start if any precondition is off. Hash-pinned manifests in a knowledge template. Eval identity records in a scoring project.
I had not designed a harness. I had been growing one for years, one habit at a time, and only noticed when I finally put all the pieces in one place.
The tool
Harnessie is a brain-agnostic multi-agent harness: one orchestrator, cheap swappable workers, independent verifiers, and verification gates between every phase. The model is a config entry, not a dependency. Swap Claude for Qwen on your own machine by editing one YAML file. All the gates, sandboxes, and budgets don’t care.
The pitch is simple: the harness structure carries the quality floor, the model carries the ceiling. But the honest version of the design story is those five habits. Here they are, in case they are useful to you even if you never touch my tool.
🔍 Habit 1: Deterministic checks run before any model judgment
Every gate runs mechanical checks first: does the file exist, do the tests pass, does the schema validate. Only then does an independent verifier model weigh in. The PII filter is the same idea: pure regex, under a millisecond, no model in the safety path. A filter with a model in it can be prompt-injected. A filter made of regex cannot be talked out of anything.
🧪 Habit 2: The eval comes before the implementation
A governance mechanic without a red-then-green scenario pair does not merge. Write the scenario that fails, then build until it passes. This sounds like TDD because it is TDD, applied to agent safety claims instead of functions. The payoff: every safety claim in the README points at a named test someone can run.
📅 Habit 3: Remembered facts expire visibly, never silently
Facts in project memory carry two dates: when they were verified and when they need re-verification. Stale facts get surfaced and archived, never deleted, and never trusted quietly past their date. I first built this for a site that tracks AI regulations, where a fact that was true in March can be false by June. Agent memory has exactly the same failure mode.
⛓️ Habit 4: Every action lands in a hash-chained timeline
Each event records a hash of the previous one. Agent actions and human approvals live in the same chain, so the audit answers “who authorized this” and “what happened next” in one place. Tamper-evident, not tamper-proof, and the docs say so plainly, because overclaiming security is worse than none.
🚫 Habit 5: A control that cannot be enforced fails closed
No usable sandbox on this machine? Shell tools refuse to run, rather than running unconfined. No spending ceiling configured? A live run refuses to start. The overnight pipeline that inspired this rule has a line I think about a lot: a run that cannot satisfy all its preconditions does not begin.
The part that went wrong
While building the red-team evals (fake canary credentials that must never leak into logs), I wrote a check that truncates leaked values in failure messages to 20 characters. My AWS-shaped canary was exactly 20 characters long. The “truncated” message would have contained the entire fake key. A test-of-the-test caught it before it shipped. Write tests for your safety checks; they are code too, and they are exactly as wrong as the rest of your code.
Tech stack
- Python 3.11+, stdlib-first (the model adapters need no vendor SDK)
- Any OpenAI-compatible endpoint: Ollama, vLLM, llama.cpp, or hosted
- OS sandboxing: Seatbelt on macOS; bubblewrap, firejail, or docker on Linux
- No daemon, no database. Files, YAML, and a hash-chained JSONL event log.
One tradeoff worth naming: everything fails closed, which means on a machine with no sandbox backend, shell-using workflows simply do not run. Some people will find that annoying. It is the point.
Try it
git clone https://github.com/snapsynapse/harnessie && cd harnessie
pip install -e ".[dev]"
python3 -m harness.cli eval # deterministic scorecard, zero dollars, no networkDocs live at harnessie.com.
Your turn
Here is the exercise I accidentally ran on myself: if you audited your last five projects, what safety mechanism have you rebuilt more than twice? That thing is probably your harness trying to exist. I would genuinely like to know what yours is.