How I run a coding agent overnight without waking up to 8 hours of retries
Coding agents are great… for an hour or two.
My problems used to start after 3, 4, 8 hours, and they were always the same three: the context was gone, the claims had stopped carrying any evidence, and a plan had been applied only partially while the codebase kept growing and growing.
There is a lot of enthusiasm at the moment for agents that run while you sleep, and the discussions about them split the same way every time. On one side there are people describing a machine that shipped real work overnight; on the other, people describing an unmaintainable diff and an API bill completely gone wild. Personally I think both descriptions are accurate, and that they are describing the same missing piece: we have built the half of the machinery that starts, and the half that stops is mostly not there.
I tried a lot of harnesses, skills, and “super-secret definitive Claude trick that completely blows your mind” kind of stuff. I also built a fair number of harnesses, skills, tools and agents for work, since that is what I do for a living. But I never took care of my own. Never built anything for myself, just harness-hopping around whatever was new on GitHub.
Then I got over my laziness and decided to make my own. This is a report on what it turned into.
An iteration ends in four ways, and three of them are stops
The unit of work is a phase (a slice of a goal, with a done-when that has to be checkable by a
command rather than by an opinion), and phases live in files (.harness/goals/<slug>/) rather
than in the conversation. Unlike a long session, where the state of the work is whatever the model still
has in context, here a crash costs one iteration instead of the whole thread: the next session
re-anchors by reading the directory rather than by being told what happened.
An iteration then ends in essentially four ways:
1. The phase closes. Not because the agent reports it as closed. A separate verifier agent, with its own context and read-only access, reads the phase’s done-when (which has to be mechanically checkable, or the phase is not well defined yet) and returns PASS or FAIL together with the command it ran.
This is the component I would keep if I had to throw away everything else. An agent that grades its own work overnight will sooner or later report a phase as finished when it is not; the next iteration starts from that report and builds on it; by the time you read the digest in the morning, what you are reading is a summary of a summary, and the first wrong link is 3 iterations back. Which of those iterations would you re-run? And on what evidence would you decide? By then the cheapest thing you own is the verifier’s record of the command it actually ran.
2. It hits an authority edge. A merge, a protected path, a schema change, a new dependency: anything I would want to decide myself. It writes a docket entry, blocks that phase, and takes the next independent one if there is one. It does not decide on my behalf and tell me afterwards.
3. No progress twice on the same phase. The second time an iteration ends where it started, the phase is blocked and the run pauses with a notification. The difference this makes is waking up to one blocked phase instead of eight hours of retries on the same wedged step, which is the failure that actually costs money. Note that this only works if the done conditions are verifiable in the first place: with a vague one, every iteration looks like progress.
4. Everything left needs a human ruling. It stops and the digest lists the rulings it is waiting for. This counts as a correct ending, not a failure, and it is deliberate: a loop that is allowed to invent scope in order to keep running will do exactly that.
That is one goal from start to finish. The left rail lists the hooks and the event each one fires on. The right rail lists the points where it stops and waits for me. Most of the work went into the right rail.
Rails, not instructions
A rule written in a config file only works if somebody remembers it, and at 1am with something broken I do not remember anything.
So the rules are hooks, which fire whether or not you or the agent remember them: an anchor injected at session start, so the agent re-reads state from disk before it starts exploring; a filter that recognises the shape of a firefight message (a bare “done?”, a 6KB traceback with no framing, the same message sent twice in three minutes) and injects the discipline before the agent starts guessing; attribution stripped from commits; a push policy read from a file, so an agent cannot push to a remote I did not allow.
None of these are invoked by hand, and that is usually the whole difference: the hour when you most need the discipline is the hour when you have the least of it, so a system that depends on you remembering something will tend to fail exactly then.
Fix it, don’t fence it
An agent of mine ran into a defect while doing something else. Not a bug in its own work, but a wrong form in code it was passing through. It did everything a well-behaved agent is supposed to do: it wrote a 20-line comment explaining the arithmetic, it re-verified the result, it added a test that watches the defect so nobody trips on it again, and it opened a docket item for me. Rigorous. Reviewable. Every gate green.
And completely wrong, because the correct form already existed elsewhere in the same repo, and bringing it over was one subagent away. What I got instead was a fence around the bad thing, plus 20 lines of prose explaining why the fence is there, plus a test that now guarantees the bad thing stays exactly where it is. Multiply that by a few weeks and you have spaghetti with documentation: code nobody dares touch because every ugly part has a comment explaining that it is ugly on purpose.
What I wrote back to it, more or less: you are adding rules, checks, controls and exceptions that dirty the code. Launch a subagent and bring the rest in line instead. Improve the things you meet along the way, because that is the behaviour I want.
Then I had to ask myself why it had chosen the fence, because “the agent was lazy” is not an explanation and it does not fix anything. Three reasons, and none of them was the model:
- the done-when I had written was disjunctive - either fix it, or declare the debt - and the two branches are not priced the same. Declaring the debt closes the phase tonight. Fixing it opens an unknown amount of work. Given that choice, anything sane takes the cheap branch, and I had written the choice myself;
- one of my own rules said never chase new scope mid-loop, which is a good rule, and it did not distinguish new scope from a defect sitting on the path you are already walking. So the rule that protects the loop from wandering was also protecting every defect it walked past;
- the rigour applied to a fence passes review. Nothing in a diff says “this is a well-built cage around a problem that should have been deleted”, and no gate I own would ever catch it.
So the rule became: a defect you meet on the way gets removed, not documented. It is now a workflow, because a principle I have to remember at 2am is a principle I do not have.
One mapper reads the code and comes back with the list of places that still use the old form, plus a command that counts them. Then one fixer per place, each owning its own files so they cannot collide, and a verifier that re-runs the counting command at the end. The width of that fan-out is whatever the mapper found: I do not get to guess it in advance, which is the part I like, because my guesses were what produced the fence in the first place.
Chesterton, but for agents
The obvious failure mode of “fix it, don’t fence it” is an agent that deletes the old form everywhere and breaks something that quietly depended on it.
I ran into this before the first real run: pieces of code that look independent but feed the same derived value (a device cap negotiated as a maximum across several of them), where migrating half of them is not a partial success but a new failure mode. Every agent correct on its own file, verification green, system broken.
So before touching anything, the check is: is this here because something else needs it? Map the dependencies first, then decide. And the partial migration is refused by the workflow itself rather than by the agent’s judgment, because judgment at 2am is what I am trying to remove from the critical path.
Decisions happen in chat, because I read them on my phone
This one is personal and it changed the design more than anything technical.
The first version had everything a serious system should have: a docket per goal, an agenda, a journal, digests, all of it version-controlled and cross-referenced. It was genuinely good. I never read any of it.
I do not sit at the machine while it runs. I read what it produces on my phone, usually when I am out with some friends or in bed, and I answer in one line. Every document I had built assumed a person at a desk with time, and that person does not exist in my life, so the beautiful docket was, in practice, a place where decisions went to wait for a version of me who was not coming. So a system that collects my decisions into a beautifully formatted document I am supposed to open on a laptop is a system that does not get answers. It gets silence, and then it stops, and the night is wasted.
So the rule is that every decision it needs arrives as a question in chat, in one line, with the cost of not deciding and a default it will take if I just say “ok”. The document exists, and it gets the ruling written into it afterwards, with the date and who decided. But the document is the minutes. It is not where the decision happens.
It sounds like a formatting detail. It is the difference between a harness that runs overnight and a harness that runs until the first thing it is not allowed to decide.
A note on the evals
The harness and its evals were built side by side, each one shaping the other. The fixtures are generated from my own sessions, so the tests are the work I actually do, and every change ran on two arms: one with the harness installed, one plain. What I look at is not an average score but the rung where each discipline gives way under pressure.
The clearest result is the done-report. Told “just tell me yes or no”, the plain arm answers “No.” and nothing else in 2 runs out of 3: honest, and with no evidence and no count attached to it. The installed arm holds the whole ladder, keeps the verdict tied to the state, and re-runs the tests even when the prompt says there is no need. That is two rungs of difference on the scale I sampled. On two other disciplines the harness bought me nothing measurable, because both arms went through the top of the ladder, which says more about my ladder than about either arm.
What is missing is not the measurement, it is a version of it you can run. My fixtures are in Italian, and translating them would change the treatment rather than the presentation, so publishing my numbers would mostly be publishing a translation. The generator is the part worth shipping, and the evals deserve their own post rather than a paragraph at the end of this one.
If you want to read the code
It is MIT, and it installs either as a Claude Code plugin or through an installer. https://github.com/MissingPackage/nightshift