Reporting Shouldn't Be the Model's Job

The harness files the report; the model is not asked to remember. A Stop hook fires when a Claude Code session finishes a turn and writes the report to a file, so the record survives compaction, exhaustion and an outright crash. We gave Claude Code sessions an org chart to test that idea. Then one live run falsified four assumptions our test suite had already blessed. If you run more than one Claude Code session at a time, you have almost certainly tried the obvious thing: tell one of them to go manage the others.

We tried it repeatedly, across different projects, and it failed the same way every time. Eventually we stopped treating it as a prompting problem and started treating it as an architecture problem. This is what we found, what we built, and — more usefully — the six harness behaviours no unit test could have caught, four of which surfaced only when a real session ran.

Three failures, not one

The obvious symptom is that subordinates don’t report. You tell a session “let me know when you’re done,” and it doesn’t. But that is only the first of three failures, and it is the least interesting one.

Subordinates don’t report. The instruction lives in the model’s context. Context gets compacted, exhausted, or simply outcompeted by the task itself. By the time the work is finished, the instruction to report has been squeezed out by four hundred lines of test output. Discipline is the wrong frame here. The instruction sits in a memory hierarchy, and no amount of better phrasing raises its priority.

Supervisors can’t observe. This one is easy to miss. A Claude Code session only executes while a user turn is running. Between turns it is inert — it isn’t polling, isn’t waiting, isn’t thinking. So even a subordinate that does report has nobody awake to receive the report. The message lands in a session that will not read it until you, the human, type something.

Nothing notices failure. Two sessions edit the same file and silently clobber each other. A session runs out of context mid-task and dies. Another gets stuck in a loop. From the outside, all three look identical to a session that is simply taking a while.

The third failure is the one that actually costs you time, and it is the one every “just tell it to report back” approach ignores entirely.

The reframe: reporting has to happen to the model

Here is the shift that made the rest tractable:

Reporting happens to the model. The harness files it when the turn ends, whatever the model’s context looks like.

If the report depends on the model choosing to file it, the report is only as reliable as the model’s remaining context. Move it out of the model’s control and it becomes as reliable as the process lifecycle.

Claude Code gives you exactly the hook for this. A Stop hook fires when a session finishes a turn, executed by the harness itself, and the hooks reference documents it as a lifecycle event of the CLI rather than a tool the model chooses to call. Whether the model remembered, ran out of context, or died halfway, the hook runs.

{ "hooks": { "Stop": [ { "hooks": [ { "type": "command", "command": "node", "args": ["${CLAUDE_PLUGIN_ROOT}/hooks/stop-report.mjs"], "timeout": 15 } ] } ] } }

The 15-second timeout is the safety budget the hook gets, and its real work is one appended line. That is the entire load-bearing idea. Everything else we built is scaffolding around it.

What we built

NestManager runs Claude Code sessions as an organisation, in three layers. The objective travels down the chart and results travel back up it, and you only ever talk to the Executive at the top.

• An Executive that owns the objective and answers to you.
• Managers that own workstreams and chase their own reports.
• Staff that do the work and settle overlaps with their peers.

One Executive fans out to two Managers, and each Manager to two Staff. A role is decided by position in the chart: a Staff member that hires its own team becomes a Manager for that subtree. Every member reports to exactly one supervisor, and escalation travels one hop at a time. Peers settle overlapping work between themselves, and only involve a Manager when they can’t.

Three pieces are worth explaining, because each one encodes a decision that could have gone differently.

State is an append-only event log. Up to a dozen OS processes write to the same directory at once. Rather than locking, every state change is a single appended line. POSIX is what makes that safe: with O_APPEND set, IEEE Std 1003.1-2017 requires the offset to move to the end of the file before each write and no intervening file modification to occur between that move and the write itself. The Linux manual page records the one documented exception, NFS, where appends can corrupt a file because the protocol has no append operation of its own. The org chart is never stored as a document; it is a projection folded from the log on read. A session that is force-killed cannot corrupt anything; worst case it leaves a stale working record, which is exactly what stall detection exists to catch.

Supervisors chase. Every supervisor classifies its direct reports on every turn, and escalates through a fixed ladder of three steps: nudge, read the transcript, reassign. The middle step matters more than it looks. A long test run and an infinite loop are indistinguishable from the outside; you have to read the actual output to tell them apart, and that judgement belongs to a model rather than to a timeout.

Budgets are chained and denominated in dollars. This one drew an objection from a colleague — for a subscription user, dollars are abstract, and tokens are the unit you actually feel. We still chose dollars, for two reasons. --max-budget-usd is the only cap the harness will actually enforce, and it takes a plain dollar figure — claude --max-budget-usd 5.00 is the example the CLI reference gives. A token ceiling is a number in a JSON file that nothing is obliged to respect. And tokens are not comparable across model tiers: the same million tokens differ in cost by roughly an order of magnitude between the cheapest and most capable model. A token budget therefore cannot bound a hierarchy whose entire purpose is mixing tiers — a manager could stay comfortably inside it while multiplying the bill tenfold. Tokens are recorded as a gauge; dollars are the constraint.

One assignment, end to end. A single assignment moves through five steps: the supervisor writes the brief, spawns the session, the session works, the session reports, and the Stop hook files a report regardless. Step 5 is the load-bearing part, because it exists so that step 4 is never something the system has to rely on.

The state a clock cannot see

Stall detection sounds like a timeout problem. It mostly is. But the most urgent state in the whole system is invisible to every silence-based check:

A member that reports blocked has just spoken.

Its last-heard-from timestamp is zero seconds ago. Every idle-time check rates it perfectly healthy. Meanwhile it has stopped working entirely and is waiting for a decision only its supervisor can make — and nothing will ever flag it, because it did the polite thing and told you it was stuck.

We found this in our own smoke test, before any of the harder bugs. A subordinate reported blocked: beyond_capability and the supervisor’s status view cheerfully listed it as OK. The fix is trivial once you see it; seeing it is the part that took a live run. Classification now reads the last event type before it reads the clock, so blocked outranks all three idle thresholds instead of racing them.

Then we ran it for real

At this point we had 74 unit tests, all green. CI passed on Linux and Windows, across two Node versions. The design held together on paper and in the test suite. We were confident.

Then we launched one real background session with a trivial task — write a file containing the word “hello” — and watched it fail four times in a row, for four unrelated reasons. None of them were coding errors. Every one was the harness behaving differently from how we had assumed, and every one failed silently.

1. --session-id is ignored for background sessions

We had designed identity around a simple idea: the parent assigns the child’s session ID upfront, so the org chart is written before the child exists and there is never a window where a subordinate is running but unregistered. Deterministic, clean.

claude --bg --session-id <uuid> does not use that UUID. The background session generates its own. The CLI reference lists --session-id as the specific conversation ID to use, and that description holds for a foreground run; the background path is where the two diverge.

The consequence was almost poetic. The Stop hook looked up the harness’s session ID, found no matching member in the org chart, and returned without filing anything. The one mechanism whose entire job was guaranteeing a report was the thing that quietly did nothing.

The fix: pass the parent’s assigned ID through an environment variable and treat that as authoritative, falling back to the harness ID only when it is absent.

2. A background session has nobody to answer a permission prompt

Our subordinate followed its brief perfectly. It read its assignment, understood the protocol, and ran its first command — nest inbox — exactly as instructed.

Then it stopped, forever, on a permission prompt. --permission-mode acceptEdits covers file edits but not shell commands, and there is no human attached to a background session to click “Yes.”

From the supervisor’s side this is indistinguishable from a stalled member. The subordinate is in fact doing everything right, waiting for an approval that cannot arrive.

The fix: pre-authorise the CLI at spawn time with --allowedTools. Note that you need both shell tools listed — which one a model reaches for is platform-dependent, PowerShell on Windows and Bash elsewhere. Listing both means one brief runs unchanged on either platform.

3. --allowedTools is variadic and will eat your prompt

Having added the permission rules, we spawned again. The session started and immediately reported: idle — send a prompt to start.

--allowedTools <tools...> is variadic. It keeps consuming arguments past a boolean flag. So this:

claude --allowedTools "Bash(node:*)" --bg "<the actual prompt>"

swallows the prompt into the tools list and launches a session with no instructions at all, waiting patiently for input that will never come.

It stops cleanly at an option that takes a value of its own, so this works:

claude --allowedTools "Bash(node:*)" --model haiku --bg "<the actual prompt>"

We now emit the rules first, immediately before --session-id, so correctness doesn’t depend on argument order elsewhere in the command.

4. Background sessions want a worktree before editing

Fourth attempt. The session announced: “I need to enter a worktree for this background session before making edits.” It tried to set one up, failed because our scratch repository had no commits (Failed to resolve base branch "HEAD"), then tried to git commit — which needed an approval nobody could give.

Partly our test environment’s fault: one commit in the scratch repository cleared the whole class of failure. The failure also reproduces from the Git side alone — git worktree add checks a commit-ish out into the new linked working tree, and a repository with zero commits has nothing for HEAD to resolve to. The underlying fact holds outside our scratch repository too: background sessions isolate writes in a worktree by default, so a spawn plan built around one shared working tree hits this path.

Two more behaviours we found by probing rather than crashing

--append-system-prompt-file was undocumented when we found it. It did not appear in claude --help, and probing it returned option '--append-system-prompt-file <file>' argument missing rather than an unknown-option error; the CLI reference lists it now, as loading additional system prompt text from a file and appending it to the default prompt. This matters more than it sounds: a role brief is about thirty lines containing quotes and newlines, and passing that inline breaks three separate ways on Windows — cmd.exe cannot carry newlines in an argument, quote escaping differs across shells, and the command line caps at 8,191 characters in cmd.exe, a ceiling the 32,767-character Win32 environment limit does not lift, because Command Prompt discards any inherited variable longer than its own 8,191. Writing the brief to a file and passing a path sidesteps all three.

--plugin-dir does not reach spawned sessions. Running a plugin from a clone applies to that session only. Sessions it spawns start with no hooks and no role skills — they ignore a protocol they were never handed. An installed plugin has no such problem, which is why installing properly is the recommended path and why the config carries an explicit forwarding option for the clone case.

The green suite wasn’t enough either

One more, because it cuts against the easy conclusion that “you just need live tests.”

Our CI caught a bug that our local runs never did. Events in the log were ordered by millisecond timestamp, with ties broken by a random event ID. On a fast machine a burst of events routinely lands in a single millisecond — so nudge, nudge, report could fold as report, nudge, nudge, leaving a member that had just reported still counted as un-nudged. The projection was non-deterministic, and it read differently on every load.

Locally, on Windows, 74 tests passed. On Linux, CI went red twice.

Finer-grained timestamps would not have fixed it. Appends are atomic, so the physical line order in the file is the write order across every process. Tie-break on line index and the ordering becomes both deterministic and causally correct, at zero extra cost on read.

What we’d take from four live runs

“AI-assisted development is unreliable” is the wrong reading. The code was the cheap part and it was mostly right.

The lesson is about where confidence comes from. A green test suite tells you the system behaves as you modelled it. It says nothing about whether your model of the platform is correct — and every one of the failures above lived in that gap. Unit tests couldn’t have caught them, because the tests encoded the same wrong assumptions as the implementation.

What worked was making falsification cheap and then doing it repeatedly. Four live runs, each taking a few minutes and costing almost nothing on the smallest model, found four architectural defects that 74 tests had blessed.

If you are building on top of an agent platform, budget for that loop explicitly. The interesting bugs live in the seam between your code and a harness whose behaviour you inferred instead of verifying. We now run the probe loop before every change to the spawn path.

We wrote up all eight harness behaviours in docs/harness-notes.md, separately from the design spec, because they are facts about Claude Code rather than decisions about our project. If you are building anything on Claude Code, that file is probably more useful to you than the project itself.

NestManager is MIT licensed and on GitHub: github.com/tecnonest/NestManager. It is early — version 0.1.0, and we mean the 0.x. Issues and contributions welcome, particularly from anyone who has hit these walls from a different direction.

Frequently Asked Questions

Why should a Stop hook file the report instead of the model?

Because the two fail for different reasons. An instruction to report lives in the model’s context and competes with four hundred lines of test output for space; a Stop hook is a lifecycle event the CLI runs when a turn ends, so it fires after a compaction, after a context exhaustion, and after a crash. Our first live run proved the difference in the worst possible way: the hook itself ran, looked up the wrong session ID, and filed nothing. We now pass the parent’s assigned ID in an environment variable and read the harness ID only as a fallback.

Why denominate agent budgets in dollars instead of tokens?

Because --max-budget-usd is the only cap the harness enforces, and because a million tokens are not one unit across model tiers: the same million costs tenfold more on the most capable model than on the cheapest. A manager running three subordinates on a top tier can stay inside a token ceiling while the bill multiplies. NestManager records tokens as a gauge and chains dollars as the constraint down all three layers.

How do you detect a subordinate that reported blocked?

No clock will find it: a member that reports blocked has a last-heard-from timestamp of zero seconds, so every idle threshold rates it healthy while it waits for a decision it cannot make alone. Classification reads the last event type first and treats blocked as an escalation, independent of elapsed time. We found the gap in a smoke test when a supervisor listed a member reporting blocked: beyond_capability as OK.

Do 74 green unit tests tell you an agent integration works?

They tell you the system behaves as you modelled the platform, which is a different claim. Our 74 tests passed on Linux and Windows across two Node versions while four assumptions about the Claude Code CLI were wrong: --session-id on background sessions, permission prompts with no human attached, variadic --allowedTools, and worktree isolation before edits. Each one took a live run of a few minutes on the smallest model to surface, and each one failed silently until then.

A version of this piece is also on Medium. NestManager itself is at github.com/tecnonest/NestManager.