A long-running agent is a regression manager

A long-running agent is a regression manager

Patchwork, the book's fictional software-change agent, fixes a parser. That change makes two previously blocked jobs possible: a formatter can now consume the parsed representation, and an emitter can turn it into output. The formatter lands. Then, while adapting the emitter, Patchwork changes a shared token type and quietly breaks the parser.

If the controller thinks “long-running” means “allow more turns,” this looks like progress followed by a surprising failure. If it thinks in dependencies, the failure is less mysterious. Fixing the parser changed the set of work that was ready. It also created a permanent obligation: every later edit must preserve the parser behavior that made the formatter and emitter reachable.

That is the central idea I want to test: a long-running agent is managing both a changing dependency frontier and a growing regression suite. More context, more retries, or another worker may help it operate, but none of those mechanisms defines what is newly possible or what must remain true.

LoopsBench v2↗ makes this problem unusually inspectable in one benchmark. Its 112 coding tasks span eight programming languages and nine domains and contain more than 5,300 development units. Each task is represented as a directed acyclic graph, or DAG, whose nodes are separately testable units and whose edges encode source-evidenced prerequisites. The evaluation runtime releases tests along the ready frontier: a unit becomes ready when its predecessors have passed. Once a node passes, its tests remain active as regression obligations while later nodes are attempted.

The paper's results are sobering within that exact setup. Its strongest reported model-and-loop configuration resolved 25% of tasks. Recorded plans recovered only part of the source-recovered prerequisite DAG, and regression events remained visible across the evaluated loop profiles. The DAG is explicitly an evaluation contract, not a claim that real development is always monotonic or that the recovered order is uniquely optimal.

What the source shows is that this benchmark can expose ordering, partial planning, and backsliding that a terminal pass rate hides. The synthesis is mine: sustained agent control should treat “done” nodes as active invariants, not historical chat. At any checkpoint, the controller needs at least three distinct sets:

• obligations already satisfied and still enforced;
• work ready because its prerequisites are satisfied;
• blocked work with named unmet dependencies.

The sets are coupled. When a regression invalidates the parser, formatter and emitter work may need to move out of the ready set even if their own local edits are untouched. A credible progress record therefore cannot just count completed subtasks. It must say which observations established each completion, which later work depended on it, and which checks must be rerun after a relevant change.

Anthropic's first-party agent-evaluation guidance↗ offers useful production vocabulary for the second half of this idea. It distinguishes capability evals—what difficult work can the agent do?—from regression evals—does it still handle work it previously handled? The post is not evidence for the LoopsBench results or for my controller design. It supports the narrower operational distinction between expanding capability and protecting established behavior.

There is a fair countercase. A bounded one-shot task with no meaningful internal dependencies, no durable intermediate state, and no future edit that can invalidate earlier work may not need a frontier or an accumulating regression manager. A single environment check at the end can be enough. Adding a DAG and continuous regression machinery there would create ceremony without buying control.

What this does not prove: LoopsBench v2 does not establish that every software task has a recoverable DAG, that its source-derived edges are the only valid dependencies, or that a regression-aware controller will solve long-horizon work. Patchwork's parser example is an original teaching construction, not a LoopsBench task or reported experiment. The claim is narrower: when later work depends on earlier verified behavior, duration creates obligations that turn count alone cannot represent.

In your longest agent workflow, which “completed” step is still being enforced as a live obligation—and what downstream state changes when that step regresses?