“The harness matters” is not a book thesis

“The harness matters” is not a book thesis

Here is the uncomfortable realization: the field already knows that runtime, environment, tool, and evaluator choices matter. A different interface can change what a model manages to do. A different sandbox can change what it is allowed to do. A different grader can change what gets called success. Saying “the harness matters” is useful, but it is topic coverage, not differentiation.

Recent work makes that hard to miss. The AI Harness Engineering preprint↗ explicitly studies a model together with its harness and environment. Anthropic's managed-agents report↗ describes one production architecture that separates sessions, harness workers, and sandboxes. The OpenAI Agents SDK runner documentation↗ specifies one concrete loop for model calls, tool calls, handoffs, and stopping. These sources play different roles—a frontier research proposal, a named production report, and official documentation for one implementation. None hands us a universal architecture, and none proves the book's working thesis.

So the question I want to test is narrower: once a model proposal can touch the world, what must remain explicit for the resulting trajectory to stay controllable?

The current answer is four contracts.

The transition contract says who may propose an effect, authorize it, execute it, observe it, stop, and retry. The state contract says what survives a timeout, how work and artifacts are identified, and what must be reconciled before resumption. The evidence contract says which observations justify “done,” which collateral state must also be checked, and what uncertainty remains. The change contract says which model, prompt, tool, policy, evaluator, memory, workflow, or harness version produced the run—and how a candidate enters or leaves service without rewriting history.

That sounds abstract until an effect escapes the process. Suppose an operations agent proposes a $40 customer credit. Policy authorizes up to$50, the billing API receives the request, and the client times out before the response arrives. The transition contract distinguishes proposal and authorization from execution and observation. The state contract preserves an operation identity and a pending-effect record. The evidence contract requires a billing-system observation before calling the credit applied. The change contract tells us which policy and tool adapter were active if this failure appears only after a rollout.

Without those distinctions, “the agent issued a credit” compresses several incompatible realities. It may mean the model wrote a persuasive sentence. It may mean a request left the process. It may mean the customer ledger changed. It may mean an evaluator inferred success from the transcript. Those are not the same event.

Patchwork, the book's fictional pedagogical software-change agent, is where this frame will be made executable. It starts with a simpler external effect: editing a repository. The four contracts are intended to help a reader locate the earliest broken responsibility without turning the book into a framework tour.

But this is a book-synthesis being tested, not a standard. Four is not a magic number. A system may combine session and harness ownership, embed evaluation in an environment, use deterministic software instead of a model, or need a responsibility this frame hides.

What would break it? I am looking for a concrete agent-system failure where every transition, durable state boundary, evidence rule, and component change is correctly specified, yet the failure still cannot be localized without adding or replacing a contract. What is that counterexample?