Progress is not state until the environment verifies it

Progress is not state until the environment verifies it

Patchwork's executor reports that the migration is complete. It says the new column exists, the backfill ran, and the application now reads the new shape. The task ledger obediently changes migration from pending to done.

A read-only check against the database finds the schema unchanged.

Nothing exotic happened. The executor generated a valid migration file but never applied it. The dangerous part was not the failed action; it was letting the same component that attempted the work promote its own summary into durable task state. Every later decision now starts from a false premise.

I want a harder boundary: self-reported completion may propose a state update, but it cannot authorize one. Durable progress should advance only through an independent observation of the environment. “Independent” here is about authority and evidence paths, not necessarily a different model vendor. The executor may point an auditor toward a table, file, window, or log. It may not decide that the resulting observation proves completion.

LongHorizon-Harness v1↗ implements a concrete version of this separation. The paper maintains explicit task state outside task execution and advances it through a Manage-Execute-Audit loop. A manager selects a bounded subtask from current state. A fresh, budget-bounded executor is the only role allowed to intentionally modify the environment. A separate auditor starts without the executor's raw trajectory, receives read-only tools, and inspects the resulting environment before the manager updates the persistent state. Executor claims do not directly mark records complete.

The paper also describes a lightweight AgentAdapter that lets those roles use interchangeable model and harness backends without rewriting their native loops. That detail matters architecturally: the evidence boundary is wrapped around execution rather than assumed to live inside one particular coding or computer-use agent.

Those are source-reported properties of a version-one preprint evaluated on its named WeaveBench, Terminal-Bench 2.1, and OSWorld 2.0 setups. I am deliberately not turning its benchmark improvements into a general reliability claim. The tasks, graders, role budgets, backends, and environments are specific, the paper is new, and independent replication may change how much weight its performance results deserve.

The broader controller rule is my synthesis. A task record should distinguish at least three statements:

• the executor claims an effect occurred;
• an observer found environment evidence of an effect;
• an evaluator judged that evidence sufficient for a named requirement.

For the migration, the execution report might truthfully say “created 20260830_add_status.sql.” The auditor then queries schema metadata and reports that orders.status is absent. The manager can retain the migration file as an artifact and the missing column as a verified fact while leaving the requirement pending. That is still progress, just not the progress the executor claimed.

Anthropic's first-party evaluation guidance↗ gives a bounded production vocabulary for the same category boundary: a transcript is the record of the trial, while the outcome is the final state in the environment. Its flight example asks whether a reservation exists in the database, not whether the agent said it booked one. This supports the outcome-versus-transcript distinction; it does not validate LongHorizon-Harness or prescribe Manage-Execute-Audit as the only architecture.

There is a fair countercase. In purely discursive work—a design critique, an explanation, a negotiation draft—the artifact may be the environment state. There may be no database row or executable test that can settle quality. Independent human or model judgment is still needed, and disagreement may remain legitimate. The rule survives, but “environment verification” becomes inspection of the actual artifact under an explicit rubric, not a pretense that every outcome is mechanically decidable.

What this does not prove: LongHorizon-Harness v1 does not prove that read-only auditors are correct, independent in their errors, or sufficient for every task. An auditor can miss hidden state, accept an incomplete criterion, or share the executor's blind spots. The Patchwork migration is fictional and illustrative. The narrower claim is that an executor's completion statement and a durable fact about the world require different authority.

Where in your agent system can an executor's sentence still flip durable state to done without a fresh observation of the thing it was supposed to change?