# Studies the gates stopped

Source: https://dinand.com/research/studies-the-gates-stopped/
Author: Dinand Tinholt

> Two pre-registered studies stopped on rules written before they ran. One of them also retracts a published claim.

## E10 hard mode: harder, or impossible?

Retraction · 22 October 2026

The published finding read a 44-point gap between easy and hard documents, 97.5% against 53.75%, and called it headroom. That claim is retracted. In 43% of the hard-mode documents the answer was no longer on the page, from the same document defect that affected Error Independence. Roughly three quarters of the apparent 44-point headroom is documents that cannot be answered at all.

A pre-registered pilot set out to measure how much room an agent had to improve on harder invoice documents. Its tranche pass rate of 0.5134 fired the kill condition, and the programme retired, which was the right call under the rule as written. Verification against the raw ledger reproduces every gate number and shows that the reading which killed it confuses difficulty with answerability: 173 of 400 tranche documents (43.3%) no longer carried at least one value the verifier demanded. On the answerable documents, hard mode reads 0.8293 against 0.9750 on easy, with overlapping intervals.

- [Difficulty That Removes the Answer](https://dinand.com/research/difficulty-that-removes-the-answer/)Paper, 2026-09-08
- [Harder, or Impossible?](https://dinand.com/research/harder-or-impossible/)Executive brief, September 2026

## The Refinement Tax

The campaign asked what share of an agent workflow's tokens and energy goes to verification, repair and coordination rather than the work itself. It ran for two hours and thirty-eight minutes and stopped on its own smoke gate at 05:40 on 9 September 2026. Neither model rung produced both correct and incorrect answers, so the gate had nothing to compare, and zero of 150 production tasks ran. Stopping early avoided a run of 9 to 11.5 hours against an 11-hour limit.

- [The Refinement Tax Pilot](https://dinand.com/research/refinement-tax-pilot/)Paper, 2026-09-09
- [What Measuring the Refinement Tax Costs](https://dinand.com/research/refinement-tax-brief/)Executive brief, 2026-09-09

## Correction note: the document defect

Correction · 22 October 2026

The invoice generator behind these studies adds scanner-style noise to each document. In the runs below it ran at nearly three times the intended noise rate, so some documents ask for a value the page no longer carries. A model scored wrong on those documents was transcribing the page correctly.

In [Error Independence](https://dinand.com/research/campaigns/error-independence/), 63 of 150 documents (42%) were affected. With them removed, the same-weights pair measures +0.521 against +0.447 as published, so that headline holds. In E10 hard mode, 173 of 400 documents (43.3%) were affected, which retracts the published 44 points of headroom.

---
Site guide for AI agents: https://dinand.com/llms.txt