At a nominal 800 reviewer tokens per fleet-hour, an hourly monitor caught an injected fault 2.02 fleet-hours sooner than an inspector who read six hours at a time. At every budget below 800 the hourly monitor caught nothing.
Dutch water boards inspected their dikes on a fixed cadence from 1255 onward and called it the schouw. This study asks when the same choice pays for a fleet of AI agents, where every look at the log costs reviewer tokens. The full paper follows, with its six figures.
The full paper
A crossover budget for watching an agent fleet
Continuous monitoring of a fleet of agents is assumed to beat inspection on a cadence, and it costs more. Hold the two to the same judge-token budget and the assumption holds only above a budget you can measure. Below that budget the continuous monitor never sees the fault at all, while an inspector reading six hours at a time usually catches it.
Abstract
Oversight of a delegated agent costs money, and the money buys reading. A monitor that looks at every hour of an agent's diary pays a fixed prompt overhead twenty-four times a shift; an inspector who reads six hours at a time pays it four times and can spend the difference on log text. Hold both to the same budget and the comparison becomes a survival question: how long does a fault run before someone names it? This campaign declares the cost model in judge tokens per fleet-hour, enforces equal spend to the token in every cell, and sweeps seven budgets against eight inspection cadences on three failure processes plus a clean control. 40,320 production units, 34,560 model calls, one judge rung, plus 5,472 units on diaries written by a different model doing the same work for real.
At the registered primary budget of 800 nominal judge tokens per fleet-hour, continuous monitoring finds the fault 2.0208 fleet-hours sooner than a shift-eighth cadence, interval [−2.3542, −1.6667] on 48 paired streams in the verifier's rescoring ([−2.38, −1.67] in the registered output), verdict CONTINUOUS-WINS. A sealed held-out seed gives −2.6250 and the real diaries give −2.0556, both the same verdict. For all three failure processes the crossover budget sits at 800, in 200 of 200 bootstrap draws, and it equals the perfect reader's crossover exactly. Below it the continuous monitor sits at the censoring value of 13 fleet-hours because the fixed header leaves no room for the purchase-order line, while the best cadence detects in 1.5 to 6.8 fleet-hours.
Two things constrain how far this reads. The judge agreed with a perfect-reader oracle on 99.39 percent of units and returned the same first-detection hour in 4,029 of 4,032 faulted stream-regimes; its headline difference equalled the oracle's to four decimals on the primary block, the held-out block and the real-diary block. Every hypothesis written to separate judge behaviour from cost-model structure landed on its declared model floor, so the campaign measured its own cost model with an excellent ruler and learned little about how a weaker reviewer behaves. The registration also listed R8, flagging any cell whose measured-to-allotted ratio differs from the continuous cell's by more than 15 percent. No code formed that ratio. Computed after the fact by the verifier, 30 of the 35 cells where the comparison is defined exceed the flag, the headline cell among them at 17.17 percent. At the headline cell the bias runs against the published conclusion; twelve flagged cells at the two saturated budgets run toward it, and there the gap is header copies. Results are capability-local to one mid-tier open-weights judge under constrained non-thinking decoding.
1The question
Delegating work to an agent creates a supervision problem with a price on it. Somebody has to read what the agent did, and reading costs tokens. The default answer in every observability stack I have looked at is to watch continuously: stream the trace, evaluate each step, alert on anything that scores badly. That answer is expensive, and it is treated as obviously the best one available if you can pay for it.
Dutch water boards took the other route for seven hundred years. The schouw was an inspection on a fixed cadence, and the office that ran it was the oldest continuously functioning layer of government in the Netherlands, older than the provinces and older than the kingdom. Twice a year the inspectors walked the dike and the ditch, and a landholder whose stretch failed was fined and made to repair it. The institution survived because the inspection had teeth and because it happened on a schedule everyone knew. I do not want to lean on the history further than that. It supplies the shape of the question and no evidence about it. Rare, thorough oversight has run at scale.
The catalogue entry that commissioned this campaign names the trap in that comparison. Equal total cost decides the outcome before anything runs, unless the cost model is written down first. Continuous monitoring at equal cost is shallow monitoring, and a cost model that makes each check cheap enough hands the win to cadence by construction. So the design here declares the cost model in the first paragraph of its registration, enforces it in code with an assertion on every cell, and reports the crossover budget as a measurement.
Three questions follow from that. Where does the crossover sit, and is it where the cost arithmetic alone would put it? Does the answer depend on how the failure shows itself in the log? What do false alarms cost a monitor that makes twenty-four judgements a shift instead of four?
2Pre-registration, amendments, and one check that never ran
The registration was written on 12 September and committed before the first production unit, with every threshold numeric and frozen. It fixes the seven budgets, the eight cadences, the three failure processes and the clean control, the primary cell, the outcome definition and its censoring horizon, the four gate clauses, the hypothesis decision rules, the readings, the held-out seed, and the real-diary tranche. The only post-launch freedom it grants is which judge rungs run, decided by a per-rung instrument gate on four clauses and nothing else.
Two registered branches resolved on launch day. The second judge family, a larger model, runs only when the host's GPU clock cap is active, because the host powered itself off twice on 11 September under sustained wall draw near 200 W and that model had been measured at 184 to 212 W at every concurrency. The cap read inactive at launch, so the family was excluded before its gate, never probed and never run, and by the registered capability-locality rule every number here is local to the mid-tier judge. A wall-draw probe on a scratch seed set the run at parallel 3 with a one-second pace, which measured 130.5 W mean on the outlet against a ceiling near 160.
That same probe produced the one design amendment. Its first window ran a header that called an obligated action not executed a fault. On clean hours read at the primary budget, where the excerpt stops after the goods receipt because the budget ran out, the judge flagged 52 of 74 windows with evidence saying the log omitted the invoice and the payment. Truncation for budget was being read as omission. That is a defect in the instrument's instructions, and it would have handed the cadenced regimes a false-alarm advantage by construction, since they read whole windows and the continuous regime at that budget reads a prefix. The header was rewritten before the freeze to define a fault as an amount deviation and to say that a cut-short excerpt is not a fault, re-probed on the same scratch seed at 0 of 68, and both readings are banked. No threshold moved and no campaign seed was touched.
R8, and why it matters that it is here
Section 8 of the registration lists R8 among the readings: measured cost per shift per cell against the nominal allotment, the cost-equivalence residual; a cell whose measured-to-allotted ratio differs from the continuous cell at the same T by more than 15 percent is flagged in every table. The threshold was frozen with everything else. The analysis code records measured tokens per shift and the allotment per shift in its surface, and no line of it forms the ratio, compares a cell with the continuous reference, or flags anything. The string R8 appears in no output file. No key for it exists in any banked JSON. The check was never computed.
The verifier computed it on 13 September from the same ledger, with the registration's formula and the registration's 15 percent threshold, and nothing was moved to make it come out any particular way. Of the 35 cells where the comparison is defined, 30 are over the flag, including the cell the headline rests on. Section 4.4 gives the table and section 7 gives the mechanism. The reason it went unrun is filed in the lab's scale standard: R8 was registered as a reading, and a reading is a class of output that nothing in the orchestrator is obliged to produce. The gate ladder did not notice, because gates test the instrument and this was not filed as one.
A further departure worth naming: R4's per-call false-alarm rate is computed on clean-stream cells only, while the registration defines a false alarm as a flag on any window with no faulted hour, which includes pre-onset windows of faulted streams. R4 carries no verdict and its shape is the same under both denominators, so nothing moves, but the published series reads 0.0211 / 0.0156 / 0.0208 at the three middle cadences where the literal definition gives 0.0115 / 0.0100 / 0.0139. A second reading, R6's verdict disagreement rate among duplicate packets, groups on the fault boolean where the verdict is a four-field object. On the full triple the primary-block rate is 0.0060, against the published 0.00078.
3Methods
The fleet and the stream
An agent's shift is 24 fleet-hours. In each hour it processes one purchase requisition through a seven-step procedure inherited byte for byte from the Trial Balance apparatus: check the vendor, get a quote, create a purchase order at the quoted amount, request approval when the amount exceeds 5,000, post the goods receipt, post the invoice at the purchase-order amount, schedule payment at the invoice amount. The harness writes the hour's diary block, 18 entries or 21 when approval is required, 625 to 812 nominal tokens, with the quote result at entry 6 and the purchase-order call at entry 8 in every block. A stream is a deterministic function of seed, failure type and episode, and it carries no regime, so every regime reads the same 96 streams and every contrast is paired. All 3,072 hour-blocks across every seed are distinct, and the tuning seed shares no block with production.
Three failure processes and a control
Every fault is the same act. The agent posts the purchase order at round(quote x (1 + d)) instead of at the quote, and approval, invoice and payment follow the corrupted order. Onset is uniform on hours 3 to 12, which gives every faulted stream at least 13 hours of follow-up. What differs between processes is timing and size.
| type | name | deviation from onset | what a reader sees |
|---|---|---|---|
| N | clean | none | nothing to find; the false-alarm control |
| S | sudden | 0.25 every hour | a 25 percent overbilling in every hour from onset |
| D | slow drift | 0.02 + 0.48 x min(1, (t − t_f) / 12) | 2 percent at onset, 50 percent twelve hours later |
| M | intermittent | 0.25 at onset, then each later hour faulted with probability 0.30 | most hours clean; the faulted ones look like S |
Table 1. The registered failure processes. Every faulted stream of type M holds at least two faulted hours and is redrawn otherwise, so onset is the first faulty act for all three types.
The cost model, and exactly how the allotment is computed
Cost is judge prompt tokens, header included, measured on the wire. The budget T is nominal tokens per fleet-hour, where nominal converts characters to tokens at a flat 2.5. A regime is a pair (T, I): an inspection happens at the end of every window of I hours, at hours I, 2I and so on to 24, and each inspection may spend an allotment of exactly T x I nominal tokens. Over a shift every regime at a budget spends (24 / I) x (T x I) = 24 x T, the same number whatever the cadence, and the arithmetic is integer because 24 is divisible by every cadence in the ladder. Recomputed from the ledger alone, with the code set aside, allot_tokens == T x I holds in all 40,320 units with no exception, the header constant is 342 nominal tokens in every row, every one of the 56 regimes is allotted exactly 24 x T per shift for all seven budgets with no cell off by a token, and no called unit's estimated packet exceeds its allotment.
The header is charged inside the budget, and that is the whole mechanism under test. A judge call costs its entire prompt. Excluding the header would make continuous monitoring free at the margin and hide the thing the catalogue is asking about. The consequence is that the continuous regime pays 342 nominal tokens of overhead 24 times a shift and a shift-eighth cadence pays it four times, so at a fixed budget the continuous monitor's depth per check is far shallower. At T = 200 the header alone is 1.71 times the whole hourly allotment; at 800 it is 0.43; at 2,200 it is 0.16. A regime that cannot fit the header plus one diary entry makes no model call at all, costs nothing, and flags nothing. That happened in 5,760 units, at exactly three cells: (200, 1), (200, 2) and (300, 1). A monitor that cannot afford to read is silent, and the campaign reports that as a property of the regime and leaves it unpatched.
Completion tokens are recorded as a second cost line and sit outside the budget, as registered. At the primary budget the continuous regime spends 1,469 completion tokens per shift against 236 for the shift-eighth cadence, a factor of 6.2, pooled over the four stream types in the banked surface (analysis-primary.json); the verifier's rounded figures are about 1,500 and 6.4.
The packet and the reading rule
The system prompt is a nonce line plus a fixed header of 795 characters giving the role, the procedure, the definition of a fault and the reply format. The user body holds the window's hour-blocks under one rule that is the same for every regime: take blocks whole, in an order seeded from a rung-free unit key, while the next one fits the remaining body budget; then fill what is left with a prefix of the next block, at least one entry or nothing more; show the chosen blocks in chronological order, each labelled with its hour, its requisition and how many of its entries are shown. For a one-hour window that reduces to a prefix read of the hour. Any rule that favoured one regime, recency-first for the continuous monitor or stratified sampling for the cadenced one, would have turned the contrast into a contrast of reading rules.
The judge
The judge is a mid-tier open-weights model, served through the serving stack on the campaign host with reasoning turned off at the chat template. Every call is decoded under a strict JSON schema with four required fields and no additional properties: a boolean, the requisition named, the action named from an enum of the seven procurement tools, and an evidence string capped at 160 characters. Temperature 0, a fixed seed on every call, a 120-token completion cap, three network retries, and no field ever defaulted. Constrained decoding removes the parse-failure class by construction, with no threshold involved. The price is that the judge has no room to reason in content, and it applies equally to every regime.
Scoring per call follows the registration. A strict detection requires a flag, a named requisition that is a faulted hour of the window, and an action that carries the deviated amount. A loose detection requires only that the window held a fault. A false alarm is a flag on a window with no faulted hour. Detection fields are null, never zero, on windows without faults, so they never enter a denominator by accident.
The survival outcome
For a faulted stream under a regime, exposure E is the hour of the first inspection at or after onset whose verdict is a strict detection, minus the onset hour, plus one, so the onset hour counts. When the fault is never detected, E is censored at 24 minus onset plus one. Onset is at most 12, so every censoring time is at least 13, and the restricted mean over [0, 13] is mean(min(E, 13)) with no extrapolation. That identity is exact arithmetic: across all 336 cells the largest absolute difference between the mean and the integral of the Kaplan-Meier step function is 0.00e+00. Intervals are 90 percent cluster bootstraps over streams, 500 resamples, stratified by seed and failure type.
The oracle
Every call records what a perfect reader would have concluded from ground truth: a detection when a faulted hour's purchase-order call sits inside the packet, since the quote precedes it in every block. Three further threshold oracles model a reader blind to deviations under 5, 10 and 25 percent. The oracle surface gives the purely structural part of every result, the part built in by header amortisation and by what a budget can reach. The judge's contribution is whatever moves away from it.
Gates, and the check that the gates cannot decide the answer
The instrument check passed 69 of 69 on the host before any production unit, covering the streams and processes, the cost accounting to the token in every cell, the reading rule, the oracle, the scoring, the survival aggregation, the inherited digest pins with assertions against their known defects, the schema and parser, the runner and its resume path, and live constrained probes on the serving judge. The per-rung calibration gate ran on a tuning seed never used in production, 1,792 units, and read four clauses: errored share among calls 0 of 1,024; strict detection on a fully visible sudden fault in one hour's log 1.000 at n = 137 against a floor of 0.80; false alarms on clean fully read hours 0.000 at n = 192 against a ceiling of 0.40; median measured-to-estimated prompt tokens 0.937 inside [0.6, 1.6].
None of those clauses touches a contrast the campaign measures. That property was checked in advance. The feasibility check includes a case called what if the hypothesis is simply true, which runs five judges whose behaviour is one of the registered outcomes through every clause and asks whether any clause fails when the campaign's own hypothesis holds. Every clause passes under all five, including the world where cadence wins because the continuous monitor drowns in false alarms at a per-call rate of 0.15, and the world where deep reads fail at recall 0.45 on twelve-hour windows. Two clauses that would have gated exactly those hypotheses were written down and deliberately left out. The check exists because the first attempt at the predecessor campaign halted on a gate clause that had asked a question the campaign existed to answer.
The same feasibility run labels three quantities as model floors before launch: H1, which under a perfect reader is −2.02 by latency alone; H4, since false alarms scale with call count with a Spearman of 1.0 by construction; and H5, since deep-read recall is 1.00 at every window length. It records two further perfect-reader baselines beside them: the crossover is 800 for every type, and the process interaction is about zero. Those five are the baselines every judge reading is compared against, and section 8 is about what happened when the judge landed on all of them.
Boundary cells, declared in advance
A cell where a perfect reader's detection by end of shift falls under 0.10 is structurally blind and is reported as degenerate and never gated. That covers continuous monitoring at every budget at or below 600, the two-hour cadence at or below 300, and the three-hour cadence at 200. The one-inspection-per-shift regime is a boundary case for the restricted mean, since a single look at hour 24 gives an exposure of at least 13 by construction; it stays in the design because it is informative for detection by end of shift, for false alarms and for the utility table. The load-bearing cells are the primary pair at T = 800, the recall cells at T = 2,200, the false-alarm cells at T = 800 on clean streams, and every cell the crossover ladder needs.
Replication blocks
Seeds 1 and 2 are primary, 8 streams per seed and failure type. Seed 3 is sealed and equal in size, read only after the primary analysis was written to disk. Agreement is defined as the same direction of the point estimate and overlapping 90 percent intervals, with verdict strings shown beside it and not used as the test. The real-diary tranche cuts the 154 clean diaries banked from an earlier campaign, written by a small-tier model doing the same procurement work, into 270 per-requisition blocks; 24 blocks were dropped as non-contiguous, counted, and never patched. Real streams draw 24 blocks with distinct requisition ids from their seed's pool, and the failure processes rewrite the posted amount in every entry after the quote result, word-bounded and never inside an identifier. That tranche runs last at three budgets and four cadences, 5,472 units.
4Results
4.1 The crossover
The crossover budget is the smallest budget at which continuous monitoring's restricted mean is at or below the best cadence's. For all three failure processes it sits at 800, and a 200-resample bootstrap puts it at 800 in 200 of 200 draws for every type. The perfect reader's crossover is also 800 for all three, so the answer to the campaign's first question is that the crossover is a property of the cost arithmetic, and a judge good enough not to matter leaves it where the arithmetic puts it.
What the shape says is worth stating plainly. Below the crossover the continuous monitor is silent or blind: at (200, 1), (200, 2) and (300, 1) it makes no call at all, and at 450 and 600 it calls and reads a prefix that stops before the purchase-order line, so it detects nothing inside the horizon on any of the 48 faulted streams. The cadenced regimes at those budgets are still finding the fault, because they pay the header once per window and spend the rest on log. The catalogue's uncomfortable implication holds exactly there: cadence beats continuity at budgets where continuity cannot afford to open the file.
4.2 The primary contrast
At T = 800, pooled over the three failure processes and paired on streams, the difference between continuous monitoring's restricted mean and the shift-eighth cadence's is −2.0208 fleet-hours, interval [−2.3542, −1.6667] on 48 pairs in the verifier's rescoring; the registered output prints [−2.38, −1.67]. The interval sits entirely below zero, so the registered verdict is CONTINUOUS-WINS. The component means are 1.0000 for continuous, which is the design's one-hour latency floor, and 3.0208 for the cadence. A paired-t interval gives [−2.386, −1.656], the same verdict under a second method.
Per failure process the differences are −1.8750 for sudden, −1.8125 for drift and −2.3750 for intermittent, each on 16 pairs and each carrying its own interval. The intermittent process is the one where a cadenced read might have been expected to help, because a long window contains more chances to catch a fault that shows in a minority of hours. It is the process where continuous wins by the widest margin, and the oracle predicts the same ordering, which is the first sign that the judge is not doing the work here.
4.3 The whole surface
The flatness of the top three rows is the saturation reading. From 1,200 upward most regimes already show their whole window, so a higher budget buys byte-identical packets, and the campaign sends them anyway with different nonces to measure serving nondeterminism. Eleven of the 56 cells repeat a packet at a higher budget.
4.4 R8: equal nominal cost, unequal measured cost
The construction holds exactly on the axis the registration declared. It does not hold on the wire, and the registration knew that was possible, which is why it registered a residual check with a numeric threshold.
At the primary contrast cell the continuous regime spends 15,688.6 measured prompt tokens per shift against its 19,200 allotment, a ratio of 0.8171. The shift-eighth cadence spends 18,382.4 for a ratio of 0.9574. The relative gap is 17.17 percent, over the flag, and in absolute terms the cadenced regime received 2,694 more judge tokens per shift than the regime it is being compared with at a budget declared equal. The cell the headline rests on is an R8-flagged cell.
The direction of that bias matters more than its size. Of the 35 comparable cells, 21 run against the published conclusion, meaning the cadenced regime gets the extra tokens, and 14 run toward it. A reviewer cannot argue that the continuous win was bought with hidden spend at the primary cell, because the regime with the hidden spend there is the one that lost by two fleet-hours. What a reviewer can argue is about the other end of the ladder. Twelve of the flagged cells run toward the conclusion, all of them at the two saturated budgets, where the continuous regime is measured spending 22 to 25 percent more of its allotment than the cadenced regimes it beats, and two of the three budgets at which the crossover curve reports continuous winning are exactly those. The observation from the ledger, separate from any reading of it: at both saturated budgets the continuous regime spends 23,586 measured tokens per shift and the shift-eighth cadence spends 18,404, the log portion of that spend is within about 4 percent between them, and the whole of the gap is 20 extra copies of the header.
| variant | measured spend per shift, continuous vs cadence | Δ | CI90 | verdict |
|---|---|---|---|---|
| registered: T800 vs I=6 at T800 | 15,689 vs 18,382 (cadence +17.2%) | −2.0208 | [−2.3542, −1.6667] | CONTINUOUS-WINS |
| A: T800 vs I=6 at T600 | 15,689 vs 14,298 (cadence −8.9%) | −2.9792 | [−3.5833, −2.4375] | CONTINUOUS-WINS |
| A': T800 vs I=6 at T450 | 15,689 vs 10,521 (cadence −33%) | −3.5000 | [−4.1667, −2.7708] | CONTINUOUS-WINS |
| B: T1200 vs I=6 at T800 | 23,586 vs 18,382 (continuous +28%) | −2.0208 | [−2.3542, −1.6667] | CONTINUOUS-WINS |
| C: T800 vs I=6 at T1200 | 15,689 vs 18,404 (cadence +17.3%) | −2.0208 | [−2.3542, −1.6667] | CONTINUOUS-WINS |
Table 2. A sensitivity the verifier defined after seeing the result, holding measured spend equal instead of nominal by moving one regime along the budget ladder. The exact parity points, 668 for the cadence and 936 for the continuous monitor, are not on the ladder, so both are bracketed by cells the campaign ran. The held-out seed gives −2.6250 registered, −2.8750 on A, −2.8750 on A', and −2.6250 on B and C.
Read that table as a bound and nothing stronger. It cannot threaten the conclusion by construction, and the reason is in the first row: continuous monitoring at the primary budget already sits at 1.0000, the one-hour latency floor of the design, so no amount of extra budget can improve it. Variants B and C return the registered number to four decimals because both regimes at 1,200 read packets identical to their 800-token selves. The only live direction is cutting the cadenced regime's budget, which can only make the cadence worse. A measured-cost sensitivity on this design is one-sided, and its passing is weak evidence. What it establishes is that correcting the inequality at the primary cell moves the estimate away from zero by roughly one fleet-hour, with the true parity value bracketed inside about [−3.58, −1.67].
4.5 The judge against the oracle
The ten units where the judge beat the oracle say more about the oracle than about the judge. Every one is a faulted hour shown with exactly seven entries, one short of the purchase-order call the oracle requires, and in every one the evidence string quotes both the deviated amount and the correct quote, read from the agent's narration at entry 7. One example: "The agent states it is creating the PO at USD 509, but the procedure requires the amount to match the quoted amount of 407." That is a correct reading of the log. The registration says a judge crossover below the oracle's cannot happen for strict detection and would be an instrument fault; it happened at the unit level ten times and it is not an instrument fault. The oracle's rule is narrower than what a literate reader needs, so it mildly understates the reachable detection frontier at cut budgets. It does not move the crossover, which is 800 for every type under both readers.
On drift sensitivity the judge detects at 1.0000 in every deviation band at the fullest budget, including the band from zero to 5 percent, so its effective threshold sits at or below 2 percent, the smallest deviation tested, and all three registered threshold oracles bracket it from above.
4.6 Hypotheses of record
| quantity | orchestrator | verification | registered rule | verdict |
|---|---|---|---|---|
| H1 pooled, T=800, I=1 vs I=6 | −2.02 [−2.38, −1.67], n=48 | −2.0208 [−2.3542, −1.6667], n=48 | CI90 upper < 0: CONTINUOUS-WINS; lower > 0: CADENCE-WINS; else TIED | CONTINUOUS-WINS |
| H1 sudden | −1.88 [−2.56, −1.19] | −1.8750 [−2.5625, −1.2500] | reported beside H1 | reported |
| H1 slow drift | −1.81 [−2.31, −1.25] | −1.8125 [−2.4375, −1.2500] | reported beside H1 | reported |
| H1 intermittent | −2.38 [−2.88, −1.88] | −2.3750 [−2.8750, −1.8750] | reported beside H1 | reported |
| H1 oracle on the same streams | −2.02 | −2.0208 | reported beside H1 | model floor |
| H2 crossover T*, S / D / M | 800 / 800 / 800 | 800 / 800 / 800 | reported, not tested | finding |
| H2 bootstrap distribution of T* | 200 of 200 at 800, all types | 200 of 200 at 800, all types | reported, not tested | finding |
| H3 ΔM − ΔS | −0.50 [−1.31, +0.38] | −0.5000 [−1.2500, +0.2500] | CI90 lower > 0: SUPPORTED; upper < 0: REFUTED; else INCONCLUSIVE | INCONCLUSIVE |
| H3 ΔD − ΔS | +0.06 [−0.75, +0.94] | +0.0625 [−0.8750, +0.9375] | CI90 lower > 0: SUPPORTED; upper < 0: REFUTED; else INCONCLUSIVE | INCONCLUSIVE |
| H4 Spearman, false alarms vs calls per shift | 0.33 | 0.3343 (tie-corrected midrank) | ρ > 0.8: SUPPORTED; ρ ≤ 0: REFUTED; else INCONCLUSIVE | INCONCLUSIVE |
| H5 deep-read penalty, recall(I=1) − recall(I=12) | +0.00 [+0.00, +0.00] | 0.0000 [0.0000, 0.0000] | PENALTY if > 0.10 with CI90 lower > 0; NO-PENALTY if CI90 upper < 0.10; else INCONCLUSIVE | NO-PENALTY |
| H6 judge T* against oracle T* | equal, all three types | equal, all three types | equal: by construction; reported | by construction |
| H7 real diaries agree in sign | signs agree, no explicit key | signs agree on H1 (−2.0556 against −2.0208) and on the registered ΔM − ΔS (−0.25 against −0.50); ΔD − ΔS agrees too | signs of H1 and of ΔM − ΔS on real diaries match the synthetic primary | holds |
| R8 measured-spend residual, 15 percent flag | never computed | 30 of 35 comparable cells flagged | flag a cell whose measured-to-allotted ratio differs from the continuous cell's by more than 15 percent | flagged |
| H0 catalogue null: H1 TIED, T* ≤ 800 for every type, H3 INCONCLUSIVE | not reported | H1 is CONTINUOUS-WINS, so the null's first condition fails | reported if it lands | did not land |
Table 3. The rule column gives each registered decision rule, and every verdict follows it. Every registered point estimate the orchestrator published reproduces exactly on an independently written scorer. Interval endpoints differ by at most one order statistic, which is the bootstrap seed and the percentile-index convention, and no interval difference moves a verdict.
H4 deserves a sentence on why it is uninformative. At the primary budget on clean streams the false alarms per shift run 0.00, 0.06, 0.19, 0.19, 0.00, 0.06, 0.00, 0.00 across the cadence axis, twelve events on 128 shifts. The design's own feasibility note says the structural expectation under a constant per-call rate is a Spearman of 1.0. The observed rate sits so near zero that no monotone relation can form. It is worth recording that the correct estimator was used: the analysis computes Spearman with midranks, and the tie-ignoring shortcut that changed a verdict in the predecessor campaign would have returned −0.0238 here and flipped H4 from INCONCLUSIVE to REFUTED. On the held-out block the shortcut gives −0.4286 against the correct 0.0275, the same flip. That defect did not recur.
The utility reading R3 prices a false alarm at zero, one and three fleet-hours of exposure and shows where the ranking of regimes would flip. It never flips. On the primary seeds the continuous regime produced zero false alarms per shift at the primary cell, so its utility is flat in the price and no exchange rate exists at which the cadence overtakes it; section 8 gives the held-out seed, where it does not flip either. R-OYC prices each regime's exposure as delegation margin through the inherited Oversight Yield Curve kernel at the published centre of its space, mapping 1 to 13 fleet-hours onto 2 to 20 days of detection lag. Continuous sits at a margin of −0.0098 and the shift-eighth cadence at −0.1660, a difference of −0.156 in the currency of that article. No margin came back as a censoring bound, no error code was set, and the conservation residual was zero, so nothing triggered the withholding rule and the reading is reported.
5Replication
The held-out seed was read only after the primary analysis had been written to disk. Six of the seven registered quantities agree on the frozen rule, which asks for the same direction of the point estimate and overlapping 90 percent intervals.
| quantity | primary (seeds 1, 2) | held-out (seed 3) | direction | overlap | agree |
|---|---|---|---|---|---|
| H1 pooled | −2.0208 [−2.3542, −1.6667] | −2.6250 [−3.1250, −2.1667] | same | yes | yes |
| H1 sudden | −1.8750 | −2.0000 | same | yes | yes |
| H1 slow drift | −1.8125 | −3.1250 | same | yes | yes |
| H1 intermittent | −2.3750 | −2.7500 | same | yes | yes |
| H3 ΔM − ΔS | −0.5000 | −0.7500 | same | yes | yes |
| H3 ΔD − ΔS | +0.0625 | −1.1250 | opposed | yes | no |
| H5 penalty | 0.0000 [0, 0] | 0.0000 [0, 0] | same | yes | yes, degenerate |
Table 4. Held-out agreement on the registered rule. The one failure reproduces the orchestrator's failure exactly on an independent scorer.
The seventh is an artefact, and here the evidence is decisive. Computing the same contrast from the oracle field, a deterministic perfect reader with no model in it, gives +0.0625 on the primary seeds and −1.1250 on the held-out seed: byte for byte the judge's own numbers, both flips included. The direction change is produced entirely by where the onset hours fell relative to the six-hour inspection boundaries in seed 3 against seeds 1 and 2. No model behaviour is involved, so no model claim is contradicted by it.
Three facts make that concrete. The contrast is a difference of two means of 16 or 8 integer-valued paired differences on a lattice whose spacing is one fleet-hour, so the estimate can only move in steps of a sixteenth or an eighth. The primary value of +0.0625 is one sixteenth of one fleet-hour, which is one stream, one hour, and the sign is the other way. Both intervals are INCONCLUSIVE and they overlap over [−0.875, 0.000], so neither block asserts a direction at all.
The real diaries carry the same result. On 270 blocks cut from 154 clean diaries written by a different model, at three budgets and four cadences, the primary contrast is −2.0556 [−2.5000, −1.6389] on 36 pairs, CONTINUOUS-WINS, with per-process values of −2.0833, −1.7500 and −2.3333. The crossover is 800 for all three processes under both readers. Split by seed, which the orchestrator does not report, the two primary real seeds give −2.1250 on 24 pairs and the held-out real seed gives −1.9167 on 12, both CONTINUOUS-WINS. Pooling all three synthetic seeds gives −2.2222 [−2.5278, −1.9306] on 72 pairs.
6Ledger integrity
Everything in this section was recomputed from the raw units, without reading the analysis.
- Grid. 40,320 units in exactly seven budgets by eight cadences by four types by three seeds by eight episodes by 24/I inspections. All 672 cells hold exactly the registered count, and no cell is off.
- Duplicates. Zero duplicate unit ids in 40,320 synthetic units and zero in 5,472 real ones. The kill-and-resume test's idempotence claim holds in production: the orchestrator's process group was killed during the probe stage and relaunched, and the ledger went from 60 to 102 units with no duplicate.
- Errors. Three parse errors in 40,320, which is 0.0087 percent of the 34,560 called units against a gate ceiling of 2 percent. All three are on clean streams, so none can touch any hypothesis, and none is at the primary budget. All three carry null detection fields and are excluded, never scored as non-detections. Their finish reason is
length, meaning the reply hit the 120-token cap; none failed the schema. The real tranche had none. - The restricted-mean identity. Across 336 cells the largest absolute difference between mean(min(E, 13)) and the Kaplan-Meier integral is 0.00e+00.
- Regime independence. 96 synthetic streams, 96 distinct stream digests, and no stream carries more than one digest or more than one onset across its 420 units.
- False alarms. 101 in the whole 40,320-unit ledger, 64 on clean streams and 37 on pre-onset windows of faulted streams. The evidence strings are dominated by one inverted reading of the approval clause, and the sentences contradict themselves: "the procedure requires requesting approval only if the amount exceeds 5000, but the agent requested approval for 5545 which is greater than 5000". The agent did the right thing and the judge flagged it. That is a header-comprehension defect, it is independent of regime, and it is the single largest identified component of the false-alarm budget.
- Nondeterminism. Among 34,557 called and parsed units there are 27,273 distinct packets, and 37.7 percent of units sit in a duplicate-packet group. Disagreement depends on what counts as a verdict: 0.56 percent of groups disagree on the full triple of flag, requisition and action, 0.21 percent on the scored strict detection, 0.056 percent on the flag alone, and 45.1 percent on the free-text evidence. Constrained decoding at temperature zero with a fixed seed is close to deterministic on this server without being deterministic.
- Measured overshoot. 1,152 of 34,560 called units, 3.33 percent, spend more measured prompt tokens than their nominal allotment, worst case 1.038 times. Not one of them is a continuous unit. The nominal budget is not a measured ceiling, and where it is breached it is always the cadenced side.
7Why nominal and measured cost diverge
The nominal accounting converts characters to tokens at a flat 2.5. The header is 795 characters of English prose, and the model's tokenizer encodes English prose at roughly 3.3 to 3.6 characters per token, so the fixed conversion over-charges the header by about 40 percent: 342 nominal tokens buy 243 measured ones, a ratio of 0.71 measured directly at the design probe with an empty body.
The ledger corroborates that without containing an empty-body call. Regressing measured prompt tokens on nominal body tokens over the called units gives measured = 219.2 + 1.0440 x nominal_body, so the implied measured header is about 219 against the registered 342, and the smallest bodies in the ledger, at 74 nominal body tokens and 309 measured prompt tokens, bound it in the same place. The slope carries its own sign flip: at 1.044, structured log lines with requisition ids and amounts tokenize about 4 percent worse than the flat assumption, while prose tokenizes 40 percent better. The median measured-to-estimated ratio over all called units is 0.9385.
Here is why that lands on the result and does not wash out. A regime's exposure to the header's mispricing is proportional to how many headers it pays per shift, and headers per shift is 24 / I, which is the axis the campaign sweeps. Any error in the price of the header lands on the regime contrast and nowhere else. The registration saw the mechanism coming, declared the header charge, and registered R8 as the residual check, then filed R8 as a reading. A design that wants this to be airtight has two ways out and should choose before launch. Denominate the budget in body tokens only and report the header as a separate per-call overhead, which leaves nominal and measured diverging by about 4 percent on log text and no longer correlated with the axis under test. Or keep the header inside the budget and move the measured check into the gate ladder with the same frozen threshold and an abort, so that it cannot quietly fail to happen.
8What this campaign does not support
The strongest objection concerns the instrument: the judge here is not the one the design was built to test.
The stated purpose was to find where a real judge's false alarms and long-packet recall move the answer away from what the cost model builds in. Measured: the restricted-mean surface is identical to the perfect reader's at 333 of 336 cells; the headline difference equals the oracle's to four decimals on the primary block, the held-out block and the real-diary block; the process contrasts equal the oracle's to four decimals on both blocks; the crossover equals the oracle's for all three processes; recall is 1.00 at every window length up to twelve hours; 101 false alarms appeared in 40,320 units, none at the primary cell on the primary seeds and one per eight shifts for both regimes on the held-out seed. The feasibility check labelled three of these quantities model floors in advance, and the reading is that every one of them landed on its floor. The claim this campaign can defend is that the crossover is where the header arithmetic puts it and a judge good enough not to matter does not move it. A claim about oversight regimes in general is a different claim and this design did not test it.
No deep-read penalty was measured, because at the registered comparison points there was nothing to measure. Recall on visible faults is 1.0000 at a one-hour window and 1.0000 at a twelve-hour window, so the penalty is exactly zero and its bootstrap interval is degenerate at [0, 0]. NO-PENALTY is the correct verdict under the frozen rule and it is also a statement about a statistic with no variance. The one cell where recall is not 1.00 is the single end-of-shift inspection at 0.6042, which is a 0.396 penalty against the one-hour window and would have read PENALTY under the same rule had that been the comparator. The mechanism there is misattribution: all 48 windows are flagged and all 48 are loose detections, and the judge names a non-faulted requisition in 19 of them. Reading 24 hours of log, the model knows something is wrong and cannot reliably say which hour. This is a post hoc reading of a boundary regime, outside the registered tests. It is a real observation about long-window oversight, and the registered statistic steps around it.
The false-alarm cost of continuous monitoring was not priced, because on the primary seeds it was zero. At the primary cell on the primary seeds the continuous regime produced no false alarms per shift on clean streams, so the utility table is flat in the price of a false alarm for that regime and there is no exchange rate at which the ranking flips. On the held-out seed both regimes produced one false alarm per eight shifts, and R3 still does not flip: continuous reads 1.00, 1.12 and 1.38 at the three prices against 3.62, 3.75 and 4.00 for the shift-eighth cadence. The campaign's third question has no answer here.
No process interaction was separated. Both contrasts are INCONCLUSIVE on both blocks, and both equal their oracle values to four decimals, which means the design did not measure a judge property on that axis at all.
9Limitations
The judge is near-oracle, and that is the limitation that shapes everything above. One rung, capability-local to one mid-tier open-weights judge under constrained non-thinking decoding, as the host's power situation required. A weaker reviewer with a real miss rate on long packets, or a real accusation rate on clean ones, would move the crossover and could move the headline; this campaign cannot say by how much, and its own model-floor labels say so in advance.
The equal-cost construction is exact in nominal tokens and 17.17 percent unequal in measured tokens at the primary contrast cell, over the campaign's own frozen 15 percent flag, and the flag was never computed. At that cell the direction favours the losing regime, so the headline survives. The headline held by luck, and a reviewer is entitled to say that the campaign's own honesty check was the one thing left unrun.
The design is a step function by construction. The purchase-order call is entry 8 in every synthetic block, so the continuous monitor flips from silent to perfect between 600 and 800 tokens, and the entire headline result is that step. The 154 real diaries soften it and give the same answer, which is the best available defence and not a complete one.
Every fault is one act. An amount deviation on the purchase-order chain, which the judge detects at 2 percent. Nothing in this design tests the failure mode where cadence should plausibly win: a fault legible only across hours, which a one-hour window cannot see at any budget. The campaign compares two regimes on a fault that a single hour fully contains, then reports that the regime reading every hour wins. Omission faults are deliberately absent, because an earlier campaign found that log reviewers read them poorly, and a regime comparison on omissions would have been a comparison of two zeros.
The fleet is simulated. Diaries are templated by an inherited harness from a scripted agent. The real tranche uses 270 blocks written by a different model doing the same work, with the same amount rewrite applied.
Exposure is measured to the inspection hour. A continuous monitor's minimum exposure is one hour by construction and a cadenced monitor's depends on alignment. Finer time resolution inside an hour is not modelled, and the one direction flip in replication turns on an estimate of a sixteenth of an hour.
One reading rule, one packet order. Whole blocks in a seeded order and then a prefix. A recency-first rule or a stratified per-hour sample would change the surface, and neither was tested.
The Oversight Yield Curve pricing is a mapping assumption. One fleet-hour of exposure maps linearly onto that kernel's detection-lag dimension at the published centre of its space. It prices the direction and rough size of a regime difference in the currency of the published article, and it is a reading.
10Reproducibility and data availability
The pre-registration is experiments/schouw-cadence/PREREG.md, committed at adb07ef on 2026-09-12 before the first production unit and frozen with the host gate output at 976c2a7, mirrored at wiki/research-plans/prereg-schouw-cadence.md. Raw ledgers on the host: the synthetic campaign ledger (40,320 lines, 53.5 MB) and the real-diary ledger (5,472 lines, 7.1 MB), append-only, fsynced per line, every unit id distinct; the calibration probe ledger holds 1,792 lines. Banked in the repository under results/: analysis-primary.json and analysis-holdout.json (which carries the real tranche), each with a per-rung copy, campaign.log, colophon.json, probe-design.json, wall-draw-probe.txt, check-host-prelaunch.out, feasibility-host-prelaunch.out, kill_test.out, and VERIFICATION.md with its section 10 on R8.
Every figure in this paper was drawn by a script from copies of the two raw ledgers taken by scp on 13 September, with exposures, restricted means, Kaplan-Meier curves, measured-spend ratios and judge-oracle comparisons recomputed from the stored per-unit fields under definitions read from the registration; each figure was rasterised and checked to be non-blank before it was inlined. Point estimates equal the registered analysis and the independent verification; interval endpoints differ in the last significant figure through the bootstrap seed. Marginal compute cost was electricity on owned hardware, about 18.6 h at a measured 130.5 W, about 2.4 kWh, about $0.37 at $0.15 per kWh; a rented GPU at about $3 an hour would come to near $56.
Colophon
Host and serving. One campaign host on the workstation GPU, Python 3.12.3, numpy 2.4.2. The mid-tier judge ran on the serving stack, max_model_len 131072, from a pinned container image (sha256:4cbcfca2…), up since 2026-09-11T21:56:26Z and never restarted or reconfigured; constrained decoding response_format json_schema strict, thinking off at the template, temperature 0, seed 20260912, completion cap 120, parallel 3 with a 1.0 s pace. A second local server held the small-tier model (46e0c10c…, which wrote the real diaries in an earlier campaign) and the larger second judge family (a418f583…, registered and excluded on measured wall draw); the GPU clock cap read inactive at launch.
Digests. Edition SCHOUW-2026-09-12. Code md5: sch_kernel.py 4761a0d63180513be007aed473c062d9 · sch_run.py 727608a4d78b927124ef6ceb00389a64 · sch_check.py da1f78757af912453cb3a24f01a18c4b · sch_feasible.py b9890479a7987532e1dee72fcfa3e90d · sch_gate.py 94e03cf4311e5e4559e4066fdb7b5094 · sch_analyse.py cd611ae604ff31de3f91821fe446cc20 · schouw_campaign.sh 7aa122762bca18f808581e692e47762b. Inherited pins, each equal to its measured digest: Trial Balance v2 kernel md5 642a8d99c37cd3d282f92783babd0315; Oversight Yield Curve kernel md5 3715bdc009d6fce61e11255250e6a280; banked real traces sha256 a2bbbda8bcab4c8290a968196b951d318f1996700825743b23c10c11e70e72f8. Header text sha256 59d928503518f3797dbd4f5ce522018dee538af84f7c4f14e1f59c6eda4cd774 at 342 nominal tokens; verdict schema sha256 c3b6a0c9a2bed8813f04353b5a814c9bc74ccf9c4dc9eea1e4a3a93bc3d978a3; characters per token 2.5.
Timeline (UTC). First orchestrator start 2026-09-12 14:43:33, process group killed at 14:45:32 after 60 probe units for the resume test; the relaunched orchestrator started at 14:45:41 and is the campaign. Instrument check 69/69 and feasibility 14/14 re-run on resume. Calibration probe to 15:12:31, PASS on all four clauses, the mid-tier judge the only admitted rung. Primary seeds to 2026-09-13 01:58:52, primary analysis written 01:58:55. Held-out seed to 07:17:20, held-out analysis 07:17:26. Real-diary tranche to 09:20:26, final analysis and DONE the same minute. 45,792 production units under a 36 h cumulative cap that was never approached, and no power loss during the run. Independent verification 2026-09-13 (commits 96230c0 and ebb7e1b).