The publicly accountable sentiment engine

Sample integrity note (appended 2026-08-29, before grading)

Two things went wrong with this entry after we published it. We are recording both before we know the result, because a disclosure made after the outcome is worth nothing.

1. We contaminated our own sample. On 28 August we ran a burst that deepened our review collection. It worked — one day brought 56,402 reviews against a normal ~4,000 — and it landed very unevenly across the twelve chains named above. Locations passing this entry's filters, recomputed for the pre-registration date of 27 August, using only the data we held that day versus all the data we hold now: Domino's 71 → 177, Dunkin' 27 → 96, Chipotle 31 → 70, AT&T 16 → 28, Verizon 27 → 31, Shake Shack 41 → 44, Subway 75 → 78, In-N-Out 29 → 31, Wendy's 30 → 32, LA Fitness 16 → 17, Taco Bell 58 → 58, Trader Joe's 16 → 16.

The four chains that gained most are all in the declining group. The improving group gained ten locations in total. This entry compares those two groups against each other, so grading on the fuller data would weigh one side of the bet on a substantially rebuilt sample against the other on its original one.

2. We can reproduce the method, but not every number. Reconstructing the sample, we found that the code which produced this entry was never saved — the query was run ad hoc.

Working from the method as written above, the reconstruction matches the entry where it decides the shape of the test: 23 of our 79 chains qualify, exactly as pre-registered. That is not a weak check — a plausible wrong reading of the method, counting by the date we collected a review rather than the date it was written, yields 17.

At chain level it reproduces nine of the twelve location counts exactly. Three differ by one or two locations, in both directions: Domino's −2, AT&T −1, Taco Bell +1. A missing filter produces error in one direction; differences this small running both ways are what window edges produce — timestamps sitting on a boundary, and ties at the ≥3 and ≥5 thresholds.

The failure here is not in the numbers, it is that nobody stored the query. That is now closed: no entry is pre-registered without the code that produced it saved alongside it. Reproducibility is the entire product here, and “almost” does not qualify.

What we are doing, decided before grading. (a) Collection on all twelve chains is frozen until this entry is graded on 10 September; deepening continues on our other 67 chains. (b) The binding grade is the one computed on the frozen sample — the locations as they stood on 27 August, which our point-in-time capture makes recoverable: 437 locations across 12 chains. That is the sample this entry was pre-registered against. (An earlier reconstruction of ours counted 449; 437 is the figure produced by the single definition published here, and it is the one that governs.) (c) We will also publish the grade on the full data, as context and as proof that we are not hiding it — it is not the grade of record, and if the two disagree, the disagreement is published too. (d) The exact definition used for both is published with the grade, so the frozen sample can be rebuilt and checked by anyone.

This note was written and published before either grade was computed. We do not know which way the result goes.

Threshold note: this entry can VOID, and we are saying so now (appended 2026-08-31, before grading)

A third thing about this entry is worth recording before it is graded, because after the result it would only read as an excuse.

The grading rule can end this entry in a VOID, and the reason is arithmetic, not bad luck. The entry qualifies a location with at least 3 reviews in the recent window and at least 5 in the baseline, and qualifies a chain with at least 15 such locations. That filter was set on a 28-day window. Grading applies it to a 14-day one, so the same threshold is roughly twice as hard to clear. Rule 4 then voids the entry if fewer than four chains in either group still hold 15 locations.

We measured how close that is, on a window that cannot tell us anything about the outcome: 13 to 27 August, which closes on the day this entry opens. On that window the binding reading keeps only three of the six improving chains at 15 or more locations - LA Fitness 11, Trader Joe's 13, Wendy's 13 - and would have voided. The declining group keeps five, losing AT&T at 12. The four fragile chains are exactly the four that qualified with 16 or 17 locations at pre-registration: they have no margin, so a thin fortnight drops them out.

We are not changing the rule. We now know that the strict reading points toward VOID on a comparable window while two looser readings of the same sentence would have graded, and that is precisely why the rule stays as written. A threshold moved while holding that knowledge is a moved goalpost, whatever the reasoning attached to it. The ledger exists to make that impossible.

Two things pull the other way, and we state them so this note is not read as a prediction. The grading re-scrape covers all 437 pre-registered locations at once, while the rehearsal window was covered only by our normal collection rotation, which reaches a fraction of them on any given day. Better coverage means more locations clearing the threshold. So VOID is a live possibility, not a forecast.

Finally, the failure recorded in the note above - that the code producing this entry was never saved - is closed for the grade as well. The grading code was written, reviewed and committed on 31 August, before any of the data it will read existed, and it records its own SHA-256 (d53f5ff6e0e9...) in the result, so the published grade names the exact code that produced it. It refuses to grade a sample staler than the two days this entry allows, and it computes the binding reading on the frozen 437 locations, never on a chain re-qualified with today's data.