The publicly accountable sentiment engine

Sample integrity note (appended 2026-08-29, before grading)

Two things went wrong with this entry after we published it. We are recording both before we know the result, because a disclosure made after the outcome is worth nothing.

1. We contaminated our own sample. On 28 August we ran a burst that deepened our review collection. It worked — one day brought 56,402 reviews against a normal ~4,000 — and it landed very unevenly across the twelve chains named above. Locations passing this entry's filters, recomputed for the pre-registration date of 27 August, using only the data we held that day versus all the data we hold now: Domino's 71 → 177, Dunkin' 27 → 96, Chipotle 31 → 70, AT&T 16 → 28, Verizon 27 → 31, Shake Shack 41 → 44, Subway 75 → 78, In-N-Out 29 → 31, Wendy's 30 → 32, LA Fitness 16 → 17, Taco Bell 58 → 58, Trader Joe's 16 → 16.

The four chains that gained most are all in the declining group. The improving group gained ten locations in total. This entry compares those two groups against each other, so grading on the fuller data would weigh one side of the bet on a substantially rebuilt sample against the other on its original one.

2. We can reproduce the method, but not every number. Reconstructing the sample, we found that the code which produced this entry was never saved — the query was run ad hoc.

Working from the method as written above, the reconstruction matches the entry where it decides the shape of the test: 23 of our 79 chains qualify, exactly as pre-registered. That is not a weak check — a plausible wrong reading of the method, counting by the date we collected a review rather than the date it was written, yields 17.

At chain level it reproduces nine of the twelve location counts exactly. Three differ by one or two locations, in both directions: Domino's −2, AT&T −1, Taco Bell +1. A missing filter produces error in one direction; differences this small running both ways are what window edges produce — timestamps sitting on a boundary, and ties at the ≥3 and ≥5 thresholds.

The failure here is not in the numbers, it is that nobody stored the query. That is now closed: no entry is pre-registered without the code that produced it saved alongside it. Reproducibility is the entire product here, and “almost” does not qualify.

What we are doing, decided before grading. (a) Collection on all twelve chains is frozen until this entry is graded on 10 September; deepening continues on our other 67 chains. (b) The binding grade is the one computed on the frozen sample — the locations as they stood on 27 August, which our point-in-time capture makes recoverable: 437 locations across 12 chains. That is the sample this entry was pre-registered against. (An earlier reconstruction of ours counted 449; 437 is the figure produced by the single definition published here, and it is the one that governs.) (c) We will also publish the grade on the full data, as context and as proof that we are not hiding it — it is not the grade of record, and if the two disagree, the disagreement is published too. (d) The exact definition used for both is published with the grade, so the frozen sample can be rebuilt and checked by anyone.

This note was written and published before either grade was computed. We do not know which way the result goes.

Threshold note: this entry can VOID, and we are saying so now (appended 2026-08-31, before grading)

A third thing about this entry is worth recording before it is graded, because after the result it would only read as an excuse.

The grading rule can end this entry in a VOID, and the reason is arithmetic, not bad luck. The entry qualifies a location with at least 3 reviews in the recent window and at least 5 in the baseline, and qualifies a chain with at least 15 such locations. That filter was set on a 28-day window. Grading applies it to a 14-day one, so the same threshold is roughly twice as hard to clear. Rule 4 then voids the entry if fewer than four chains in either group still hold 15 locations.

We measured how close that is, on a window that cannot tell us anything about the outcome: 13 to 27 August, which closes on the day this entry opens. On that window the binding reading keeps only three of the six improving chains at 15 or more locations - LA Fitness 11, Trader Joe's 13, Wendy's 13 - and would have voided. The declining group keeps five, losing AT&T at 12. The four fragile chains are exactly the four that qualified with 16 or 17 locations at pre-registration: they have no margin, so a thin fortnight drops them out.

We are not changing the rule. We now know that the strict reading points toward VOID on a comparable window while two looser readings of the same sentence would have graded, and that is precisely why the rule stays as written. A threshold moved while holding that knowledge is a moved goalpost, whatever the reasoning attached to it. The ledger exists to make that impossible.

Two things pull the other way, and we state them so this note is not read as a prediction. The grading re-scrape covers all 437 pre-registered locations at once, while the rehearsal window was covered only by our normal collection rotation, which reaches a fraction of them on any given day. Better coverage means more locations clearing the threshold. So VOID is a live possibility, not a forecast.

Finally, the failure recorded in the note above - that the code producing this entry was never saved - is closed for the grade as well. The grading code was written, reviewed and committed on 31 August, before any of the data it will read existed, and it records its own SHA-256 (d53f5ff6e0e9...) in the result, so the published grade names the exact code that produced it. It refuses to grade a sample staler than the two days this entry allows, and it computes the binding reading on the frozen 437 locations, never on a chain re-qualified with today's data.

Grader amendment: the baseline is frozen with the sample, and the freshness gate reads the graded rows (appended 2026-09-01, before grading)

The grading code committed on 31 August (sha256 d53f5ff6e0e9...) was reviewed independently on 1 September - before the grading re-scrape, and before any of the data it will read exists. The review found two defects, both of the kind this ledger exists to catch. We fixed them the same day. The previous hash stays in the note above; the fixed code records a new one, d7643a13507e73eed8621fae722ba578ebd734277364a26d982f545653ee5e25 (commit f6d4597).

First, the baseline was not frozen. The code froze the 437 locations but applied a single capture cutoff - the moment of grading - to both windows. The grading re-scrape reaches every frozen location at once, so it would have pulled in older reviews our rotation had never captured, and those would have landed in the baseline and moved it after pre-registration. The baseline now counts only reviews captured by 27 August, the same cut the frozen sample was made with; the forward window counts reviews captured by the grade. Both cutoffs are written into the result.

Second, the freshness rule was checked against each chain's most recent capture anywhere, while the grade runs on the frozen locations only. A chain could have passed the gate on a location outside the sample. The gate now reads the same rows the grade reads.

What the fix changed, measured on the rehearsal window (13 to 27 August, the only window the code is allowed to read before 10 September): the verdicts did not move - strict VOID, matched HIT, frozen_all HIT, as reported on 31 August - but the per-chain numbers moved for all twelve chains, and Verizon's qualifying locations fell from 15 to 5 once its baseline stopped absorbing later captures. That is the defect made visible: same rule, same locations, different baseline. No rule, threshold or reading has changed.

Sample composition note: 30 of the 437 frozen locations are not the chain they were filed under (appended 2026-09-10, before grading)

Recorded today before the grader runs, so it cannot be read as a reaction to the result. Chronology for the record: the Rule 2 re-scrape of the 437 frozen locations ran at 18:26–18:45 UTC (374 locations returned new reviews, 2,279 reviews saved, all twelve chains now within the two-day freshness rule); this note is written after that re-scrape and before any grade is computed.

30 of the 437 frozen locations (6.9%) are venues that are not the chain they were filed under. All 30 are in Germany. By chain and side of the bet: In-N-Out Burger 25 of 29 (declining group) — Berlin burger restaurants that our location discovery of 12 February filed under a chain that has no restaurants outside the United States; Wendy's 3 of 30 (improving group); Shake Shack 2 of 41 (declining group). The other nine chains: 0. The full list with venue names is published at data/proof_xsec01_foreign_locations.json in our repository (available on request until the repository is public) (sha256 761707b0b7dd09aadd38e865e48e0406aaf252303fa3aeb4e7253dfffe856cc4). It was built from the frozen ids and the venue names only; no review data was used.

How it got in. The sample freeze filtered on operational status, chain visibility, manipulation flags and review counts. It had no country filter and no check that the venue name matches the chain; the chain assignment came straight from location discovery. The heading of this entry, "23 US chains", describes the listed companies, not where the locations are: the frozen sample spans 13 countries (Domino's has 16 of its 71 frozen locations in the United States, Subway 14 of 75). We should have said that on 27 August.

The binding grade is unchanged: the strict reading on all 437 frozen locations, exactly as fixed on 31 August, computed by the unmodified grader (sha256 d7643a13507e73eed8621fae722ba578ebd734277364a26d982f545653ee5e25). Removing locations after seeing the problem but before seeing the result is still a change to the sample, and this ledger does not make those.

What we add instead is a third reading, non-binding: strict, on the 407 frozen locations that remain when the 30 are removed. It is computed by a separate script (scripts/proof_xsec01_context_exclusion.py, sha256 bddb51fe9cd7232d…, commit 8b84288, committed before the grader ran) that imports the grader's own filters and verdict logic and modifies none of them. It will be published next to the binding grade and the full-data context, whatever it shows. If the 30 venues change the verdict, we will say so in plain words.

Rule for the next entry, fixed now: a sample freeze must apply a country filter and the location-quality flag before the ids are frozen. This is mandatory before xsec-02.