Methodology

• One season, shared luck. All eight teams draw from the same set of NFL outcomes. A hamstring in Week 3 is not a reasoning failure.
• Small sample. Fourteen regular-season weeks and roughly forty lineup decisions per model. Not enough to separate close finishers.
• Draft luck is real and unremoved. The slot auction converts draft position from luck into a priced decision, but the draft itself still contains luck.
• The cohort is not price-matched. It spans $0.58 to $5.00 per million input tokens. Cost per decision is published next to results.
• Head-to-head adds timing luck on purpose. See below.

The winner is the best manager of this season, not the best possible manager.

• The rulebook is generated, never written. It is produced from the same configuration that drives the scoring engine, so the rules the models are told cannot drift from the rules that are enforced.
• Comprehension is tested, not assumed. Before any consequential decision every model answers a fixed set of questions whose answers are computed from the rulebook and graded in code. A model below 100% has the rulebook re-injected and re-answers; both attempts are published.
• No tools, no web search, no function calling for anyone. Eight models searching independently would return different results at different times and destroy both fairness and reproducibility.
• Identical retry policy and identical deterministic fallbacks, with every fallback publicly flagged rather than quietly repaired. Provider outages are recorded separately from model errors, because a model should not be blamed in the standings for its provider's downtime.
• Memory parity. A fixed-size, identically-structured continuity block. Unbounded history would degrade the smallest context window in the cohort first.
• List-order bias is controlled. Models measurably favour items earlier in a list, so the available-player list is ordered by projection, identically for everyone.

Model IDs are pinned before the draft and never swapped mid-season. That rule always left the pre-season open, and the pre-season is exactly when labs ship — so the cohort is frozen on a stated date: 2026-08-24. After it, no model ID changes for any reason short of a provider withdrawing one.

The date exists because “whatever was newest on the day someone happened to look” is not a rule. On 3 August 2026 Alibaba released a model newer than, and a tier above, the Qwen entry in this cohort. We did not take it — it had no track record three weeks out, all eight incumbents had passed the comprehension gate together, and swapping one would have invalidated that shared result until the new model was re-gated on its own.

On 14 August that objection stopped applying, and four seats moved. Adding the playoff rules bumped the rulebook to v3, which forced all eight to re-sit the comprehension check anyway — so the re-gate was happening regardless and the marginal cost of taking each lab's current top tier fell to zero. xAI, Meta, DeepSeek and Alibaba were re-pinned; OpenAI, Anthropic, Moonshot and Google were already on their lab's top tier. Every seat then re-sat the gate and passed, 8/8, first attempt. Ten days later the freeze date arrived and the draft was run against it.

One seat needed a second correction, and it is the more interesting one. A pinned ID is only a pin if the provider treats it as one. On 4 September we called all eight IDs and asked each what it had actually run: seven named exactly the model we sent, and Alibaba's named a different one. qwen/qwen3.8-max was a floating alias, and it had come to point at a snapshot dated 3 September — ten days after the draft. So that seat drafted its roster under one set of weights and would have played the season under another, and we cannot say which drafted, because a decision row stores the answer a model gave and not the snapshot that gave it. The seat is now addressed by its dated snapshot. Same lab, same model line, same context window, same price — an address change rather than a seat change, made so the promise on this page is true rather than approximately true.

The costs here are real and stated rather than hidden. This league runs models that are no longer their labs' newest, and it will run them in December. One seat spent ten days on weights we did not choose. The alternative — a cohort that changes whenever we check, and an address that can change without telling us — would make every comparison in the season unfalsifiable.

• Return touchdowns belong to the DEF/ST unit, never the individual returner. Crediting both would pay 12 points league-wide for one return. There is a test asserting it pays exactly 6.
• Two projection inputs are our own derivation. Sleeper does not project points allowed for defenses at all, and omits every sub-40-yard field goal for kickers. Both are reconstructed from the prior completed season and labelled. This affects projections only — actual scoring never uses it.
• Validation is strict on outcomes, lenient on cosmetics. A fifth bullet or an over-long one is recorded and shown publicly, but does not trigger a fallback, because reporting a model as having failed over formatting would misstate what happened.
• Lineups lock weekly, not per player. Stricter than Yahoo and deliberately so: per-player locks would give a team with a Thursday player less information than one deciding on Sunday.

Before any model is asked for a lineup, the league's code sets one for every team: the highest projection at each slot, skipping anyone on bye or listed Out. That is the autopilot, and it is what a team starts if its model fails.

The headline lineup figure is calls: the model's score minus the autopilot's, from the same roster in the same week. Every player both lineups started cancels out, so what is left is only what the model chose to change. A model that starts the autopilot's nine scores exactly zero, however its week went. Every box score lists each change and what it was worth.

The autopilot is replayed from the projections, injury tags and byes in each team's stored prompt — the numbers the model was actually shown — not from our projections table.

Lineup efficiency (points scored ÷ the best lineup the roster held) used to lead. It is still published as “hindsight best”, but it grades the week rather than the decision: in week 1 a model that started the autopilot's nine ranked last on it, because of which of its players happened to score. It also penalises a deep bench, because every good reserve raises the ceiling it is divided by.

Every week, before the week's first kickoff, each model picks the winner of every NFL game and gives a probability from 0.5 to 1 that its pick wins. These picks are not part of the fantasy league and change nothing in it.

The models see the market and bet on it (changed 25 September 2026). Each game in the data block now carries a moneyline: the median price across US sportsbooks, fetched once when the picks are made and stored whole, so every bet replays against the price it was offered. Each model has $100 of play money that has to last from week 4 through week 18, the end of the regular season, with no top-ups, and the prompt tells it how many weeks are left. It may bet whole dollars on either team in any game, or on nothing, up to what it has not already got riding. The pick and the bet are made in the same answer, so every probability was formed with the market price in view and are not comparable to a blind forecast. That is why the market itself, the favourite at its probability with the bookmakers' margin removed, is graded on the board beside the models. The data block is still identical for all eight. Each model's own balance is the one fact that differs, so it is stated after the block, outside the published hash. The money is not real, no book is named, and nothing here links to a sportsbook.

Picks were announced for week 3 and began in week 4. The database tables they are stored in had not been created in production, so no week 3 picks were ever stored. We found this on 30 September and created them before week 4. Every model's $100 therefore first went on the table in week 4. The same day the prompt (picks-v3-horizon) was changed to say how many weeks the money has to last, before any bet had been placed.

This is the one place models may use their own football knowledge. Everywhere else they reason only from the data we send. Here we send this season's results, the current injury report for starters and the projected starting quarterback for each team, and tell them the data wins where it disagrees with what they remember. All eight get the same block, byte for byte, and none sees another's picks. Its hash is published every week.

Grading is code. A game's final score comes from Sleeper's team defense stat line, where each defense's points allowed is the other team's score. We checked this against a published box score before relying on it. Grades are recomputed every time the page is built, so a stat correction regrades a pick rather than leaving a stale verdict.

The board ranks by Brier score: the mean of (stated probability − outcome)², where a win is 1, a loss 0 and a tie 0.5. Lower is better, and 0.250 is what a coin flip scores. Accuracy alone would reward a model that says 99% on every game. Three baselines are published beside the models: the majority pick of all eight, always picking the home team, and the market favourite. Bets settle on the same final scores. A tie returns the stake, and money on a game not yet scored is held at risk rather than counted either way.

sha256(seed) — committed:
537bc77bf8df760a84962b1736426c6fa531951a6ddd29f725db8a0627bc41e4

seed — revealed after the auction:
c4ddf7f4ca0721b28faca980cf769173738437342331cfd795c54dcf2bf755fb

The seed was generated by the project owner, not by the model that built this league. Its leverage is small — it only breaks tied bids — but “the builder never touched it” is a cheap property to be able to state.