TUESDAY MODEL LAB / RESEARCH NOTE 01

Four games settled.
No three-way winner.

November 3, 2026 · Data checked November 3, 2026, 10:43 a.m. ET · NFL Week 8

Our first forward-recorded Model Lab cohort is settled. That is progress for the experiment—but not evidence that GPT-6.0, GPT-6.1 or Grok is the better forecaster. One challenger did not produce a verified forecast, and the four GPT-6.1 observations simply reproduced the betting market's own probabilities.

What we actually tested

On November 1, four NFL games were selected before kickoff by a fixed rule: upcoming games from the stored multi-league slate, ordered by start time and game ID, with a Fanatics two-sided moneyline already available and at least 90 minutes before kickoff. The selected games all began at 1 p.m. Eastern. The cohort was committed at 15:13:53 UTC, before their 18:00 UTC starts. We did not substitute more convenient matchups after the results.

The GPT-6.1 route identified as gpt-6.1-sol / High produced four timestamped home-win probability records at 17:06:56 UTC, committed at 17:07:33 UTC—also before kickoff. Each was a normalization of opposing Fanatics moneyline prices observed at 13:00:03.866 UTC, with zero independent model adjustment. Those four probabilities are a market proxy, not an AI-generated improvement on the bookmaker.

GPT-6.0: no independently verified GPT-6.0 forecast exists for these four games. A separate GPT-6 Astra reviewer and the production algorithm do not fill that missing slot. Grok: the existing grok-shadow-v1.1 experimental algorithm has four pregame game-matched records, but it used its own data/time horizon and needs a like-for-like semantics audit. The algorithm version is not proof of a particular underlying Grok LLM runtime.

Coverage before rankings

Coverage counts describe distinct games, not the number of repeated live or pregame revisions. There are no NCAA, NBA or NHL games in this inaugural four-game cohort. Market type: NFL home moneyline only. GPT-5.6 High and official Value Picks are outside this public experiment.

What settled—and what the number means

All four games finished with home-team wins, according to the stored final NFL results cross-checked against the independent forecast-results feed. Across those four binary home-win outcomes, the normalized market probabilities averaged 64.31%. Their mean squared probability error—Brier score, calculated as the average of (predicted home-win probability minus the actual 0-or-1 outcome) squared—is 0.13617.

The GPT-6.1 market-only entries have exactly the same Brier score as that frozen market baseline: incremental improvement is 0.00000 by construction. A lower Brier score is better, but this is only four observations, all home victories. It does not establish predictive calibration, an enduring edge, or superiority over Grok. Different timestamps and underlying inputs also prevent an honest two-way comparison here.

Betting record: all four GPT-6.1 records explicitly said abstain from wagering. Thus W–L–push for actual hypothetical picks is not applicable (0 picks); net units and ROI are not applicable, not zero-profit or positive-return claims. We will not pretend those four winning home outcomes were bets placed at the recorded prices. Grok's experimental probabilities likewise do not become bets without prospectively frozen market selections and price/settlement rules.

Why a three-way contest is still missing

A fair head-to-head needs three valid forecasts for the same game and question, created before kickoff, with documented model identity, input time, market conventions and reproducible outcomes. We have a frozen cohort and settled results. We do not yet have a GPT-6.0 forecast arm for this cohort. The GPT-6.1 arm is only a market transformation; Grok's experimental algorithm has a separate information set. Inventing a league table from these mismatches would be less informative than reporting the gap.

What changes next week

The next prospective batch should fix the experiment before the games begin: execute verifiable GPT-6.0 and GPT-6.1 routes on the same frozen input bundle, align Grok to a documented comparable question and observation window, explicitly define NFL overtime and tie treatment, and record any actual wagering selection with bookmaker price and stake before kickoff. We will continue to disclose missing coverage rather than quietly removing hard games or changing past records.

Method: prospective private Git history for cohort and forecast commit times; immutable source quote times; read-only Sports Supabase NFL final status and forecast results; read-only Grok experiment revisions. External result feed: ESPN Week 8 scoreboard data. No private prediction ledger or internal source file is published. All returns discussed are hypothetical research metrics; Benny's Lab places no wagers through this experiment.