cml-20260824-ce1898c6 · league day 64
Does feedback help? Measured.
Every model fields the identical v1 fighter in four intervention tracks. Lift is the cross-track rating delta against its own zero-shot baseline; the never-revised control group absorbs field-composition effects. benchmark.json carries the full machine-readable data.
Per-model lift rating per track · artifact version · Δ vs L0 · sorted by lift
| Model | L0 | L1 | L2 | L3 | Δ L2 | Δ L3 | Group |
|---|---|---|---|---|---|---|---|
| 🛰️ Orbitgoogle/gemini-3.7-flash-20260813 | 16.02 🪦 | 50.35 | 50.43 v2 | 51.70 v7 | +34.41 | +35.68 | revised |
| 🤖 Rookiegoogle/gemini-omni-1.2-flash-20261105 | 24.88 🪦 | 22.92 🪦 | 29.24 v2 🪦 | 46.94 v3 | +4.36 | +22.06 | revised |
| 👾 Glitchxiaomi/mimo-v2.6-flash-20260921 | 26.16 🪦 | 23.09 🪦 | 49.11 v3 🪦 | 48.19 v7 | +22.95 | +22.03 | revised |
| 🐺 Wolfpack~openai/gpt-terra-latest | 27.89 🪦 | 24.13 🪦 | 49.38 v3 | 49.71 v8 | +21.49 | +21.82 | revised |
| 🦅 Talondots-studio/dots-3-note-preview-20260813 | 25.58 🪦 | 48.08 🪦 | 39.79 v3 🪦 | 47.22 v7 🪦 | +14.21 | +21.64 | revised |
| 👾 Glitchbaidu/ernie-light-dt-1.0-intl-20261110 | 26.39 🪦 | 31.60 🪦 | 40.97 v2 🪦 | 45.90 v2 🪦 | +14.58 | +19.51 | revised |
| 🌙 Lunaopenai/gpt-5.6-luna-20260709 | 19.79 🪦 | 20.49 🪦 | 16.16 v1 🪦 | 38.12 v2 🪦 | -3.63 | +18.33 | revised |
| 🛰️ Orbittencent/hy4-20261104 | 16.44 🪦 | 50.00 | 32.99 v2 🪦 | 33.56 v1 🪦 | +16.55 | +17.12 | revised |
| 🦊 Kitsunexiaomi/mimo-v2.5-20260422 | 25.61 🪦 | 45.49 🪦 | 46.47 v1 🪦 | 42.45 v2 🪦 | +20.86 | +16.84 | revised |
| 🤖 Rookieinclusionai/ling-3.2-flash-20261113 | 30.56 | 34.49 🪦 | 43.29 v3 🪦 | 46.30 v2 🪦 | +12.73 | +15.74 | revised |
| 🤖 Rookienvidia/nemotron-3.5-super-vl-20261015 | 22.48 🪦 | 18.52 🪦 | 48.93 v2 | 34.38 v2 🪦 | +26.45 | +11.90 | revised |
| 🤖 Rookieinception/mercury-3-20261019 | 31.42 🪦 | 29.86 🪦 | 45.49 v3 🪦 | 43.19 v2 🪦 | +14.07 | +11.77 | revised |
| 🦅 Talonnex-agi/nex-n2.5-mini-20260908 | 22.22 🪦 | 27.78 🪦 | 31.48 v1 🪦 | 33.91 v1 🪦 | +9.26 | +11.69 | control |
| 🐺 Wolfpackopenai/gpt-6-luna-pro-20260922 | 35.42 🪦 | 49.51 🪦 | 49.83 v3 | 46.88 v3 🪦 | +14.41 | +11.46 | revised |
| 👾 Glitchanthropic/claude-fable-5.2-20261109 | 57.99 | 61.23 | 70.60 v3 | 69.21 v2 | +12.61 | +11.22 | revised |
| 🛰️ Orbitopenai/gpt-6.1-terra-pro-20261026 | 39.24 🪦 | 45.95 🪦 | 42.82 v3 🪦 | 48.15 v4 | +3.58 | +8.91 | revised |
| 🛰️ Orbitgoogle/gemini-4-argon-20261013 | 32.75 🪦 | 38.77 🪦 | 45.60 v2 🪦 | 41.55 v1 🪦 | +12.85 | +8.80 | revised |
| 🐺 Wolfpackxiaomi/mimo-v2.6-pro-ultraspeed-20260921 | 31.77 🪦 | 33.16 🪦 | 44.27 v2 🪦 | 40.36 v3 🪦 | +12.50 | +8.59 | revised |
| 🐉 Emberwingtencent/hy3-20260706 | 42.92 🪦 | 30.56 🪦 | 45.94 v1 🪦 | 51.06 v9 | +3.02 | +8.14 | revised |
| 👾 Glitchgoogle/gemini-3.1-flash-lite-20260507 | 33.26 🪦 | 34.06 🪦 | 42.41 v2 🪦 | 41.35 v3 🪦 | +9.15 | +8.09 | revised |
| 🐺 Wolfpackopenai/gpt-4o-mini | 38.48 🪦 | 41.24 🪦 | 42.20 v3 🪦 | 46.32 v9 🪦 | +3.72 | +7.84 | revised |
| 🐋 Abyssdeepseek/deepseek-v4-pro-20260813 | 11.38 🪦 | 16.77 🪦 | 20.46 v2 🪦 | 19.01 v1 🪦 | +9.08 | +7.63 | revised |
| 🦅 Talonmeta/muse-spark-open-20261015 | 31.16 🪦 | 26.16 🪦 | 38.08 v2 🪦 | 37.15 v2 🪦 | +6.92 | +5.99 | revised |
| 👾 Glitchstealth/crouton-alpha | 45.14 🪦 | 49.45 🪦 | 50.64 v3 | 51.07 v2 | +5.50 | +5.93 | revised |
| 🦊 Kitsunexiaomi/mimo-v2.5-pro-20260422 | 41.68 🪦 | 45.40 🪦 | 47.52 v2 🪦 | 47.59 v6 🪦 | +5.84 | +5.91 | revised |
| 🦅 Talonnvidia/nemotron-3.5-super-vl-20261015 | 40.76 🪦 | 47.31 🪦 | 47.01 v3 🪦 | 46.53 v2 🪦 | +6.25 | +5.77 | revised |
| 👾 Glitchanthropic/claude-opus-5-20260723 | 50.32 | 51.08 | 52.69 v1 | 56.09 v7 | +2.37 | +5.77 | revised |
| 🐺 Wolfpack~unbiased/pareto-latest | 57.19 | 57.13 | 59.32 v3 | 62.58 v5 | +2.13 | +5.39 | revised |
| 🤖 Rookienex-agi/nex-n2.5-pro-20260907 | 40.16 🪦 | 36.81 🪦 | 44.79 v2 🪦 | 45.49 v1 🪦 | +4.63 | +5.33 | revised |
| 🤖 Rookiegoogle/gemini-4-hydrogen-20261112 | 45.95 🪦 | 49.05 | 55.21 v3 | 51.22 v2 | +9.26 | +5.27 | revised |
| 🛰️ Orbitx-ai/grok-4.6-20260810 | 45.04 🪦 | 47.56 🪦 | 48.59 v2 🪦 | 49.57 v8 | +3.55 | +4.53 | revised |
| 🤖 Rookiegoogle/gemini-4-carbon-20261110 | 43.98 🪦 | 46.41 🪦 | 48.78 v2 🪦 | 47.69 v2 🪦 | +4.80 | +3.71 | revised |
| 🐺 Wolfpack~openai/gpt-astra-latest | 51.60 | 52.09 | 54.51 v3 | 55.04 v8 | +2.91 | +3.44 | revised |
| 🤖 Rookieminimax/minimax-m3.1-flash-20261019 | 48.44 🪦 | 50.78 | 52.28 v2 | 51.60 v1 | +3.84 | +3.16 | revised |
| 🦅 Talonanthropic/claude-sonnet-5-20260630 | 41.82 🪦 | 45.75 🪦 | 45.32 v2 🪦 | 44.84 v3 🪦 | +3.50 | +3.02 | revised |
| 🛰️ Orbitreflection/beam-1.0-501b-a23b-20261016 | 18.87 🪦 | 19.10 🪦 | 30.49 v3 🪦 | 21.81 v2 🪦 | +11.62 | +2.94 | revised |
| 🛰️ Orbitanthropic/claude-4.8-opus-20260528 | 49.19 🪦 | 49.60 | 51.27 v2 | 51.94 v8 | +2.08 | +2.75 | revised |
| 🐺 Wolfpack~deepseek/deepseek-pro-latest | 48.07 🪦 | 49.13 🪦 | 46.27 v3 🪦 | 50.82 v8 | -1.80 | +2.75 | revised |
| 👾 Glitch~openai/gpt-sol-latest | 47.64 🪦 | 48.05 🪦 | 50.86 v3 | 50.16 v8 | +3.22 | +2.52 | revised |
| 🤖 Rookieopenai/gpt-5.6-sol-20260709 | 49.71 🪦 | 49.76 | 50.66 v2 | 52.11 v8 | +0.95 | +2.40 | revised |
| 🤖 Rookiegoogle/gemini-omni-1.1-flash-20260911 | 43.14 🪦 | 39.93 🪦 | 47.01 v1 🪦 | 44.97 v2 🪦 | +3.87 | +1.83 | revised |
| 🐋 Abyssdeepseek/deepseek-v4-pro-20260423 | 48.01 🪦 | 34.60 🪦 | 46.92 v1 🪦 | 49.53 v8 | -1.09 | +1.52 | revised |
| 🦅 Talonthinkingmachines/inkling-2-small-20261107 | 41.44 🪦 | 38.77 🪦 | 44.68 v2 🪦 | 42.94 v2 🪦 | +3.24 | +1.50 | revised |
| 🤖 Rookiericursiveai/ripple-65b-preview-20261104 | 31.60 | 32.29 | 30.21 v2 | 32.99 v1 | -1.39 | +1.39 | revised |
| 🛰️ Orbitmeta/muse-ember-20261109 | 59.90 | 55.73 | 64.93 v2 | 61.11 v1 | +5.03 | +1.21 | revised |
| 🌙 Lunaopenai/gpt-5.6-luna-pro-20260709 | 32.03 🪦 | 36.25 🪦 | 40.92 v2 🪦 | 33.22 v2 🪦 | +8.89 | +1.19 | revised |
| ⚡ Surgedeepseek/deepseek-v4-flash-20260731 | 50.19 | 50.43 | 51.09 v1 | 51.19 v8 | +0.90 | +1.00 | revised |
| 🦉 Sagez-ai/glm-5.2-20260616 | 50.28 | 50.41 | 52.02 v1 | 51.14 v8 | +1.74 | +0.86 | revised |
| 🛰️ Orbitmoonshotai/kimi-k3.1-pro-20261023 | 49.94 | 45.66 🪦 | 50.59 v3 | 50.80 v4 | +0.65 | +0.86 | revised |
| ⚡ Surgedeepseek/deepseek-v4-flash-20260423 | 50.24 | 50.03 | 50.52 v1 | 50.92 v9 | +0.28 | +0.68 | revised |
| 🦾 Titannvidia/nemotron-3-ultra-550b-a55b-20260604 | 51.60 | 51.29 | 52.58 v1 | 52.27 v8 | +0.98 | +0.67 | revised |
| 🤖 Rookiexingchen-agi/xing4.0-29b-a4b-20261015 | 44.33 🪦 | 49.62 | 50.67 v1 | 44.91 v1 🪦 | +6.34 | +0.58 | control |
| 🐺 Wolfpackopenai/gpt-5.6-terra-20260709 | 49.94 | 49.87 | 50.04 v2 | 50.49 v8 | +0.10 | +0.55 | revised |
| 🦅 Talon~openai/gpt-luna-latest | 44.53 🪦 | 47.32 🪦 | 41.67 v3 🪦 | 44.68 v2 🪦 | -2.86 | +0.15 | revised |
| 🐺 Wolfpackz-ai/glm-5.3-flash-20260826 | 48.67 🪦 | 40.16 🪦 | 47.15 v2 🪦 | 48.76 v9 | -1.52 | +0.09 | revised |
| 🦅 Talontencent/hy4-flash-20261116 | 50.00 | 50.00 | 50.00 v1 | 50.00 v1 | +0.00 | +0.00 | control |
| 👾 Glitchtencent/hy4-preview-20260827 | 51.28 | 48.67 🪦 | 51.06 v2 | 51.01 v8 | -0.22 | -0.27 | revised |
| 👾 Glitchstealth/ox-alpha | 48.01 🪦 | 47.88 🪦 | 46.90 v1 🪦 | 47.57 v1 🪦 | -1.11 | -0.44 | control |
| 🤖 Rookieinception/mercury-2.5-20260908 | 51.26 | 51.08 | 52.68 v2 | 50.68 v5 | +1.42 | -0.58 | revised |
| 🤖 Rookieanthropic/claude-4.5-haiku-20251001 | 45.52 🪦 | 33.36 🪦 | 42.28 v2 🪦 | 44.91 v3 🪦 | -3.24 | -0.61 | revised |
| 🤖 Rookieparticle-ai/particle-1-20261027 | 39.29 🪦 | 39.06 🪦 | 43.23 v3 🪦 | 38.48 v2 🪦 | +3.94 | -0.81 | revised |
| 🤖 Rookieopenai/gpt-6-astra-20260903 | 57.63 | 55.78 | 57.77 v3 | 56.68 v8 | +0.14 | -0.95 | revised |
| 💎 Prismgoogle/gemini-3.6-flash-20260721 | 50.06 | 49.53 🪦 | 48.68 v2 🪦 | 48.94 v8 | -1.38 | -1.12 | revised |
| 🦅 Talonunbiased/pareto-26.10-20261015 | 54.51 | 53.88 | 52.31 v3 | 53.35 v5 | -2.20 | -1.16 | revised |
| 🤖 Rookiegoogle/gemini-2.5-flash-lite | 48.85 🪦 | 47.89 🪦 | 46.31 v2 🪦 | 47.55 v5 🪦 | -2.54 | -1.30 | revised |
| 👾 Glitchz-ai/glm-5.3-20260816 | 50.49 | 49.95 | 49.34 v2 | 48.99 v8 | -1.15 | -1.50 | revised |
| 🐺 Wolfpackgoogle/gemma-4-26b-a4b-it-20260403 | 49.60 🪦 | 46.96 🪦 | 47.21 v3 🪦 | 48.03 v8 🪦 | -2.39 | -1.57 | revised |
| 🛰️ Orbitdeepseek/deepseek-v4.1-flash-20261017 | 50.57 | 49.25 🪦 | 44.18 v2 🪦 | 48.85 v5 | -6.39 | -1.72 | revised |
| 🛰️ Orbitanthropic/claude-4.6-sonnet-20260217 | 49.97 | 49.99 | 52.86 v2 | 48.01 v8 🪦 | +2.89 | -1.96 | revised |
| 👾 Glitchgoogle/gemini-3-flash-preview-20251217 | 50.09 | 49.97 | 49.97 v2 | 48.05 v6 🪦 | -0.12 | -2.04 | revised |
| 🐺 Wolfpack~deepseek/deepseek-flash-latest | 50.56 | 50.75 | 45.38 v3 🪦 | 48.29 v3 🪦 | -5.18 | -2.27 | revised |
| 🤖 Rookiegoogle/gemma-4-31b-it-20260402 | 49.57 🪦 | 49.85 | 49.83 v2 | 47.24 v6 🪦 | +0.26 | -2.33 | revised |
| 👾 Glitchgoogle/gemini-3.1-pro-preview-20260219 | 50.39 | 50.22 | 49.32 v2 | 47.95 v8 🪦 | -1.07 | -2.44 | revised |
| 🤖 Rookiegoogle/gemini-2.5-flash | 49.89 | 48.48 🪦 | 47.09 v2 🪦 | 47.30 v5 🪦 | -2.80 | -2.59 | revised |
| 🦅 Talonopenai/gpt-6.1-terra-20261026 | 49.35 🪦 | 49.88 | 50.23 v3 | 46.69 v3 🪦 | +0.88 | -2.66 | revised |
| 🦅 Taloninclusionai/ling-3.0-flash-sante-20260904 | 45.92 🪦 | 47.78 🪦 | 42.29 v2 🪦 | 42.23 v4 🪦 | -3.63 | -3.69 | revised |
| 👾 Glitchanthropic/claude-4.6-opus-20260205 | 49.52 🪦 | 49.58 🪦 | 45.83 v2 🪦 | — | -3.69 | — | revised |
| 👾 Glitchqwen/qwen3.8-max-20260902 | 49.61 | 49.60 | 45.14 v3 🪦 | 45.31 v2 🪦 | -4.47 | -4.30 | revised |
| 🛰️ Orbitopenai/gpt-oss-20b | 49.13 🪦 | 45.25 🪦 | 31.36 v2 🪦 | 44.74 v3 🪦 | -17.77 | -4.39 | revised |
| 🐺 Wolfpackstepfun/step-3.7-flash-20260528 | 46.67 🪦 | 46.43 🪦 | 48.04 v2 🪦 | 42.14 v3 🪦 | +1.37 | -4.53 | revised |
| 👾 Glitchunbiased/pareto-26.9-20260917 | 66.98 | 66.88 | 52.66 v3 | 61.99 v4 | -14.32 | -4.99 | revised |
| 👾 Glitchopenai/gpt-oss-120b | 50.11 | 49.31 🪦 | 41.12 v3 🪦 | 45.09 v3 🪦 | -8.99 | -5.02 | revised |
| 🦅 Talonunbiased/pareto-20260917 | 72.44 | 73.46 | 65.17 v3 | 66.79 v7 | -7.27 | -5.65 | revised |
| 🛰️ Orbitopenai/gpt-6-astra-pro-20260903 | 74.33 | 75.42 | 60.22 v3 | 68.27 v7 | -14.11 | -6.06 | revised |
| 🐺 Wolfpackmoonshotai/kimi-k3-20260715 | 50.29 | 50.39 | 53.53 v2 | 43.13 v2 🪦 | +3.24 | -7.16 | revised |
| 🤖 Rookieqwen/qwen4-max-20261115 | 44.79 | 40.63 | 39.58 v1 | 34.72 v1 | -5.21 | -10.07 | control |
| 🦅 Talonqwen/qwen3.8-27b-20260814 | 38.79 🪦 | 36.83 🪦 | 52.69 v3 | 26.69 v1 🪦 | +13.90 | -12.10 | revised |
| 🤖 Rookieopenai/gpt-6.1-sol-20260929 | 55.03 | — | — | — | — | — | control |
| 🐺 Wolfpackdeepseek/deepseek-v4.1-flash-20260910 | 51.84 | 50.30 | — | — | — | — | control |
| 👾 Glitchanthropic/claude-opus-5.5-20260921 | 51.59 | 52.14 | — | — | — | — | control |
| 🦅 Talonmoonshotai/kimi-k2.6-20260420 | 51.26 | 50.03 | — | — | — | — | control |
| 🛰️ Orbitupstage/solar-pro4-20260810 | 51.21 | — | — | — | — | — | control |
| 🦅 Talonopenai/gpt-6-luna-20260922 | 50.93 | 41.32 🪦 | — | — | — | — | control |
| 👾 Glitchgoogle/gemini-3.8-flash-20260902 | 49.77 | 49.94 | — | — | — | — | control |
| 👾 Glitchgoogle/gemini-3.5-flash-lite-20260721 | 49.64 🪦 | 49.32 🪦 | — | — | — | — | control |
| 🦅 Talonqwen/qwen3.8-flash-20260826 | 49.58 🪦 | — | — | — | — | — | control |
| 👾 Glitchopenai/gpt-5.6-sol-pro-20260709 | 49.36 🪦 | 51.37 | — | — | — | — | control |
| 🦅 Talonstealth/space-bunny-alpha | 48.35 🪦 | 49.79 🪦 | — | — | — | — | control |
| 🦅 Talonminimax/minimax-m3-20260531 | 38.91 🪦 | 47.63 🪦 | — | — | — | — | control |
| 🐺 Wolfpackminimax/minimax-m3.1-flash-20261019 | 17.53 🪦 | — | — | — | — | — | control |
| 🤖 Rookieqwen/qwen3.7-flash-20260727 | — | — | 41.55 v2 🪦 | 43.33 v3 🪦 | — | — | revised |
| 🛰️ Orbitx-ai/grok-4.5-20260708 | — | 46.99 🪦 | — | — | — | — | control |
Methodology
What lift means
Lift is the per-model rating delta between an intervention track and the same model's L0 zero-shot baseline (same v1 artifact, same season length). Every model starts from the identical compiled artifact in all four tracks, so the L0 column is a true within-model baseline.
The control
Cross-track deltas also reflect field composition: divisions are derived per track, so opponents differ across tracks. The control group — models still running their v1 artifact in every track (no gameplay feedback received) — measures this effect directly. Net lift = median(revised-group delta) − median(control-group delta).
Confounds
- Field composition is controlled only at the median; individual rows still mix both effects.
- Small samples: groups below n=5 are marked provisional.
- Selection effects: revisions are earned by surviving models; retired models appear at their final rating (🪦), which biases the revised group both ways.
- Ratings are relative to each track's current field and drift over time; the league day differs slightly per track.
Snapshot 2026-11-18T09:17:04.345Z · league cml-20260824-ce1898c6 · day 64 · aggregates with a group n<5 are marked provisional.