Model comparison
Several models, trained on identical data cut at identical moments, asked about the same matches and scored by the same code. The question this page answers: if each of these had existed before these matches were played, how many would it actually have called correctly?
Backtested Everything here is retrodiction. Each match was priced from a fit over matches played strictly before its own kickoff, which is an honest test — but the code was written knowing how these seasons ended, so none of it is a forecast published in advance.
| Model | Matches | Correct | Accuracy | RPS | Log loss | Overconfidence |
|---|---|---|---|---|---|---|
| Ensemble | 15,404 | 8,050 | 52.26% ±0.79 | 0.1999 | 0.9855 | +0.2% |
| Dixon-Coles | 15,404 | 8,050 | 52.26% ±0.79 | 0.2000 | 0.9854 | -0.7% |
| Elo | 15,404 | 8,022 | 52.08% ±0.79 | 0.2017 | 1.0008 | +1.4% |
| Poisson | 15,404 | 7,963 | 51.69% ±0.79 | 0.2022 | 1.0056 | +1.6% |
| League base rates baseline | 15,404 | 6,641 | 43.11% ±0.78 | 0.2304 | 1.0751 | +0.6% |
| Random baseline | 15,404 | 5,111 | 33.18% ±0.74 | 0.2353 | 1.0986 | +0.2% |
| Always home baseline | 15,404 | 6,641 | 43.11% ±0.78 | 0.4271 | 2.6285 | +54.9% |
Are these differences real?
McNemar's test on the matches where exactly one of the pair was right. Matches they both got right, or both got wrong, say nothing about which is better.
- Ensemble vs Dixon-Coles indistinguishable χ² 0.00
- Dixon-Coles vs Elo indistinguishable χ² 0.76
- Elo vs Poisson indistinguishable χ² 2.39
- Poisson vs League base rates significant χ² 428.13
- League base rates vs Random significant χ² 316.52
- Random vs Always home significant χ² 316.52
Ensemble — in detail
Confusion matrix
Rows: what happened. Columns: what it said.
| Home | Draw | Away | |
|---|---|---|---|
| Home | 5,415 | 10 | 1,216 |
| Draw | 2,621 | 9 | 1,297 |
| Away | 2,198 | 12 | 2,626 |
By outcome
Precision: when it said this, how often was it right. Recall: of the times this happened, how often did it say so.
| Outcome | Said | Happened | Precision | Recall |
|---|---|---|---|---|
| Home | 10,234 | 6,641 | 52.9% | 81.5% |
| Draw | 31 | 3,927 | 29.0% | 0.2% |
| Away | 5,139 | 4,836 | 51.1% | 54.3% |
This model almost never predicts a draw. That is normal for argmax over three outcomes — draws are rarely the single most likely result — but it means its accuracy figure is carried entirely by home and away calls.
Does confidence mean anything?
Grouped by how sure it was. A calibrated model matches its own claim.
| Band | Matches | Said | Happened | Gap |
|---|---|---|---|---|
| < 40% | 2,399 | 37.8% | 37.6% | +0.2% |
| 40-45% | 2,684 | 42.5% | 41.3% | +1.2% |
| 45-50% | 2,499 | 47.4% | 47.1% | +0.4% |
| 50-55% | 2,146 | 52.4% | 50.8% | +1.6% |
| 55-60% | 1,666 | 57.4% | 56.5% | +0.9% |
| 60-65% | 1,366 | 62.4% | 63.8% | -1.4% |
| 65-70% | 1,082 | 67.4% | 68.3% | -0.9% |
| 70-75% | 769 | 72.4% | 75.2% | -2.7% |
| 75-80% | 501 | 77.3% | 79.2% | -1.9% |
| 80-90% | 290 | 82.9% | 84.5% | -1.6% |
| 90-100% | 2 | 90.4% | 100.0% | -9.6% |
If you only followed it when sure
Coverage matters as much as accuracy: a threshold that is right 80% of the time but fires four times a season is a curiosity, not a strategy.
| At least | Matches | Accuracy | Coverage |
|---|---|---|---|
| all | 15,404 | 52.3% | 100% |
| 40% | 13,005 | 55.0% | 84% |
| 45% | 10,321 | 58.5% | 67% |
| 50% | 7,822 | 62.2% | 51% |
| 55% | 5,676 | 66.5% | 37% |
| 60% | 4,010 | 70.6% | 26% |
| 65% | 2,644 | 74.2% | 17% |
| 70% | 1,562 | 78.2% | 10% |
| 75% | 793 | 81.2% | 5% |
| 80% | 292 | 84.6% | 2% |
When the models agree
Only the pure models are counted here — agreement with the bookmaker benchmark would be measuring something else.
| Agreeing | Matches | Correct | Accuracy |
|---|---|---|---|
| 2 / 4 | 378 | 139 | 36.8% |
| 3 / 4 | 1,754 | 697 | 39.7% |
| 4 / 4 | 13,272 | 7,229 | 54.5% |
By competition
| Group | Matches | Accuracy | RPS |
|---|---|---|---|
| Super League 1 | 1,728 | 53.0% | 0.1880 |
| Serie A | 2,921 | 52.8% | 0.1962 |
| La Liga | 2,947 | 52.0% | 0.1980 |
| Premier League | 2,902 | 53.0% | 0.2026 |
| Bundesliga | 2,301 | 51.8% | 0.2037 |
| Ligue 1 | 2,605 | 51.1% | 0.2076 |
By season
| Group | Matches | Accuracy | RPS |
|---|---|---|---|
| Super League 1 2025/26 | 236 | 50.8% | 0.1837 |
| Super League 1 2024/25 | 235 | 52.3% | 0.2071 |
| Super League 1 2023/24 | 236 | 55.5% | 0.1783 |
| Super League 1 2022/23 | 240 | 51.7% | 0.1789 |
| Super League 1 2021/22 | 239 | 54.4% | 0.2021 |
| Super League 1 2020/21 | 242 | 47.5% | 0.1957 |
| Super League 1 2019/20 | 240 | 54.2% | 0.1764 |
| Super League 1 2018/19 | 60 | 71.7% | 0.1632 |
| Serie A 2025/26 | 378 | 52.1% | 0.2011 |
| Serie A 2024/25 | 379 | 52.2% | 0.1901 |
| Serie A 2023/24 | 380 | 52.1% | 0.1909 |
| Serie A 2022/23 | 379 | 51.5% | 0.1998 |
| Serie A 2021/22 | 378 | 53.2% | 0.2004 |
| Serie A 2020/21 | 378 | 55.0% | 0.1885 |
| Serie A 2019/20 | 377 | 54.4% | 0.2058 |
| Serie A 2018/19 | 272 | 51.1% | 0.1917 |
| Premier League 2025/26 | 379 | 47.5% | 0.2085 |
| Premier League 2024/25 | 379 | 52.0% | 0.2051 |
| Premier League 2023/24 | 379 | 57.0% | 0.1896 |
| Premier League 2022/23 | 379 | 52.8% | 0.2078 |
| Premier League 2021/22 | 379 | 54.9% | 0.1938 |
| Premier League 2020/21 | 379 | 50.1% | 0.2188 |
| Premier League 2019/20 | 377 | 52.5% | 0.2015 |
| Premier League 2018/19 | 251 | 59.4% | 0.1923 |