Ce que chaque modèle de prédiction de victoire déclare sur lui-même

Files published beside the post (exist; not downloaded): https://media.loldraftai.com/blog/draftgap-vs-loldraftai-comparison/predictions.csv.../draftgap-current-patch.json.../draftgap-30-days.json. ECE is described as the 'average gap between stated probability and observed frequency' (bins not stated); no AUC. The post adds: 'The shipped LoLDraftAI product additionally uses side information and fine-grained elo (features DraftGap doesn't have), but they weren't needed to produce this result.' A vendor's comparison against a rival.

The Dota 2 rows are the same paper's second dataset (Dota 2 is not this file's game). Generic baselines in the same tables, not listed above: MC (majority class, 'Blue for LOL and Radiant for Dota2') 0.5040 LoL and 0.5180 Dota 2; LR 0.5255 during and 0.5323 after the draft (LoL), 0.5750 and 0.6126 (Dota 2); NN 0.5263 and 0.5335 (LoL), 0.5748 and 0.6108 (Dota 2). After the Dota 2 draft, LR (0.6126) prints above DraftRec (0.6110). Outcome prediction is the paper's secondary task; its main task is champion recommendation. Code and data: github.com/dojeon-ai/DraftRec.

One of the earliest studies to document that pre-game prediction on an honest hold-out test set in solo queue hits an inherent ceiling of ~55% accuracy.

Per-match predictions published: https://media.loldraftai.com/blog/loltheory-vs-loldraftai-comparison/predictions.csv (exists; not downloaded). 'Parenthetical ranges are 95% bootstrap confidence intervals'. The ECE bin count is not stated; no AUC is reported. The post describes LoLTheory as, 'primarily an Overwolf overlay app (~166,000 downloads)', and notes. Headline: 'On every one of these, LoLDraftAI beats LoLTheory.' A vendor's comparison against a rival, not an independent evaluation.

Sample counts disagree inside the paper (4,552 NA + 12,458 LAN; '12,458 samples... from Gonzalez'; 'a dataset of 28000 samples'). Each feature becomes a 1/0 'dominance' value ('If team A's performance is better than team B's for a feature, assign a value of 1; otherwise, assign 0'), and the text also says 'The feature's value indicator is a binary value of 1 if the blue team won and 0 if the blue team did not win'. The weights are computed from counts of 'data classified as 1' and 'data classified as 0' over the whole dataset, and the paper does not state that the 'class avg' and 'class med' accuracies are scored against the real match outcome; pregame is therefore 'unclear'. The paper frames the model for 'Solo Queue Ranked Match (SQRM)' but does not name its data's queue. F-measure values print between 4.0000 and 5.0160. Table 1's accuracy column is misaligned with the text (the text gives Lee et al. 62.26% at 5 minutes and 73% at 15, Silva et al. 63.91% and 83.54%, Ani et al. over 90%, Do et al. 75.10%, Gonzalez 82-90.48%); the cross-reports follow the text.

The Dota 2 rows are the same paper's second dataset. Majority-class baseline (BL-MC): 53.22% ± 0.16% LoL, 52.65% ± 0.12% Dota 2.. The weights are learned from match outcomes, so under 10-fold CV a player's weight is fit on that player's other matches, earlier and later. The paper's significance rule: mean1 − mean2 > 2(std1 + std2).

Features as defined in the paper: champion mastery points ('lifetime experience'), player-champion win rate, season games on the champion, and ranked games on the champion within the last 20. The paper does not say whether the season aggregates were read before each match or at collection time. Other models in Table 1: RF 74.7%, SVC 74.3%, kNN 72.7% (kNN uses only 'the team's average win rate'). The authors prefer the DNN over GBOOST because GBOOST's standard error is much larger.

Full tournament data published as drafts_scored.jsonl (exists; not downloaded). The post infers iTero's model ('consistent with a gradient-boosted tree ensemble over lolalytics-style aggregates') from 'its API response signature'; that is a rival's inference, not iTero's statement. Its feature table says iTero offers 'per-candidate score only; no team WR', which is why no calibrated metric could be scored for iTero. No real match outcomes are involved, so none of these numbers is a forecast score.

Secondary source: it relays the Reddit post https://www.reddit.com/r/leagueoflegends/comments/1k5hmlf/ai_can_predict_the_winner_of_767_of_your_games/ (not read; Reddit is unreachable to the fetcher). pregame is 'unclear' because the rank feature is taken after the games were played.

Retracted Reddit post: https://www.reddit.com/r/leagueoflegends/comments/1joumtm/i_made_an_ai_model_that_predicts_62_of_ranked/ (not read; Reddit is unreachable to the fetcher). The author thanks '/u/Impossible_Concert88 for trying to verify the accuracy claims' and writes: Queue: the post says 'ranked' (from the Reddit title) without naming solo/duo; level set to solo queue on that basis.

Training logs linked from the post (not opened here): solo queue https://wandb.ai/loyd-team/draftking/runs/jy5hf0bv?panelDisplayName=val_win_prediction_accuracy&panelSectionName=Charts; pro play https://wandb.ai/loyd-team/draftking-pro-finetune/runs/jg3ls0xp?panelDisplayName=val_pro_win_prediction_accuracy&panelSectionName=Charts. 'We mask randomly because the riot data doesn't have information about the original pick order'... 'the model is not aware of "blind pickability".' The page header 'Last model update: September 19 on patch 16.18' is the site-wide banner for the current model, not the one the post describes.

FAQ answer in full: No test set, split, n, curve or ECE is given anywhere on the page. The landing page also shows 'Outperforms DraftGap · Outperforms iTero · Outperforms LoLTheory' badges and a 'Side by side' table whose 'Calibrated win %' row gives LoLDraftAI a check and DraftGap, iTero and LoLTheory a '?'; no metric is attached, so no cross-report is recorded. 'Read the full breakdowns' links to the three comparison posts. Other page claims: 'Re-trained weekly', and.

. The abstract does not say how the data were split or whether the win rates exclude the predicted match. Queue not named ('ranked matches').

Hero lineup / draft-only claim of 75.32% accuracy sits far above the honest draft band (53-56%); split and data scope unstated in the abstract, indicating strong likelihood of whole-dataset feature calculation.

Other Table 3 rows (test %): post-draft AutoLog+Rolling+LR 72.0, AutoLog+LR 71.8, Loginit+Rolling+(momentum)+LR 70.8, Rolling+(momentum)+LR 68.8, LR baseline 68.3; pre-draft teams AutoLog+Rolling+LR 65.6, AutoLog+LR 65.1, LR baseline 62.3, MTL+RNN 61.6; pre-draft solo AutoLog+Rolling+LR 54.03, AutoLog+LR 53.59, LR baseline 53.38, AutoLog+MTL-TL+RNN 52.48, AutoLog+RNN 52.38. Participant profiles were loaded from op.gg 'Once a valid match is found' (season totals and up to the last 20 games); the paper does not say they exclude the target match. Stated limitation.

Feature importance per the abstract: meta strength 26.3%, team synergy 24.8%, counter-matchups 20.6%, runes and spells 6.7% combined.

The Dota 2 rows are the same paper's second dataset (Dota 2 is not this file's game). Generic baselines in the same tables, not listed above: MC (majority class, 'Blue for LOL and Radiant for Dota2') 0.5040 LoL and 0.5180 Dota 2; LR 0.5255 during and 0.5323 after the draft (LoL), 0.5750 and 0.6126 (Dota 2); NN 0.5263 and 0.5335 (LoL), 0.5748 and 0.6108 (Dota 2). After the Dota 2 draft, LR (0.6126) prints above DraftRec (0.6110). Outcome prediction is the paper's secondary task; its main task is champion recommendation. Code and data: github.com/dojeon-ai/DraftRec.

'This is model-predicted lift, not measured outcomes' — the model scores its own suggestions; no game result enters these numbers. Naive baseline: 'uniformly at random among the revealed slots'; strongest-visible. The model 'implicitly assume[s] the opponent's downstream picks don't change in response to ours'. Only Bottom and Utility (roles) and Silver and Master+ (elo) are broken out in the text; no overall average is printed. This study is the source of the homepage's '+4–7%'.

Level set to solo queue from 'ranked games in silver elo' (queue not named). 'still in the experimental phase'. Code not audited here; whether any input leaks the outcome is not answerable from the README.

'This is a correlation, not a causal experiment. We didn't tell anyone to follow the recommendation.' 'That's a 5.4-point spread between top and bottom' (Top 10). DraftGap left out ('static aggregate stats... rather than a learned model'); LoLTheory left out ('LoLDraftAI published a head-to-head where they outperformed it'). The article describes iTero's model as 'gradient-boosted trees with linear models'. Only the anonymous winrate.gg model was tested; the personalized model (~40 more features) was not. The realized win rate is not an accuracy, so the five metric fields are null.

Notre vérification : The article does not say whether the statistics behind winrate.gg's own suggestions excluded the 12 hours of games being tested.

Secondary source: it relays the Reddit post https://www.reddit.com/r/leagueoflegends/comments/1k5hmlf/ai_can_predict_the_winner_of_767_of_your_games/ (not read; Reddit is unreachable to the fetcher). pregame is 'unclear' because the rank feature is taken after the games were played.

README... Found via the lolninja.net article's code link; one fetch only, not chased further. The two players' profile links in the README are left out here.

Other Table 3 rows (test %): post-draft AutoLog+Rolling+LR 72.0, AutoLog+LR 71.8, Loginit+Rolling+(momentum)+LR 70.8, Rolling+(momentum)+LR 68.8, LR baseline 68.3; pre-draft teams AutoLog+Rolling+LR 65.6, AutoLog+LR 65.1, LR baseline 62.3, MTL+RNN 61.6; pre-draft solo AutoLog+Rolling+LR 54.03, AutoLog+LR 53.59, LR baseline 53.38, AutoLog+MTL-TL+RNN 52.48, AutoLog+RNN 52.38. Participant profiles were loaded from op.gg 'Once a valid match is found' (season totals and up to the last 20 games); the paper does not say they exclude the target match. Stated limitation.

In-game Stage 4 results exist but use live game state, so they are left out of self_reported: '73.7% (80.5% at 25 min)' against a 56.3% baseline (28 features); live variant without XP 72.9%; isotonic re-test (2026-08-24) 'ECE 0.0351 → 0.0160', accuracy '77.2% → 77.3%'.. 'There is no server any more (since 2026-09-13)'; the public site ra1nforest.github.io/LOL_Predictions runs the stages in the browser. No forward record is published.

V1–V3 log loss is not printed. Top V4 feature: 'patch_wr_delta_diff' at 25.3% importance . The live demo (hget15.github.io/lol-esports-predictor) runs, not the evaluated model. The README credits V4 with 'the single biggest jump — AUC 0.698 → 0.797'.

Notre vérification : We read the code (upstream commit e50e9ce, 2026-09-20). The V4 features come from trackers filled with every game of 2021-2026 before any feature is computed, so a team's win rate on the current patch includes the game being predicted and the games after it (src/v4_enhancements.py: build_all_from_raw, line 223; get_patch_wr, line 85). The V1-V3 results do not use those features.

AUC is the only metric. Gini importance from the RF models on the Complete set ranks the blue jungler's win rate on the picked champion first (Figure 4). The player statistics come from. Author name printed 'Fracisco' in the paper.

The abstract: the accuracy is.

In-game Stage 4 results exist but use live game state, so they are left out of self_reported: '73.7% (80.5% at 25 min)' against a 56.3% baseline (28 features); live variant without XP 72.9%; isotonic re-test (2026-08-24) 'ECE 0.0351 → 0.0160', accuracy '77.2% → 77.3%'.. 'There is no server any more (since 2026-09-13)'; the public site ra1nforest.github.io/LOL_Predictions runs the stages in the browser. No forward record is published.

Numbers match the repo. In the same Table IV the outcome-only 'PScore + OpenSkill' row prints 65.56% on all games and 65.51% intra-region, above PandaSkill's 64.98% and 64.86% (it trails on inter-region games and on ECE); the paper's own summary: ECE bins are not stated. Generic EWMA rows (all / intra / inter): PScore + EWMA 63.06 / 63.33 / 52.34 (ECE 1.43 / 1.57 / 5.51), PI + EWMA 63.53 / 63.75 / 54.68, PlayeRank + EWMA 62.25 / 62.80 / 51.67. Expert survey (not a win metric): 80.63% majority concordance (SD 6.52), 88.98% unanimity concordance (SD 5.99). The repo's 'chronological replay' is, in the paper's words, a one-month rolling window with a logistic regression trained on the previous year.

V1–V3 log loss is not printed. Top V4 feature: 'patch_wr_delta_diff' at 25.3% importance . The live demo (hget15.github.io/lol-esports-predictor) runs, not the evaluated model. The README credits V4 with 'the single biggest jump — AUC 0.698 → 0.797'.

Notre vérification : We read the code (upstream commit e50e9ce, 2026-09-20). The V4 features come from trackers filled with every game of 2021-2026 before any feature is computed, so a team's win rate on the current patch includes the game being predicted and the games after it (src/v4_enhancements.py: build_all_from_raw, line 223; get_patch_wr, line 85). The V1-V3 results do not use those features.

The paper names its score RPS and says. No accuracy, AUC or ECE is reported. Table 6 gives single-feature ROC AUCs of end-of-game and 15-minute features against the outcome (e.g., final gold 0.997, towers 1.000, gold at 15 minutes 0.784); those are associations, not forecasts. On the market window the two architectures tie (Δ = +0.0003 on 963 walk-forward games). Rank validity (Spearman, 50 teams): 0.438 proposed, 0.436 candidate.

no forward record, P&L or ROI is published. Unit: the backtest table counts 'Games'; a 'Bo3/Bo5 series probability extension' sits on top. Author's limitations include 'Champion draft not modeled' and 'Player-level ratings not used'.

Hit rate and return go in other_metrics and accuracy is left null because the hit rate covers selected picks, not all games. The methodology page's 585/895 equals the sum of the four models' 'All leagues' match-winner cells (176+146+130+133 of 266+229+197+203, my arithmetic; the page names no model). Consensus is 'a weighted blend of the three above, weighted by each model's recent log loss'; log loss itself is not published. Other markets on the page are not win predictions and are not recorded (e.g. Total Maps LightGBM 100% 14/14 +97.9% B 0.154; Series Handicap Winner; Exact Score; total-kills markets). A 'Download CSV' link exists (not downloaded). Every figure moves as matches resolve.

Log loss counts a draw as 0.5; accuracy excludes draws; LoL draw rate 0.0261 (Table 1). The dataset card (last modified 2026-08-09) lists releases 1.0 to 10.0; its 10.0 line reads (train and test overlap as printed) while its intro says 'Date is complete up to 2026-03-31'. The paper hopes EsportsBench could become 'a dynamic or near-real-time leaderboard' (future work). Rating systems are implemented in the author's riix package.

About page (https://leagueofelo.com/about): 'AR is derived directly from Brier Score'; 'XP = min(1, 1/3 log10(N))'; 'a 50/50 prediction yielding 75 points'. The highest human account on the All-Time board reads Adjusted AR 79.16, Up/Down 68.4% on 1378 matches (a single user, not a crowd aggregate, so not recorded as a cross-report).

No probability score is reported (results.csv holds probabilities; the README reports accuracy only). README: 'We are determining the "stronger" team and saying that it will win' and 'In the result's file, sometimes a > 0.5 probability is listed and the team still wins. Indeed, there was an error in the code, but the prediction output teams were always correct.' Whether a row is a game or a series is not stated.

Intro: Pre-match caption. Hermes caption. ✓ marks the lower Brier in each row. Two pre-match rows (EMEA Masters Sep 2026, LFL Aug 2026) show '—' for the market, so they have no cross-report. Two Hermes rows are printed truncated as 'North Amer' and 'Prime Leag'. No log loss, AUC or ECE is published, and the page never says whether a prediction is for one game or a series ('match' is not defined). Numbers change as games resolve.

No margin-of-victory term appears in this article (the Stomp Factor arrives with the 2025 revision; see riot-gpr-2025-26-revision).

The level rests on the abstract's wording ('the outcome of the championship'; metrics 'based on the defining characteristics of professional-level matches'); the dataset, unit and split are not described.

The README says predictions cover 'game or match win probabilities' (unit not fixed). 'this model is intended purely for academic purposes'.

Random-forest baseline (Table I, lookback of 1, 3, 5 previous games): raw 0.563 ± 0.009, 0.573 ± 0.010, 0.572 ± 0.005; delta 0.560 ± 0.009, 0.569 ± 0.008, 0.578 ± 0.012; Table IV lists 'Random Forest (lookback=5) + delta 0.578'. On timing the paper says both that (Section I) and that a one-convolution GCN's 'fair' prediction is 'predicting two games ahead', with labels attached 'from future game results' (Section IV-C, Fig. 3); hence pregame 'unclear'.

The abstract concludes that more accuracy 'would require substantially more data and contextual information'. MMU e-space lists a camera-ready copy (e-space.mmu.ac.uk/629272/1/EAI_ArtsIT_Robbie___cam_ready.pdf) whose download redirect expired before it could be read.

The Dota 2 rows are the same paper's second dataset (Dota 2 is not this file's game). Generic baselines in the same tables, not listed above: MC (majority class, 'Blue for LOL and Radiant for Dota2') 0.5040 LoL and 0.5180 Dota 2; LR 0.5255 during and 0.5323 after the draft (LoL), 0.5750 and 0.6126 (Dota 2); NN 0.5263 and 0.5335 (LoL), 0.5748 and 0.6108 (Dota 2). After the Dota 2 draft, LR (0.6126) prints above DraftRec (0.6110). Outcome prediction is the paper's secondary task; its main task is champion recommendation. Code and data: github.com/dojeon-ai/DraftRec.

In-game results of this version, not recorded as rows because they are not in benchmarks.md §7: Mixed-InGame best 76.17% (RF, CFS selection) and Pro-InGame best 75.22% (LR, single feature Kills R-D), 20-minute mark (Table 3). The authors conclude Hero-model parameters were compared 'on the training data set'; for the in-game models they.

The Full Heroes Model is Conley & Perry's hero-presence regression, antisymmetrized, re-run on this report's data; it is kept as a self-report because it is the report's headline. The authors note its top five heroes.

The Dota 2 rows are the same paper's second dataset. Majority-class baseline (BL-MC): 53.22% ± 0.16% LoL, 52.65% ± 0.12% Dota 2.. The weights are learned from match outcomes, so under 10-fold CV a player's weight is fit on that player's other matches, earlier and later. The paper's significance rule: mean1 − mean2 > 2(std1 + std2).

The abstract says the best model's performance, meaning the bettors' 62.8%, which is measured on a different unit. Data were collected with the OpenDota web API (contributions section).

Under the owner's rule no error rate is converted to an accuracy here. The authors warn that the combo improvement 'might because of the increasing number of match data' (6,000 against 3,000 matches).

A baseline of always predicting a radiant win scores 'around 63%' (text; drawn in Figure 7). Recommender success rates, all judged by the neural network: Table 2 (random enemies) 56.9%, 60.5%, 73.4%, 76.4%; Table 3 (enemies among the 10 most picked) 59.9%, 60.2%, 71.7%, 74.9%.

The authors note the game version did not change during collection. Their web interface shows 'Chance we win based on picks: 96%' for an example lineup; that is a demo, not a metric.

Wan & Sándor 2023 (§2.2, reference [17]) quote the same two figures. A search-engine summary, not the source, said the classifier was tested on the UCI Machine Learning Repository DOTA2 dataset; unverified.

Secondary evidence only, not the source itself: Hodge et al. (IEEE ToG) Table I lists this paper as 'FM & XGB', 5,071,858 games, skill 'Normal-V. High', '70%**', footnoted as having 'calculated AUC rather than accuracy'. Akhmedov & Phan 2021 (arXiv:2106.01782) say it was evaluated 'on ten-fold cross-validation' and that; if right, the split is random k-fold and ~0.70 belongs to XGBoost, which conflicts with the abstract's FM-best. A search-engine snippet, not the paper, gave 0.706 / 0.670 / 0.660 for normal / high / very high skill. The authors' preprint (dotascience.com/papers/aist2016, linked from their CEUR-WS Vol-1842 review) no longer resolves, and the Springer full text is behind a subscription. 'public matchmaking' rests on the Normal / High / Very High skill brackets named by Hodge's table and the repo; the abstract says only 'skill level of the players'.

Generic baselines omitted from self_reported: Gradient Boost 67.76 %, Logistic Regression 67.21 %, Random Forest 63.36 %, Gaussian Naive Bayes 60.49 %, KNN 54.37 % (Table 3). §4.1 prints confusion counts (991, 574, 500, 1183) beside; the counts alone do not reproduce 73%, and the thesis does not say which split they come from. It does not say whether the Word2Vec vectors and the log5 counter matrix were built from training matches only. Keeping only games of 20 minutes or more selects on game length, known only after the game. The literature review also quotes Hanke & Chaimowicz's recommender 'success rate of up to 74.9%', which is not a prediction metric.

Read from the ar5iv HTML. The player features are the players' MMR scores and percentiles; test matches are excluded from the match-history statistics, but no cut-off at each match's start time is stated. The 93.73% real-time result is in yang-2016-realtime.

The authors themselves say the results 'should be taken with a grain of salt' because predicting is very difficult, and that the web dataset. The paper counts 138 heroes and uses raw hero IDs per slot as features.

The printed split counts (17,000 / 1,500) do not match the 30,426 games collected. The author notes that plain logistic regression scores higher on test than on training data and suggests. The genetic algorithm starts from 'an initial population of 220 matches' and the hero communities come from 250 matches; the report does not say these came from the training split only. Hodge et al. (ToG) Table I lists this work at '91%' on '15,146' games, which matches neither figure here.

Needs a real browser. The 77% is taken from the title, which the search index shows as.

Generic baselines in Table 1 (accuracy, Casual / League / Elite): LR 0.5657 / 0.5040 / 0.5912; MLP 0.5753 / 0.5153 / 0.6041; LSTM 0.5931 / 0.5231 / 0.6336; Transformer 0.6012 / 0.5429 / 0.6528. OwO is kept as a self-report because the abstract calls Cupid 'Our prior work'. The online A/B test 'spanned over 30 days'; the novice segment saw 'a 20.73% reduction in 5-minute kill crushing', which is a matchmaking outcome, not a prediction metric. The game is not named. Read from the arXiv HTML.

The body is in Indonesian (abstract in English); decimal commas are as printed. Split into two records (this draft phase and firmansyah-2025-ingame) because the paper holds both a pregame and an in-game model. Training data: 553 games from six qualifier tournaments, 54 teams.

No absolute metric is printed in the text, so every metric field is null. Read from the ar5iv HTML through targeted extraction. The paper calls the game both 'Honor of Kings' and 'King of Honor'.