The fantasy baseball industry is more popular every day, but it sometimes seems rooted in yesterday. The statistics have moved on since it was invented in the '70s, at least. Is today's fantasy baseball too far removed from real baseball? Is it just fake?
Fantasy's traditional scoring system does seem arcane. That old 5x5 fantasy rotisserie game uses runs, RBI, batting average, home runs and stolen bases as the key batting statistics, and many of those stats have fallen by the wayside as we attempt to better evaluate players. Runs and RBI, in particular, are not consulted at all when it comes to the modern stats of the day. They are just too context-dependent, since your teammates are heavily involved in both.
On the pitching side, the story is the same. Wins are one of the five categories, and a prominent numbers-savvy analyst has declared war on that statistic. Strikeouts, like home runs, are remarkably clean in that they require two participants and no judging from a scorer. But WHIP (Walks plus Hits over Innings Pitched) is full of noise -- each hit is not only declared as such by a scorer, but it's made into a hit through some nebulous combination of pitching and fielding as well.
But if that makes you want to pet fantasy baseball on the head, you might be surprised by how well those 10 "old" statistics track with the more modern versions we have today.
Over at FanGraphs, we're debuting a fantasy auction calculator and ranking tool today. The basic mechanism that powers the calculator is an attempt to look at the spread of certain statistics over a common baseline, look at a player's production in each stat compared to that spread, and then adjust for position. If that sounds familiar, it's because it generally follows the roadmap for Wins Above Replacement, one of today's more sophisticated and complicated advanced summary stats.
If you compare the results of that auction calculator using 5x5 roto stats -- batting average, home runs, RBI, runs, stolen bases, wins, ERA, WHIP, strikeouts and saves -- to the current WAR projections for next year, the two valuations are actually fairly close. In laymen's terms, the "old" 5x5 stats predict over three-quarters of the variance in WAR projections. That's a strong relationship.
Here are the relevant correlations for a few different sets of scoring systems. (All p-values less than .0001, r value shown.)
| Correlations to WAR | 5x5 | SABR points | Scoresheet |
|---|---|---|---|
| Hitters | 0.872 | 0.877 | 0.718 |
| Pitchers | 0.789 | 0.569 | 0.687 |
SABR Points is a points system designed by Justin Merry for FanGraphs' ottoneu fantasy game. It's derived from the same linear weights style research that created weighted on-base average (wOBA), so it's not surprising that it's slightly more correlated with WAR for hitters. Scoresheet is a game that includes defense, but the rankings were not produced by the same calculator, so perhaps that's why we see the lower correlations for that game.
Of course there are many problems with this analysis. We're using the same basic stats on both sides of the ledger, even -- Steamer projections. Batting average includes singles, doubles and triples, and those go into wOBA and then into WAR. So we're correlating these against themselves, in a way.
And we're also testing our ability to rank fantasy players correctly, as you can perhaps see by the very different correlations we got when we went with a different set of rankings.
On the other hand, this isn't a piece of research meant to change baseball -- we're just trying to see how far we've come. And all the stats that are in both sides of the correlation are weighted differently. Batting average treats a triple the same as a single, while wOBA more than doubles the value of a triple with respect to a single. So by testing one rating system against the other, we're really testing to see how far our two valuation systems have diverged.
We've worked very hard to improve our statistics, and we have. Any improvement is good, particularly when you are tasked with running a real-life team. But we haven't pushed the stats *that* much further from our old boring 5x5 roto categories.
There are some parts of fantasy that even more closely mirror real life baseball than the sabermetric work meant to analyze that real game on the field. Take relievers, for example. All fantasy systems value the save -- even the sabermetric points fantasy game puts value on the saves statistic -- and therefore relievers become more expensive in fantasy.
Look at the 5x5 values of Aroldis Chapman ($24.80, ninth-best pitcher) and Craig Kimbrel ($19.80, 14th-best), for example. Dellin Betances was last year's best reliever by WAR (3.2) and he was only good for 34th-best among all pitchers with more than 50 innings. Only 20 pitchers are currently paid more than Jonathan Papelbon, so the market seems to value relievers more highly than WAR. Though Papelbon has been worth $19 million by FanGraphs' dollars per WAR feature, he's been paid nearly double that. And that mirrors my finding going into the 2010 season that only nine of the last 33 relievers had been worth their contracts by $/WAR.
It's certainly possible that some general managers continue to overvalue closers despite evidence they aren't worth their contracts. But it's also possible also that our advancements in valuing relievers lag behind the reality of the game. As "bad" as the save stat may be, it could help us better value the handful of relievers who are consistent from year to year.
With the advent of more "dynasty" fantasy games, we're better approximating other facets of real-life baseball in our fake games. The game at FanGraphs -- ottoneu -- is a year-round auction "dynasty" game, where your players are bought for a price, and kept at that price with some inflation for as long as they are affordable. That mimics much of what a general manager has to consider when he thinks about surplus value, the cost of his players, and his window for contention. Play that sort of game, with linear weights stats, and you are doing a fairly good job mimicking the real life game, actually.
And so far we've ignored the research that fantasy baseball's fervor has inspired. Just this month on FanGraphs alone, fantasy writers have looked at how release point cluster tightness relates to pitcher outcomes, how the number of pitches a batter sees in the zone can predict breakouts, how to best translate Cuban plate discipline stats, and how batter times to first might show declining athleticism.
We've come a long way with our statistics. And yet, we haven't gone that far. A good old-fashioned box score game still does an adequate job valuing players. Fantasy baseball isn't so fake.
