Spring training and the fallibility of sabermetrics

Spring training and the fallibility of sabermetrics

Published Mar. 6, 2015 5:12 p.m. ET

It's been too long since I've mentioned this: Dan Rosenheck is one of my favorite people.

Dan sorta flies under the radar in these parts, because he doesn't publish often, and when he does publish it's just under his initials (because of editorial policy where he works). But he's crazy smart and enthusiastic and opinionated, which makes for a delightful combination. For me, the worst thing about missing the MIT-Sloan Analytics Conference last week was not getting to see Dan.

Anyway, he's just published his latest research about the predictive qualities of spring-training statistics, and I'm not even going to tell you what's in there, except you should read the piece for yourself. 

Dan's a bit of a rabble-rouser, which is something else I admire (as long as it's backed up with actual data). Here's the part I will share with you:

ADVERTISEMENT

The first generation of quantitative baseball researchers -- ”from Bill James, who more or less invented the discipline, to the whiz kids in the Oakland Athletics' front office featured in the Brad Pitt film "Moneyball" -- ”delighted in refuting conventional wisdom about the game that players, scouts and managers had recited for decades. This group made a number of highly valuable contributions: for example, they correctly accused major-league clubs in the 1990s of undervaluing on-base percentage, and they rightly continue to criticise managers who refuse to use their best relief pitchers in tie games.

However, they also committed a number of blunders. They said that teams overstated the importance of fielding; that drafting players out of high school rather than college was too risky; that pitchers had no ability to induce weak contact; that catchers had no measurable impact on the performance of the pitchers that threw to them; and of course that spring training statistics didn'™t matter, ”and that anyone who insisted otherwise was a troglodyte whom history would soon consign to baseball'™s Stone Age. In subsequent years, as researchers have gained access to richer and more granular datasets, each of these claims has either been watered down or outright refuted. The much-mocked "traditionalists" were at least partially right all along; the "œnerds in their mothers'™ basements" conducting the quantitative analyses simply lacked sufficient data to detect what people who had lived the sport were always able to see with their own lyin'™ eyes.

Frankly, I'm a great deal more humble than I was, say, 15 years ago when I started doing this sort of work. I think most of my like-minded colleagues are pretty humble, too. Mostly because we've been proved wrong so many times. Still, it's always good to be reminded that we've been wrong before, and will be wrong again. To quote Voltaire, "Uncertainty is an uncomfortable position. But certainty is an absurd one."

share