A few days ago, the world's best podcast co-host tweeted an expression of sympathy for managers, who have to make incredibly difficult decisions for the derisive pleasure of us know-it-alls:
This raises an interesting question, one that the world's best podcast co-host and I talk about constantly on a podcast together: Are these decisions really complicated? Does the fact that we can't possibly know every factor that might affect a decision -- who is pitching with elbow pain that he hasn't yet admitted to; how the psychology of recent success or failure might conceivably matter in the individual case of a single pitch or plate appearance; the power of group dynamics and individual roles that a manager has established over the course of many months; so on and so forth, for eternity -- make these decisions more complicated? Or do all those unknowns make the job less complicated, because 99 percent of the work has been done and the final one percent is just a thornbush of competing hypotheses that are best ignored?
(I'm very sorry for this long lead-in. This article is about whether Madison Bumgarner should pitch in relief on his throw day this afternoon. Look forward to me talking about that!)
So at this point the supremely intelligent scold MGL replies to Ben Lindbergh and puts it thusly:
@BenLindbergh I actually don't think they are difficult decisions at all. That's the beauty of managing optimally.
— Mitchel Lichtman (@mitchellichtman) October 7, 2014
I think there's merit to this point, but there certainly are some situations where the decision is difficult. For instance: Sixth inning, James Shields on the mound, two on, nobody out, and Ned Yost has to decide whether to bring in Yordano Ventura.
The decision to pull Shields, I think MGL would argue, is not that difficult. The through-the-lineups effect (pitchers get worse), along with the advantage that the reliever role confers upon otherwise inferior pitchers (relievers are more effective than starters) makes it pretty clear that the average good reliever is going to be better than even a very good starter going through the lineup a third time.
Further, we know that starters are, in fact, better pitchers than relievers, and if they are moved to the bullpen those starters will pick up the same bullpen boost. So a very good starter (like Ventura) in the bullpen is probably better than most relievers, and most relievers are probably better than a tiring starter (like Shields). Therefore, Ventura in the bullpen is almost certainly better than Shields as a tiring starter. This decision seems very simple. Except:
Short rest. Ventura was pitching on short rest -- most likely on his "throw day," the day pitchers generally work in the bullpen at about 80 percent strain between starts -- having started two days earlier and thrown 70-plus pitches. In The Book, the authors (including MGL) found that starters who made emergency relief appearances were actually worse as relievers than as starters: "Because these were emergency situations, they might not have even been fully prepared to pitch. As well, they might just be out of their element when relieving, and thus don't get the normal reliever advantage."
The former wouldn't apply to a pitcher whose manager made it clear he might be used on a throw day, and who was called on in a non-emergency situation (as Ventura presumably was). The latter might. Considering how common such situations are in the postseason -- at least one pitcher every postseason is called upon on his throw day -- we ought to be able to figure this out. So, to the question:
Should the Giants use Bumgarner out of the bullpen in Game 3, on his throw day?
Since 2000, there have been 35 starters who have pitched in relief in a postseason game on no day, one day, or two days' rest after making a full start. This is the largest group we'll use for these purposes, though occasionally weâll divide them into subgroups for reasons that will be clear.
It's not hard to find postseasons that turned on the manager's aggressive use of these starters pitching on throw days. In Game 6 of the 2010 NLCS, for instance, Bruce Bochy turned to a rookie starting pitcher named Madison Bumgarner (heyyyy) to pitch innings five and six with the game tied. Bumgarner had started Game 4, and thrown 85 pitches. But he was very effective, holding the Phillies scoreless. The Giants broke through in the eighth. By Win Probability Added, Bumgarner increased the Giants' chances by about 14 percent, making him the fourth-biggest hero of the game.
Speaking of that Game 4 that Bumgarner had started three days earlier: Phillies manager Charlie Manuel turned to a veteran starting pitcher named Roy Oswalt to pitch the ninth inning of a tie game. Oswalt was working on short rest. He was not sharp, and after a couple line drives he allowed the deep sacrifice fly that ended the game. By Win Probability Added, nobody in the game did more than Oswalt to influence the outcome.
Twice, then, the series hinged on a starting pitcher relieving on short rest. Once it backfired. Once it saved the game. Which was more likely?
Let's go back to the 35. As a group, they pitched 44 innings. This is a group that includes Josh Fogg and Rick Helling, but it also includes Roger Clemens and Randy Johnson. Eyeballing it, these are mostly very good pitchers, something that is to be expected. They were good enough to make a postseason rotation, and good enough that their managers broke protocol to get them into a game. How well did they do? In their 44 innings, they had a 2.86 ERA. They struck out 8.2 batters per nine innings, walked 2.7 per nine, and allowed only one homer every 15 innings or so innings. Compared to the average reliever in the same time period -- 3.69 ERA, 8.4 K/9, 3.7 BB/9, and a homer every nine innings -- you'd say the managers were pretty happy with their decisions.
But that all tells us nothing. It was a quick and simple first step that is pointless in answering the question of whether Ventura was the right call there for the Royals. Pointless for Objection No. 1: One is that we don't really know how well these pitchers should have done -- if they were mostly ace-level pitchers, then for all we know pitching out of the bullpen on short rest made them worse. If that's the case, maybe we would have expected Ventura, who is good but not "ace level," to be worse -- and if that's the case maybe Shields, or some other reliever, would have been better. Also pointless for Objection No. 2: 44 innings of ERA and junk isn't enough data to conclude anything. We can fix both of these objections.
Objection No. 1: Expectations.
The first subset we're going to create is pitchers since 2003. This subset is the PECOTA era, which means I am able to generate a "true talent level" for each pitcher using PECOTA's preseason projections the following season. These are each starting pitcher's projections as a starter. I will weight each pitcher's projection based on how many batters he faced in relief, and see whether the pitchers, as a group, did better than expected.
So, building spreadsheet; weighting; calculating; comparing; and:
They did! The expectations for these pitchers as starters, based on their true talent levels and weighted by how much of the overall group's relief work each of them accounted for, is a 3.85 ERA, 7 K/9, 2.7 BB/9, 0.9 HR/9, and (roughly) a 3.90 FIP. Instead, the 2003-2013 pitchers produced a 2.93 ERA, 8.6 K/9, 2.9 BB/9, and 0.7 HR/9, with (roughly) a 3.25 FIP.
And here's the important thing: Tom Tango, in estimating how much better a pitcher should be coming out of the bullpen compared to starting, coined the Rule of 17: "Difference in BABIP is 17 points higher as starter. K/PA is 17% higher as reliever. And HR per contacted PA is 17% higher as starter. Walk rate is FLAT." And, as a rule of thumb, pitchers allow about one fewer run per nine innings when they're working out of the bullpen.
So what do we see? K rate and HR rate each get a bit more than 17 percent better in relief, walk rate is basically flat, and the pitchers allowed almost exactly one fewer run (well, at least earned run) coming out of the bullpen.
This strongly suggests that starters working on their throw day as relievers are just as effective as they would be expected to be coming out of the bullpen on full rest. We know that Ventura would generally be a relief ace coming out of the bullpen. This all suggests that the math doesn't change if "short rest" is introduced. Yost could have reasonably thought he was putting in a relief ace.
Objection No. 2: Sample size.
But as convenient as the results are, they're still based on 30-some innings and 150 batters faced, not nearly enough to draw sweeping conclusions. So let's switch to a new sub-group: The post-2008 pitchers. This is the PITCHf/x era. Using these 13 pitchers, we can look at something that should stabilize much more quickly than ERA.
A rule of thumb -- not quite so well established as the Rule of 17 -- is that a starter who goes to the bullpen picks up about two extra miles per hour on his fastball. But these pitchers were on short rest, and we're trying to figure out whether relieving on short rest saps pitchers of some of the advantage they would otherwise gain in the move. So: Do their fastballs add two mph?
In short, no. In long: Sometimes. Comparing the pitchers' bullpen velocities to their fastball velocities in the previous start, the 13 gained, on average, just 0.5 mph on their four-seam fastballs (from 93.9 to 94.4), and the same amount on their two-seam fastballs (from 92.7 to 93.2).
But the average belies the variance. Jordan Zimmermann added 2.6 mph to his fastball. John Lackey lost 1.4 mph from his. Oswalt, who pitched poorly in relief in 2010, gained more than one tick. Bumgarner, who pitched so well in relief in 2010, lost a half-tick.
For what it's worth -- and, considering how small we've shrunk our sample, it's not worth anything -- these 13 pitchers didn't do well in their relief appearance. They were actually much worse than we would have expected, runs and peripherals. Their numbers were overwhelmed by the impressive results of the 2000-2007 pitchers, for whom we don't have velocities to compare.
Like I said: That's not really significant. Tiny, tiny samples here. So forget I even mentioned it, even if I don't quite have the integrity to just delete it altogether. But given the range of velocity changes we saw for pitchers throwing relief innings on their throw days, is it a logical leap too far to conclude that some pitchers, on some days, are able to add velocity in relief, even on short rest, and that those pitchers should do very well; while some pitchers, on some days, aren't, and quite possible won't? It's certainly a reasonable hypothesis.
If that hypothesis is true, then we have no choice but to conclude that, in this situation, the decision of whether to use a pitcher on his throw day involves personal factors, the ability to "scout" a pitcher while he is warming up, and who knows what other factors that might resist a simple yes/no decision tree. Should the Giants use Bumgarner as a reliever on his throw day in Game 3? They should probably strongly consider it. I'd just consider it a difficult decision.
