Looking at the Bullpen: Shutdowns and Meltdowns

Not even in my most optimistic moments would have said that the Nats would win two in a row out of the gate! As I write this on Easter Sunday morning, the Nats are sitting pretty, sharing first place atop the National League’s Eastern Division with the Mets (the Mets!).

And all this despite a lackluster debut for Gio “the Motown Kid” Gonzalez. The Nats won yesterday behind the unexpected heroics of former Hiroshima Carp Chad Tracy, and some absolutely phenomenal pitching from the “B” bullpen, with Craig “Matinee Idol” Stammen in long relief, followed by Ryan “Firework” Mattheus, Tyler Clippard, and some pitching from Hot Rod that was pretty frickin’ bueno.

The Nats’ late-inning heroics aren’t great to my stomach lining, though. I’ve been wondering how I could better quantify the feeling I have when relievers come in. I attempted this earlier, of course, when I introduced my heartburn index–but I’m now convinced that the heartburn index doesn’t give a complete picture.

Fortunately, FanGraphs has ridden to the rescue again, with a new, and, I think, extremely helpful, pair of statistics for measuring relief pitcher performance: Shutdowns and Meltdowns. As the proponent of the new stats explains them:

Shutdowns (SD) and Meltdowns (MD) are two relatively new statistics, created as an alternative to Saves in an effort to better represent a relief pitcher’s value. While there are some odd, complicated rules surrounding when a pitcher gets a save, Shutdowns and Meltdowns strip away these complications and answer a simple question: did a relief pitcher help or hinder his team’s chances of winning a game? If they improved their team’s chances of winning, they get a Shutdown. If they instead made their team more likely to lose, they get a Meltdown. Intuitive, no?

Using Win Probability Added (WPA), it’s easy to tell exactly how much a specific player contributed to their team’s odds of winning on a game-by-game basis. In short, if a player increased his team’s win probability by 6% (0.06 WPA), then they get a Shutdown. If a player made his team 6% more likely to lose (-0.06), they get a Meltdown.

Shutdowns and meltdowns correlate very well with saves and blown saves; in other words, dominant relievers are going to rack up both saves and shutdowns, while bad relievers will accrue meltdowns and blown saves. But shutdowns and meltdowns improve upon SVs/BSVs by giving equal weight to middle relievers, showing how they can affect a game just as much as a closer can, and by capturing more negative reliever performances.

Nats fans are by now intimately familiar with WPA, thanks to the hard work of Federal Baseball. The squiggly-lined graphs he pots after every game show the ebb & flow of the game as measured by WPA. A “Shutdown” happens when a reliever bends the line towards the Nats’ favor. A “Meltdown” happens when a reliever bends the line in favor of the opponent. The Shutdown/Meltdown stat pair thus give us a good indication of whether a reliever is helping or hurting his ballclub–which is kind of neat!

So what does that mean for the Nats bullpen in 2012? Using my standard measuring interval (2008-2011 seasons), here’s how the pitching staff looks:

 Name  Holds  Saves  Blown Saves  Shutdowns  Meltdowns  Heartburn
 Brad Lidge  9  100  16  92  28  6.85
 Tyler Clippard  64  1  18  77  35  5.22
 Sean Burnett  54  8  9  63  42  5.62
 Drew Storen  13  48 7  59  22  4.34
 Henry Rodriguez  13  2  4  13  13  8.51
 Tom Gorzelanny  7  1  2  12  5  6.01
 Ryan Mattheus  8  0  0  7  6  5.63
 Craig Stammen  2  0  0  5  2  4.09

A few things jump out at me at once:

  • Since 2008, Brad Lidge is unquestionably the Shutdown King of the current Nats bullpen. The 100 Shutdowns mean that he left his ballclub in a better position to win after his appearance than before one hundred times–and only made them worse 28 times. This makes me wonder whether Philadelphia unloaded him more because of his relatively high heartburn factor than any other measurable quality as a relief pitcher. On the other hand, Lidge’s ridiculous 2008 season may have gone a very very long way towards inflating his stats here. In any case, Lidge was pretty good on opening day this year.
  • We all know that Tyler Clippard is an awesome relief pitcher. He was an all-star in 2011. But now we have a clearer idea why. He’s second only to Lidge in shutdowns since 2008, and leads the staff in Holds.
  • Sean Burnett has collected 63 shutdowns since 2008–apparently, while I was averting my eyes in terror. The more I study him, the more I am forced to conclude that I have been terribly unfair to Burnett over the past few years.
  • We also now have a better idea why Drew “Batman” Storen is such a good reliever. He hasn’t been relieving nearly as long as Lidge, but he’s already accumulated 59 shutdowns. His 2.68 Shutdown/Meltdown ratio is second only to Lidge’s.
  • Henry “Hot Rod” Rodriguez is, by this set of measures, not even nearly in the same class as Storen or Lidge. 13 Shutdowns and 13 Meltdowns, giving him an abysmal SD/MD ratio of 1.00–the lowest on the staff. I’m still hoping that he will improve during 2012 and pitch to his potential, though.
  • Tom Gorzelanny has a shutdown/meltdown ratio of 2.40. That’s fourth, behind Lidge, Storen and Stammen. I guess he really is better as a reliever than as a starter? Then again, he’s only recorded 12 shutdowns, total–so maybe we don’t know enough about him to judge.
  • I was expecting a tighter correlation between high shutdown numbers and low heartburn index numbers. That’s not what we see. Lidge, for instance, ought to give me more heartburn than his shutdown numbers suggest. Mattheus looks pretty bad next to his heartburn near-equivalent Burnett–but then, Mattheus hasn’t had all that many chances yet.

If the Nats’ starting rotation can routinely get through 6 or 7 innings, there are enough high-shutdown arms in the bullpen to keep the game in hand. This is very encouraging news for the rest of 2012.

Preseason Heartburn Rankings

Well, after watching Tom Gorzelanny and his Terrible, Horrible, No Good, Very Bad Day on Sunday, I’ve spent some time thinking about the Nats pitchers.

Specifically, what is it about certain pitchers that gives a Nats fan heartburn as they take the mound? Why am I always so apprehensive when Gorzelanny comes into the game? Because I remember days like Sunday, when Gorzo serves up a smorgasbord of pitching futility: walks, wild pitches, home runs.

So I got to thinking: what if I ranked the Nats pitchers by their propensity to give up home runs–that is, by their HR/9 rate? Well, if we look at totals since 2008 (my standard measuring interval on this blog), the list looks like this:

  1. Atahualpa Severino (1.93)Throw this one out for exceedingly small sample size–only 4.2 IP
  2. Tom Gorzelanny (1.19) As expected, Gorzo ranks high here.
  3. Tyler Clippard (1.08)
  4. Craig Stammen (1.00)
  5. Edwin Jackson (0.97)
  6. Jordan Zimmermann (0.95)
  7. John Lannan (0.93)
  8. Gio Gonzalez (0.92)But wait, you say: Gio pitched in the Oakland Coliseum until this season! Yup. But the Coliseum is actually just a bit smaller than Nats Park: 400 ft. to straight-away center (402 in Nats Park). Nats park is a bit smaller to right-center than the Coliseum, though.
  9. Yunesky Maya (0.92)Wait, what?!
  10. Chien-Ming Wang (0.86)
  11. Sean Burnett (0.85)
  12. Brad Lidge (0.84)
  13. Ross Detwiler (0.79)
  14. Ryan Perry (0.78)
  15. Drew Storen (0.76)
  16. Stephen Strasburg (0.49)
  17. Ryan Mattheus (0.28)
  18. Henry Rodriguez(0.28)
  19. Cole Kimball (0.00) Throw this one out for small sample size, too: only 14 IP

There are more than a few surprises in that list. I don’t think I’ve met a serious Nats fan yet who told me that he felt as queasy seeing Clippard enter a game as he did watching Gorzelanny–and yet, based purely on the home-run rate, Clippard should be giving me as much heartburn as my third half-smoke.

There’s got to be a better way to think about pitcher-induced gastrointestinal distress. What happens if we add walks to the number of home runs given up? That wouldn’t be quite it, either. No, let’s add all of the bad things we can blame on the pitcher: home runs, walks, hit batsmen, wild pitches, balks. That ought to give me a better idea of how gut-wrenching a pitcher is. Just to be clear, then, the calculation looks like this:

\text{Heartburn Index} = (\frac{HR + BB + HBP + BK + WP}{IP}) \times 9

Note that I haven’t weighted any of these bad outcomes–I’m not really all that concerned about that yet. I just want a rough indicator of how much heartburn I’m likely to suffer. How do the Nats pitchers look now?

CORRECTION: I re-checked my figures this evening, and discovered a number of rounding errors. Most baseball stat lines express innings pitched as a pseudo-decimal. That is, “4.2” stands for “four and two-thirds of an inning.” I had neglected to convert that shorthand into decimals, resulting in rounding errors. Corrected figures below.

First, we’ll have to discard two relievers whose number of innings pitched is too small for any heartburn index numbers to be even remotely fair:

  • Cole Kimball. Heartburn Index: 9.00.
  • Atahualpa Severino. Heartburn Index: 3.85

Which brings us to the Heartburn Index Rankings for “qualifying” pitchers:

  1. Henry Rodriguez. Heartburn Index: 8.51.. Hot Rod shoots to the top of the heartburn ratings because of his high walk rate and league-leading wild-pitch numbers. Fortunately, he doesn’t give up all that many home runs. Let’s hope 2012 Hot Rod gets into a groove like he did in the 2010 LVBP season.
  2. Brad Lidge. Heartburn Index: 6.85.A high walk rate (5.55 BB/9) is the culprit here, as well.
  3. Ryan Perry. Heartburn Index: 6.58
  4. Tom Gorzelanny. Heartburn Index: 6.01. Gorzo looks like he’ll be used in the long-reliever role this year, but it bears mentioning that, were he a starter, he would have the highest heartburn index among Nats starters.
  5. Gio Gonzalez. Heartburn Index: 5.88. Same old story here: Gio’s walk rate and wild pitches give him the highest heartburn rating of any of the pitchers in the current starting rotation. Indeed, his heartburn index is a full 0.89 greater than the next-nearest candidate for a starting rotation spot, Ross Detwiler. Good thing Gio’s got excellent strikeout stuff (8.59 K/9).
  6. Yunesky Maya. Heartburn Index: 5.68 I don’t think we’ll be seeing an awful lot of Maya at Nats Park this summer, for obvious reasons. But if time and chance bring Maya back to DC, let’s hope the long-range weather forecasts are right and that the summer will be a bit cooler, since he apparently wilts in the heat.
  7. Ryan Mattheus. Heartburn Index: 5.63
  8. Sean Burnett. Heartburn Index: 5.62. I’ll admit, these two puzzled me. If I saw Mattheus trotting to the mound, the game never seemed too far out of reach for me. On the other hand, watching Burnett make his entrance always filled me with dread: “Oh no, here we go again…“. Maybe I’ll give ol’ crooked-hat a chance this year.
  9. Tyler Clippard. Heartburn Index: 5.22
  10. Ross Detwiler. Heartburn Index: 4.99 Of the pitchers competing for the fifth slot in the starting rotation, Detwiler has the highest heartburn index.
  11. Edwin Jackson. Heartburn Index: 4.84
  12. John Lannan. Heartburn Index: 4.76
  13. Chien-Ming Wang. Heartburn Index: 4.38
  14. Drew Storen. Heartburn Index: 4.34. Storen doesn’t give up cookies all that often.
  15. Craig Stammen. Heartburn Index: 4.09 Now this is a surprise. It looks like Stammen’s only major flaw is a propensity to give up way too many home runs. Otherwise, his heartburn index is astonishingly low.
  16. Jordan Zimmermann. Heartburn Index: 3.71 JZ is perhaps the most workmanlike pitcher on the staff. He doesn’t make many mistakes.
  17. Stephen Strasburg. Heartburn Index: 2.54. In this, as in so many other categories, Strasburg is simply incomparable.

There you have it, folks. Again, don’t treat this as a “real” stat–think of it more as the mind confirming what the eyes see. There are some paradoxes here, though. Hot Rod, for instance, tops the heartburn charts, but has the second-lowest FIP (3.22) on the staff–a testament to his strikeout prowess.