Volatility and bullpen stats go together like horse and carriage. You can't have one without the other.
We've talked about improving the Astros' bullpen a lot lately. And volatility is a word that comes up. Really the point is that relievers pitch in such small sample sizes that year-to-year performance volatility is not unexpected.
Last month, Tango's blog pointed out an interesting pitching formula which is useful for estimating ERA in small sample sizes. The metric is called "True Independent Pitching Skill" (TIPS) and was developed by Chris Carruthers at the Breaking Blue blog devoted to the Toronto Blue Jays. The TIPS formula was not purposefully developed as a small sample size ERA estimator. Carruther's objective was to develop a fielding independent pitching statistic which is truly independent of all defensive components, including catcher pitch framing. Therefore, strike outs and walks, which are part of the FIP formula, are not included, since they are affected by catcher pitch framing. (TIPS might be a good metric to evaluate, say, a possible trade for David Price, by excluding the effect of the Rays' defense and pitch framing.)
But, for our purposes, Carruthers' more interesting finding is that TIPS is superior to ERA, FIP,, x-FIP, and SIERA in estimating future ERA for small sample sizes (less than 100 innings pitched). TIPS appears to be comparable to those metrics in somewhat larger samples, but probably is inferior to the FIP/SIERA type advanced stats when the sample size is 150 innings or greater. Since most relief pitchers compile small sample sizes of less than 80 innings per year, TIPS could have value as method for evaluating relievers' performance.
Why is TIPS a better estimator in small samples? The normal metrics are based on per inning or per plate appearance. But TIPS incorporates pitch f/x data at the "per pitch" level. The "per pitch" components comprise a larger sample size, because each inning is composed of a larger number of pitches. Thus, the pitch f/x variable is likely to stabilize more quickly. In fact, many of the pitch f/x components are reasonable predictors of strike outs and walks, but the data is more granular than strike outs and walks per inning, resulting in a more effective estimator of ERA for small inning samples.
I will use TIPS to evaluate the Astros' bullpen and small sample size starters, as well as potential free agent relief pitchers. But, first, I will discuss the components of TIPS.
Components of TIPS
Carruthers tried a number of pitch f/x measures, and he explains why he settled on swing strike% and outside swing% as the primary components. Carruthers also developed foul contact percent as the third component. Foul contact is based on the percentage of total contact which ends in a foul ball. Foul ball data is not readily available, but instead has to be calculated from batted ball data and contact percentage data. Fortunately, Carruthers made those calculations for us. The TIPS formula is shown below.
TIPS = 6.5*O-Looking(PitchF/x)% – 9.75*SwStr% – 4.8*Foul% + C
The constant percentage is used to scale the data to annual ERA averages. The constant used for 2013 is 2.57. Also, outside-looking% is used instead of outside-swing% because Carruthers preferred a positive independent variable to a negative variable. Outside-looking% is the exact inverse of outside swing%; therefore, the formula is indirectly using the outside-swing% variable.
The foul contact component of this formula may be the most interesting aspect. Foul balls are correlated with a lower ERA. This isn't surprising, given that foul balls are strikes--which is better than a ball or a hit. But, in reviewing TIPS, I learned something about pitchers' foul balls. Carruthers referenced an older article by Pizza Cutter (aka Russell Carleton at Baseball Prospectus) which researched foul balls. (Note a Mike Fast sighting in the comments of that article.)
First and most importantly, a pitcher's ability to induce foul balls is a repeatable skill. Second, there appears to be a distinction between the pitch-to-contact pitcher who induces groundballs and the pitcher whose contact is skewed toward foul balls---they don't profile as the same pitchers. Third, the foul contact pitcher is more likely to induce fly balls, but also suppresses singles, doubles, and triples. So the trade off is a few more home runs for a lower rate of all the other hits; and that is generally a positive trade off.
I really like this observation by Pizza Cutter regarding foul contact pitchers:
If a pitcher has a repetoire of "stuff" that no one can touch, that’s great! It’s hard to hit a home run if you can’t get the bat on the ball. However, if a pitcher doesn’t have world class gas, it’s OK if he has tricky stuff. It might be one of those abilities that hide in the data that no one really pays attention to. Consider, a foul ball means that the batter is thinking "hey, I can hit that!" and so he swings. He aims his bat where he thinks the ball is going, but apparently, he’s a little off and he fouls it off. The pitcher has tricked him! Perhaps foul contact is a decent proxy for how tricky a pitcher’s "stuff" really is. If a pitcher can trick a batter over and over, it means that he’s doing something right.
Astros Pitchers in 2013
Now we can get to the stuff you were waiting for---how do the Astros' small sample size pitchers fare with TIPS. Carruthers' article provides a good look up procedure for 2013 TIPS. I will start with the Astros small sample size pitchers (all less than 80 IP, but more than 30 IP), including the recent additions to the roster. Some of the late year relievers are below 30 IP and are not shown.
Small Sample Pitchers
|TIPS ERA Estimate|
As you can see, the potential relief pitcher acquired by the Astros this off-season--Downs, Qualls, Valdes, and Bass--profile as among the best in the bullpen. Downs and Valdes were practically free (claimed off waivers) and TIPS shows that they may be the best bargains. If you scratched your head over the Downs and Valdes waiver claims, maybe this explains something. Bass' 5+ ERA may hide a true 3.94 skill level last year.
I find it interesting that Clemens' TIPS is so much lower than both his ERA (5.40) and FIP (5.69). My observation of Clemens' pitching is that he looked to be pitching well throughout most of an at bat, but that poor sequencing seemed to produce an ultimate bad outcome. It's possible that Brent Strom, who talks about pitch sequencing and tunneling, could get some improvement out of Paul Clemens. Hypothetically, pitch sequencing may be an issue for a pitcher whose swinging strike%, outside contact %, and foul contact % don't line up with the expected strike out results.
Some of the small sample pitchers (Oberholtzer, Bass, Clemens, Peacock, and to some extent, Clemens) were starting pitchers; and it's unfair to compare their TIPS to relievers, since we expect starters to have a higher ERA. Below we show the TIPS for the Astros' starting pitcher candidates, including those with larger samples.
Notably, TIPS shows Brett Oberholtzer as the most effective starting pitcher candidate. This is consistent with my general view that Obie's good rookie season was not a mirage. I placed Bass at the end because it's unclear if he is a starter or reliever; but if he is in the rotation, his TIPS is second only to Oberholtzer. Feldman's ERA will regress, according to TIPS. But he also has the largest sample size---and as noted above, TIPS is inferior to other advanced pitching metrics when the sample size exceeds 150 IP. We can hope that Feldman's results will be closer to his FIP and SIERA results than TIPS. As you can see, Peacock was much better, according to TIPS, than his ERA (5.18) or FIP (4.98). . Peacock's 4.18 TIPS is closer to his 4.23 SIERA, which suggests a significant potential to become a solid starter.
TIPS indicates a significant possibility of regression for Cosart---in line with his FIP and x-FIP. Harrell is much better, based on TIPS, than any of the advanced metrics or his ERA would indicate. Again, maybe the coaching of Strom can bring Harrell's results more in line with his TIPS components.
Next, I list some of the available free agent relievers, along with their TIPS, below.
A disappointing result on this list is the significant regression indicated for Veras' ERA (4.26 TIPS vs.3.02 ERA). Since the Astros are actively negotiating with Veras, we can hope that the regression is more in line with his SIERA (3.37). Crain and Thornton, whom are also rumored as Astros' targets, appear to be better performers, based on TIPS. We've talked about Oswalt at TCB, and his TIPS indicate that he was much better than his ERA. I know some of you will hate this---but I wouldn't mind trying Marmol. And he also has a pretty decent TIPS.
TIPS is also discussed in Fangraphs' community blog.
Do you have any reactions to the TIPS ranking of Astros' pitchers or free agent relievers?