Sabermetrics: Astros and Expectations

A recent Yahoo article quipped:  "Finding a pundit who picked the Astros to win the AL West this season is like finding someone surprised that Ruben Amaro got fired."

And most of us have to admit that we didn't expect the Astros to be contending for the pennant this year.  We knew the Astros would be improved, but expected the contending seasons to occur in 2016 or 2017.  Personally, I expected the Astros to be a .500 ballclub, which would be a promising season. However, I was also aware that a .500 talent team could become a 86 or 87 win team with a bit of luck.

But here we are, nearly mid-September.  And the Astros have a 80% probability of winning the AL West, according to Fangraphs' playoff odds.  The Astros have a 94% chance of making the playoffs and a 89% chance of getting to the AL division series.

Why didn't we see this coming?  Sure, a lot of people just looked at the Astros' terrible win-loss percentages over the past few years and reached their conclusions from the immediate past--not the best approach to predictions.  But even the more sophisticated projection systems didn't predict it as likely.

A good starting point for reviewing this question is a recent Beyond the Boxscore article, "Cluster and Performance Luck 2015: An Update."  The complete article is well worth a read.  But let's focus on the Astros.  With respect to "cluster luck," or random variation based on sequencing, the Astros have experienced bad luck and should have about 8 more wins.  Another type of luck---and the author freely admits that "luck" isn't a good descriptive term---is the extent that the individual players outperformed preseason projections.

Based on individual player projections, the Astros have outperformed their preseason WAR expectations by 7.6 wins, baseball's largest deviation from performance projections.  Close behind, the Blue Jays and Royals have outperformed WAR expectations by 7 or more wins.  The Astros actually slightly underperformed offensive projections. That means the better than expected performance has mostly come from the pitching side.  As the article points out:

it's been their pitching staff that has blown away expectations, led by Lance McCullers (-1.0 projected, 2.3 actual), Dallas Keuchel (2.5 projected, 5.4 actual), and Vincent Velasquez (-0.1 projected, 1.1 actual). Just as important, they've seen almost no underperformance ... Of the 23 pitchers throwing any number of innings for the Astros, only three have underperformed preseason projections. If there's any one reason Houston's rebuilding timeline has seemingly leapt a year ahead of schedule, it's been the surprisingly excellent performance of their pitching staff.

Thus, we can focus on the pitching projections and actual 2015 performance.  Since the article specifically mentions McCullers and Velasquez, I will venture the guess that projecting rookie pitchers is less reliable than projections of veteran pitchers.  Rookie pitchers only have minor league performance as the basis for future projections, which probably is less reliable than using past major league performance to project future major league performance.

To test this point, I compared ZIPS preseason ERA projections for all qualified pitchers and all  rookie pitchers with actual ERA so far in 2015. I utilized ERA instead of WAR, in order to exclude errors in projecting innings pitched Sure enough, my assumption seems to be correct.   The average deviation of rookies' actual performance from projecton is -0.3 runs of ERA Note that the negative value means that projected ERA generally was higher than actual ERA.  The average deviation of all pitchers' actual performance from projection is -0.05 runs of ERA, thereby indicating that projected ERA for all pitchers is closer to the actual results than the average for rookie pitchers.  The standard deviation for the projection variances in rookie pitcher ERA is 1.35 runs of ERA vs. a standard deviation of 0.72 runs of ERA for all pitchers.  The comparison in standard deviations shows that variation in projection error is wider for rookie pitchers than the overall population of pitchers.

McCullers' performance is better than his projected ERA by the largest margin of any rookie pitcher (3 runs of ERA lower than expected).  This is more than 2 standard deviations from average rookie projection variance.  This is another way of saying that his performance exceeds the margin of error at a 95% confidence level.  Velasquez's actual ERA is 1.3 runs below his projected ERA, which is right at the edge of 1 standard deviation.  Not as striking as McCullers, but still impressive.  On the batting side, rookie Carlos Correa has outperformed his projections significantly, but as noted in the BtB article, the margin isn't among the highest projection variances among batters in the majors.  Besides the usual problem of translating minor league performance into major league projections, it's worth noting that the ZIPS projection for McCullers and Velasquez is based on two pitchers with experience no higher than A+ who subsequently had break outs in AA this year.

The variance between actual and projected ERA for other Astros' starting pitchers is shown below.  Again, note that a negative value means that actual ERA is less than the projected ERA.

Dallas Keuchel has bettered his projected ERA by more than any other qualified starting pitcher, except the Cubs' Arrieta, who has a -1.6 variance.  Keuchel's variance from his projected ERA is more than 2 standard deviations from the average variance for starting pitcher projections, thus exceeding the 95% interval.   McHugh is the only starting pitcher in this table whose projection is lower than his actual ERA--and the variation is relatively small.  Feldman, Fiers, and Kazmir all have pitched better than their projections.  Of course, Fiers and Kazmir would not have been included in the preseason projections for the Astros, but their addition to the team improves the Astros' current W-L projections. Keuchel struggled early in his ML career, and he was not highly rated as a prospect, so one might understand why the ZIPS projection system didn't fully believe his fine performance last year.  But for many of us who follow the Astros more closely, Keuchel's continued improvement is not a surprise.

The conclusions to this analysis are more questions than answers.  First, it's probably fair to surmise that preseason projections will have a higher error rate for teams which have a large inventory of prospects to promote into the big leagues.  Second, do the performances of Velasquez and McCullers tell us something about the Astros' pitcher development process in the minor leagues?  Given that both young pitchers are markedly better than their projections, is this just the luck of the draw, or is the Astros' development process special?  Third, do the Astros' pitchers beat the projections because the Astros organization takes into account information or variables which are not recognized in the ZIPS model?   Maybe measures of batted ball chacteristics like "soft contact" are considered by the Astros in developing/acquiring pitchers, but are not incorporated in projection models.  Fourth, is this another indication of the "Strom Effect?"  I'll leave these questions for you to address.

A SIDE NOTE

Finally, just to avoid confusion, let me distinguish projections based on "cluster luck" from individual player projections, like ZIPS.  The Astros "unluckiness" in cluster luck, as discussed in the BtB article, is based on BaseRuns, a metric which takes into account the expected sequence of events, given a set of team characteristics, which lead to run scoring.  For example, if a HR, HBP, and single occur in an inning, the run scoring effect could be 1 run or 3 runs, depending on the sequence of these events.  There is little evidence that teams or individual players can exercise significant control over the sequence of those events.  Therefore, projecting team records using BaseRuns recognizes that the actual team runs will depend on the expected sequence of events.  Just to add more confusion, BaseRuns is not the same thing as Pythagorean Record.  The Pythagorean expected W-L record is based on total runs scored and runs allowed over the season, and does not address the components which cause run scoring.   Individual projections like ZIPS derive context neutral stats for players.  Although they may take into account team-specific factors like park effects, they do not arrive at win-loss projections based on the overall team characteristics, like BaseRuns.   This may be one of the reasons that ZIPS contains a disclaimer cautioning users against summing projected WAR to project team records.