Talking Sabermetrics: How Are Astros Players Stacking Up Against Their Projections?

The sabermetric tools which seem to get the most public attention are player projections. During each off-season, the developers of numerous projection models release their player predictions, in the form of projected statistics for the season. Sabermetric-inclined baseball teams produce their own proprietary projections. Although we don't know much about the projections produced by baseball teams, the multitude of publicly available projection system results lead to a variety of opinions regarding the relative accuracy of each model. Not surprisingly, the results of the models generally are quite similar.

Given that more than half the season is behind us, it's a good time to review how Astros' players' current performance compare to their projection. Before I make those comparisons, though, I will talk about projections a bit.

Projections are different than x-BABIP, x-wOBA, FIP, SIERA, and x-FIP, which we sometimes use to predict likely regression in a player's current performance. Those stats are based on short term performance, and are intended to "de-luck" the player's current stats. Projection models, on the other hand, rely upon regression analysis of a player's historic performance over multiple seasons to predict a future season performance. The projection methods may incorporate the player's minor league performance and use Major League Equivalents (MLEs) calculations. Some of the models include aging variables. Pitch f/x may become a rich source of data for projection models in the future, though a few models supposedly include some pitch f/x data like pitch velocity.

My first exposure to player projections was Bill James' famous prediction of Jeff Bagwell's rookie season. At a time when projection models were in their infancy in 1990, Bill James released his player projections as part of STATS Inc.'s Baseball Handbook. Jeff Bagwell, a Red Sox minor leaguer who was traded to the Astros in late 1990, became an unintended test case for statistical projections. Bagwell's projection was included in a list titled "These Guys Can Play and Might Get A Shot." Peter Gammons, a reporter covering the Red Sox, used the projection to write that James predicted that Bagwell, a relative unknown Double A hitter, would win the NL batting championship. James, in fact, had not made that prediction. The .314 batting average projection shown for Bagwell, however, was higher than James' other projections for NL hitters. The story spread rapidly and evoked debate in the baseball world. Some reporters ridiculed the idea that a computer can predict baseball player statistics.

This 1991 article in the Houston Chronicle archives says that James' prediction for Bagwell was the "most sensational prediction at last year's winter's meetings." .Bagwell didn't win the NL batting championship, but (fortunately for James and the sabermetric world) Bagwell's batting line of .294/.387/.437 was good enough to win the rookie of the year. Bagwell's surprisingly good performance seemed to validate James' projection methods. And, in retrospect, Bagwell's .824 OPS was only 12 points less than James projected.

On to the comparison of Astros players' 2012 performance to the projections... after the jump.

The Astros were a difficult team to project because of the relative youth and inexperience. Many of the players have only small samples of data at the major league level.

For my purposes of identifying projected results, I relied upon an average of the following six projection sources: ESPN, RotoChamp, CAiRO, ZIPS, Steamer, and Davenport. For some players, I can't use an average for these sources, and I utilize only ZIPS, which is indicated on the tables.

In the table below, I display the actual OPS (so far this season) compared to the projected OPS.

In the table below I display the actual ERA (so far this season) for Astros' pitchers, compared to the projected ERA.

The comparisons are self-explanatory. But here's some observations:

• I previously wrote that Brian Bogusevic would be the hardest Astros hitter to predict, based on the divergent results from projection systems. So far, it appears that the lowest projections will be more accurate. ZIPS seemed pessimistic on Bogey, but it may prove to be accurate.
• Lowrie and Castro have been particularly nice surprises. Both players have been plagued by previous injuries and some inconsistent seasons, which plays havoc with projection models.
• Martinez, Snyder, Downs, and the new guy, Francisco, were projected to be fairly good offensive players, but have disappointed so far. The lineup would be tougher if these hitters played at their projected level.
• Wandy apparently is so consistent that he is easy to project.
• Is Bud Norris the most disappointing pitcher?
• The bullpen seems to have a lot of guys who either out perform or under perform their projections by a sizeable margin. This substantiates the notion that relief pitchers are naturally high variance players.
• Any surprises?