Sabermetrics: Making Sense of the Astros' Team Defense

If we rely on the advanced fielding metrics--namely Ultimate Zone Rating (UZR) and the Fielding Bible's Defensive Runs Saved (DRS)--it's hard to pin down the quality of the Astros' defense in 2014.  First, UZR and DRS don't agree on the ranking of the Astros' team defense.  Second, the Astros' proclivity for shifting infield positions tends to distort the results of the individual fielding results.

Let's start off with this factoid from the twitterverse: "The 2014 Astros have the 25th worst defense in the live ball era. http://t.co/h8yPzxiBW8"

What?  The 2014 Astros' defense was historically bad?

Not so fast.  Ignore the fact that play-by-play fielding data is available for only a small fraction of the live ball era.  DRS finds that nine teams had worse defense than the Astros in 2014.  The fWAR defensive component is based on UZR.  In the case of the 2014 Astros, the gap between UZR and DRS doesn't make sense.    I'm not buying that the Astros' fielding is "historically bad" when the other publicly available advanced fielding metric, using the same basic data, finds that one third of 2014 teams are worse defensive squads.

Both DRS and UZR utilize basic data supplied by Baseball Information Systems (BIS).  However, DRS and UZR employ different calculations and adjustments of the data.  This 2009 discussion describes some of these differences from the perspective of DRS developers.  You can read a more recent primer on UZR, DRS, and their differences in this 2013 article at Athletics Nation. Interestingly, DRS now employs a "hang time" timer, but it's unclear if DRS has adopted the same or similar measurement tool.

We don't know why the basic UZR and DRS results differ so much for the Astros.  However, given that the Astros utilize more frequent defensive shifts and positioning adjustments than any other team, the differences in the respective metrics' treatment of out of zone (OOZ) plays is an obvious candidate for suspicion. Out of zone plays refer to a fielder's play on a ball outside of the normal zone associated with the position. To the best of my knowledge, UZR excludes OOZ plays.  DRS historically included OOZ plays, but the growing trend in shifts resulted in alarming high deviations in some infielders' fielding results.  As a result, in late 2012, DRS excluded certain defined extreme shifts from individual player results and calculated separate runs saved for those shift plays on a team basis. My May 2013 article on defensive shifts describes this in more detail.  Early in the season, I updated information regarding the Astros' use of shifts.

My conclusion is that DRS' player results account for some out of zone plays and defensive positioning adjustments which fall outside the DRS definition of extreme shifts.

How many more OOZ plays were made by the Astros?  A lot.  The Astros' 643 OOZ plays is 34% higher than the major league average number of OOZ plays, and almost 100 plays more than the next highest OOZ (Yankees).  Although defensive shifts account for a large portion of this difference, the OOZ plays also include "exceptional" rangy fielder plays, where the fielder ranges into an adjacent fielder's zone.  We really don't know how many of the extra OOZ plays are included in the DRS player results vs. the DRS "shift runs saved" team calculation.

However, the 2014 DRS team impact discussed so far, and the resulting Astros' No. 21 ranking, does not include the runs saved through extreme shifts.  Because Bill James on-line (subscription required for access to stats) shows the separate calculations for team defensive runs saved due to shifts, we can obtain a better measure of the total team defense by adding the shift numbers.

TEAM DEFENSE INCLUDING SHIFTS

The method used by DRS to calculate shift runs saved is discussed in this early 2014 New York Post article. By a sizeable margin, the Astros saved more runs via the shift than any other team.  The Astros' 27 runs saved via the shift means that the shifts produced approximately 3 wins of value for the Astros--wins which are not included in either Fangraphs or Baseball-Reference WAR.  The Astros' defensive runs saved in the shift is about 70% higher than the next highest team (Cubs).  Bill James on-line does not include these savings in the team sums; so the next step is to add shift DRS into the team DRS sum.

Here is the ranking of teams based on total defensive runs saved, including shift net savings.

DRS INCLUDING SHIFTS

As shown above, the Astros rank 14th in team defense if shifts are included.  Even more notable, the Astros are now 12 runs above average, which is quite a contrast to -15 for standard DRS and -64 for UZR.   The difference between UZR and the 2014 Astros' result, above, is approximately 11 wins.

Standard DRS has a 0.11 correlation with OOZ plays, and UZR has a 0.04 correlation.  By adding shift runs saved  to DRS, shown in the rankings above, the correlation between DRS and OOZ plays jumps to 0.81.

We can use the Astros' OOZ plays to perform a sanity check on the Fielding Bible's DRS shift calculation.  The Astros' fielders made 157 OOZ plays above average.  Although I don't know the above average increment for outfield vs. infield, my calculation indicates approximately one half the Astros' OOZ plays were made on the infield.  Conservatively assuming that half the Astros OOZ above average plays are on the infield, the result is 79 OOZ. As a rough cut, we can assume a run expectancy of 0.5 per OOZ play.   This results in 39 runs saved by the Astros due to out of zone plays.

The OOZ calculation for the Astros indicates a savings of approximately 1 win more than the shift DRS value.  There could be many explanation for that difference, including the liklihood that some OOZ plays are included in the standard DRS value, instead of the shift DRS.

Individual Astros' Out of Zone Plays

The highest OOZ totals among Astros' position players:

OOZ Plays, Total Plays

Altuve  72, 299

Villar 69, 142

Gonzalez 52, 126

Fowler 51, 188

Springer 48, 108

Marisnick 46, 71

Dominguez 39, 221

Some quick observations on this leaderboard:

• If shift DRS is distributed in some fashion across the 3b, SS, and 2b positions, the Astros are probably average to above average at each of those positions.
• Maybe the Astros' in-house options at shortstop are better than we think.  Standard DRS already shows a +6 for the Astros' shortstop position, and giving credit to the position for shift runs saved probably gives the Astros one of the better shortstop  defensive combos (Villar and Gonzalez) in baseball.
• It's hard to evaluate the effect of Fowler's OOZ plays on his defensive value.  Center field is a position which normally produces substantial OOZ and his "rate" of OOZ is not outstanding.
• Springer and Marisnick: Wow!