We have no idea how the Astros or any other club in baseball evaluates the defensive ability and performance of players. As wannabe baseball analysts, we only have the tools publicly provided by our intellectual forebears. Despite the inherent level of uncertainty associated with defensive metrics like UZR, DRS, etc, the information available is still useful for creating a general picture of a player's effectiveness at certain things.
UZR, an acronym for the apocalyptically-named Ultimate Zone Rating, is one of the more commonly-used defensive measurements. Fangraphs, the Chilton manual for the engine of baseball, explains UZR in depth and provides links for further reading, if you so desire.
One of the major components of UZR is RngR, or Range Runs. Like all components of UZR, RngR is expressed in the number of Runs a player is worth based on his defensive range, compared to an average performer at his position. The data for RngR is collected manually by Baseball Information Solutions, and so is subject to the whims of imperfect human observation. It also discounts extreme positional shifts, a fact on which I will elaborate later. Regardless, it is still very good data, and can be confidently used to examine performance on a general level.
Here's an example of how range components of defensive metrics work. Pretend Michael Bourn and Bengie Molina are both shortstops. An average shortstop's range is a circle with a radius of about fifteen feet from whatever point he's standing in (I made that number up...this is just an example). Bourn, being a swift dude with godlike reflexes, has a range with a twenty foot radius. Molina, as the slowest player in the history of ever, has a range with a five foot radius. All sorts of math happens, and we learn that Bourn is worth two runs per 150 innings more than the average shortstop and Molina is worth minus four runs compared to that same fictitious average dude. Got it?
(If math isn't your thing, skip ahead starting here)
Anyway, I dumped all of Fangraph's defensive metrics for the past five seasons (up to 2012) into Excelibur for all players with at least 150 innings played. I normalized the data to RngR/150, or Range Runs per 150 innings played, then found the average, variance, and standard deviation for each defensive position. If you're not interested in gritty details, please ignore this next chart.
For our purposes, those numbers are only useful because they give us something to compare to the 2013 Astros. But before I move on, I want to point something out from this chart. The variances of the positions reflect what common sense should tell us. First base is typically manned by bigger, slower power hitters, and the defender must stay relatively close to the bag in most situations. Therefore, the relative variance among the sample of all MLB 1B's is relatively small. Likewise, the variance is very large for center field, where players' speed and reaction times have the most effect on whether or not they can reach batted balls over the largest coverage area on the field. So player range scores in center vary more than those at 1B.
Since RngR is already expressed in terms that compare to a hypothetical replacement-level defender, it seems odd to again find the average of RngR scores at each position. But I prefer to compare to actual players, and so I averaged the comparison scores. That allows me to use standard deviations to figure how much better or worse the 2013 Astros have been in terms of RngR/150 than the average defender at their position for the past five years.
(If you don't care about math, this is where you can start reading again.)
To make this more digestable, I've expressed this as a Percentile. So if a player's percentile is 75%, then he scored better in RngR/150 than 75% of players at his position over the past five seasons. Without further ado...
If nothing else, this explains why the Astros might be insisting on converting Jimmy Paredes into an outfielder. While by no means insinuating that he is, you know, good at defense, this information does show that compared to his peers, Paredes has an excellent Range component to his defense. Whether he gets good breaks, catches balls that are thrown to him, or has an arm good enough for an outfield spot are irrelevant to this data, as is his ability to not run into his teammates and injure them.
MoreTalking Sabermetrics from TCB:
Springer's K Rate Not the Whole Story Astros Hitters' Luck Astros' Cosart Proving Critics Wrong? Or Right?
Overall, the Astros have rated quite low compared to their peers in terms of Range scores. This matches what our eyeballs see on the field - that Marwin, Barnes, and Maxwell do a decent job getting to balls hit in their vicinity, and that Carter, Altuve, and Wallace do not.
The most disappointing scores on this list are Matt Dominguez and Robbie Grossman, two players touted for their defensive ability. It's worth pointing out though that neither possess the type of burning foot speed that can increase their Range (for what it's worth, Bourn has one of the highest Range/150 scores for the past five seasons, as does Maxwell).
NOT SO FAST!
Another popular defensive metric, DRS (Defensive Runs Saved) is based on the +/- system of valuating a player's leatherwork. Like URZ, its components are expressed in runs above or below an average positional player. And like UZR, it contains a component that includes range, called rPM (Plus/Minus Runs Saved). Unlike RngR, rPM does a better job of accounting for extreme defensive shifts. There are some bugaboos here, because Baseball Info Solutions only started recording shift information in 2010, but it's now generally accepted that DRS does a better job accounting for them than UZR.
I duplicated my process to evaluate the positional average, variance, and standard deviation of RPM from 2008 to 2012 (snore, I know...).
I applied that to the 2013 Astros, giving me the following chart, which again, takes defensive shifting more into account:
Now it gets interesting. While Maxwell got loved on by UZR, DRS shows a different story. Clack gives a really good explanation of how defensive shifting can effect these two measuring systems. The data here suggests that while Maxwell's range looked excellent when starting from the traditional center field position, his range suffered in all of those off-position plays that UZR disregards. This could be due to small samples of in-position plays in 2013, or a general discomfort when playing out of a traditional spot. Most likely it's a sample size issue, because Maxwell's career data shows excellent numbers. In fact, by rPM/150, Maxwell ranks 19th of 1,140 players over the 2008 to 2012 time span, behind former Astro Mark Loretta (5th, at 2B) and *ahem* Angel Sanchez (17th, also when playing 2B). For those whose brains just exploded, Sanchez ranks 1,111 as a shortstop.
Happily, this has the opposite effect on Matt Dominguez and Robbie Grossman, neither of whom are regarded as bad defenders. In this case, DRS and rPM match a little more closely with what our eyeballs tell us. Both systems agree on Paredes, too--all other factors aside, he has average-to-good range in right field.
A lot of headway has been made on improving defensive metrics over the last few years, but they still lack a completeness that, in the minds of some people, calls their credibility into question. For myself, I choose to recognize what these metrics are and what they aren't. They certainly aren't conclusive. But data is rarely bad, as long as it can be put into the proper context. In this case, range components of advanced defensive metrics give us a basis for comparison against other players at that position that is useful to validate or contradict the eyeball test, which inherently suffers from sample size problems and preconceived bias.
What do y'all think? Do either of these systems agree with what you see on the field? I can't watch the games, so I'll have to take your word for it.