Tango (at The Book blog) takes this question on with a rather ingenious analysis. The "with and without" approach to analysis is widely used in many disciplines. The idea is to ask, how many runs did the Astros allow when Adam Everett is in the lineup and when he isn't? But over the course of a season so many factors are at work, that we can't attribute all the differential in saved runs to Everett, alone.
Tango addresses this issue by examining enough great fielders over a long period that the sample size overcomes the other factors. An excerpt:
So, using WOWY (With Or Without You) based on balls in play, I selected the best twenty or so infielders (2B, SS, 3B) since 1993. It’s mostly the names you know: Everett, Sanchez, Bartlett, Rolen, Reese, Hudson, Inge, etc. By looking at a large enough number of great infielders, the idea is that all the noise around them will cancel out. My only additional constraint was that he must have been on the field for at least 1000 outs for a given team-season, and must have been off the field for at least 1000 outs for that same team-season.
I came up with 68 such seasons since 1993. The total number of games played was 5386 games on the field and 5400 games off the field. You have to admit that that’s alot of games. When the star fielders were on the field, their team allowed 4.60 runs per game, and when they weren’t on the field, they allowed 4.83 runs per game. Per 162 games, this difference comes out to 37 runs.
So his conclusion is that a great infielder like Adam Everett will be worth 37 runs, on average, defensively, per year.
He then performs the same analysis for outfielders and first basemen. The averages come out to 19 runs and 15 runs, for great outfielders and great first basemen, respectively.
Tango obtains a weighted average for the positions on the field, and concludes that a truly great defensive player on a team is worth on average 25 runs in a season.