Which WAR Works Best? (Pitching Version)

Wins Above Replacement (WAR) has become one of the most popular sabermetric stats, because it capsulizes a player's value in one number. Fangraphs.com and Baseball Reference.com are the two best baseball statistics web sites; but they provide different WAR stats. This is because each site uses different methodologies to calculate WAR. I will look at the Pitching WAR statistic.

I frequently read articles and comments on the world wide web rejecting WAR because B-Ref and Fangraphs produce different values for WAR. Apparently some people expect stats to be a fixed calculation like batting average, and reject statistics that aren't. This criticism reflects a misunderstanding of WAR. WAR is a framework for comparing players' value on the field. As with any valuation, it is an estimate. And people can utilize different WAR components to measure the facets of on field performance like, say, base running. Sabermetrics doesn't have its own Pope or Czar to tell us the correct doctrine; nor should it.

Now let's summarize the fundamental difference between f-WAR and b-WAR for pitching. Both methods are attempting to isolate the pitcher's contributions to wins, apart from the other factors , principally team defense, Baseball-Ref starts out with actual Runs Allowed and then attempts to adjust for differences in team defense, as measured by total zone or UZR, advanced pitching metrics. Fangraphs utilizes a pitching statistic which ignores actual runs allowed, and therefore is independent of team defense. However, arguably, f-WAR's use of Fielding Independent Pitching (FIP) has an advantage in taking out "luck," because FIP is unaffected by BABIP. The advantage for b-WAR is that it tends to correlate better with a team's actual wins during the season because it is based on actual runs allowed.

The preference for using b-WAR or f-WAR may depend on how you use it. If you are interested in how well the pitcher performed in the past year, it's not clear which is the best method. The answer to this question may depend on how much confidence you have in advanced pitching metrics, which are part of the b-WAR pitching calculation. (If you are satisified with this part of b-WAR's calculation, maybe b-WAR works best for this purpose, since it reflects actual runs allowed.) However, if you want to use WAR to suggest how well the pitcher will perform in the future (i.e., something closer to true pitching talent), this recent article at Hardball Times concludes that fangraphs' WAR may be the best choice.

The HT article compared the statistical performance of f-WAR, b-WAR, and BB-K stats in explaining pitchers' next year performance. On an individual basis, f-WAR had higher explanatory power than either BB-K or b-WAR. If b-WAR is combined with BB-K in a multiple regression analysis, the explanatory power of b-WAR disappeared, meaning that you would be better off just using simple BBs and Ks over b-WAR to estimate future pitching performance. This is not the case for f-WAR. So, f-WAR is the preferred WAR statistic if you want to use it to estimate future pitching WAR. As an example, if you want to know how much salary a pitcher should be worth in free agency, you ought to be more interested in expected future performance, rather than past performance.

Next let's look at f-WAR and b-WAR for the Astros' pitching.

Astros Pitching WAR

B-Ref 1.3

Fangraphs 4.1

As you can see, fWAR gives the Astros more wins above replacement for pitching than bWAR. Is that a good sign for next season? Maybe--though 4.1 wins at the three-quarters mark of the season isn't exactly impressive.

Let's look at the differences in WAR for Astros' starting pitchers.

WAR

bWAR / fWAR

Happ 0.4 / 0.6

Harrell 2.0 / 2.4

Keuchel -0.3 / -0.4

Lyles -1.5 / -0.3

Norris 0.0 / 1.0

Wandy 1.2 / 1.8

With the exception of Keuchel, all of the Astros' starters have a higher WAR using fangraphs. Bud Norris shows the biggest difference. B-Ref shows Norris as replacement level, with zero WAR. But fangraphs shows Norris as 1 win above replacement. Jordan Lyles also is more than 1 win worse using bRef instead of fWAR.

As I've mentioned in previous articles, SIERA is a more complex advanced pitching statistic which is better than either ERA or FIP in predicting next year's ERA. The Astros with a 4.09 SIERA, are ranked 23d among ML teams, compared to 26th for ERA and FIP. It would be interesting if Fangraphs developed and published an alternate WAR using SIERA instead of FIP. This might provide a better measure of the pitcher WAR based on the player's true talent or skill.

Don't take this as a knock on B-Ref's WAR; I'm not saying that you should never use it. In fact, bWAR recently underwent some sophisticated revisions which improve the details of the methodology. But the choice of fWAR or bWAR may depend on your intended purpose.