clock menu more-arrow no yes mobile

Filed under:

Risky Business: The MLB Draft, Part 3

How can a major league baseball organization reconcile traditional subjective scouting methods with historical data about player performance?

Over the past couple weeks, I’ve discussed the likelihoods of certain types of draft picks in succeeding as major leaguers, based on a 15-year sample of Top 50 picks from 1993 to 2007. Here is a tidy summary of my findings:

Most likely to reach the majors: College hitters
Least likely to reach the majors: High School pitchers

Most likely to stay in the majors: College players
Least likely to stay in the majors: High Schoolers

Most likely to excel if they make the majors: High School Pitchers and College Hitters
Least likely to excel if they make the majors: College Pitchers

Total Odds: (Reach Majors and perform in 90th percentile):


As always though, there is more to the story. If one played the odds alone, one would never draft college pitchers. Obviously there are many reasons to do so, including that they seem to have a high chance of reaching their performance floor even if their ceiling is not as high as other draft pick types.

In discussing this topic with other writers, two of them sent me very relevant feedback. First, clack points out that comparing batter and pitcher WAR overlooks the fact that all WAR’s are not created equal.

"WAR provides a fixed percent of wins from pitchers (43%) and batters (57%). Since teams generally carry equal numbers of pitchers and batters (sometimes they carry one more pitcher, if they have a 13 man pitching staff), that means the WAR/pitcher roster spot will be less for pitchers. I would expect a very good pitcher to have a lower WAR than a very good hitter. The success of a pitcher draft selection is relative to the scale of WAR for pitchers, rather WAR for all players."

– clack

The chart below shows what he means. Of the Top 300 players last season by WAR, here are the top 112 pitcher WAR graphed against the top 112 batter WAR.


He makes a good point, and it’s one I debated over for a while before coming around to agree. At first I said, a win is a win is a win. It does not matter whether the win is being added by a batter or a pitcher. But Anthony sent me a separate email that put it all together in my mind. We were discussing the ongoing Appel vs. Bryant debate taking place in the Crawfish Boxes inbox, and he had this to say:

"For me, it's at least partially about how replaceable each part is. I feel like you can find Bryant - or 80% of Bryant, at the very least - in a lot of other places. Top of the rotation starters are extremely hard to come by. And yeah, you're right. They do break a lot, which makes them even more valuable. Whereas a college bat is a surer thing.. but I think that that "sureness" makes it even easier to replace, by just picking up the next college bat. Whereas the next potential ace that becomes draftable for you may not be for a year or two."

--Anthony Boyer

Setting aside whether I agree one can find 80% of Bryant in other places, Anthony’s point about the scarcity of ace pitchers is a valid one.

The next chart shows a breakdown of players in 2013 who reached a high WAR threshold:


What this data shows is a need to account for the scarcity of ace pitchers compared to elite hitters. This can be done in many different ways, depending on the data available and the weighting that clubs want to put on such factors.

Applying all this to the 2013 Draft

I started this exercise several weeks ago because I believe that the best way to achieve a good ranking of anything is to combine subjective and objective data to come up with a resulting list. This is the method I used for coming up with my own Top Prospect rankings during the offseason and I was very pleased with the results.

The question I asked myself when beginning this draft series is, if I were running a MLB draft, how would I go about creating my draft board? Here is how I answered it:

  1. Ask a group of experts to fill out a subjective ranking chart. This chart would include all of the "tools" and intangibles that one grades when scouting prospects, and each user would weight the different categories based on how important they consider them to be. Even better, the organization would determine the weighting. The resulting grades for each prospect would be averaged, which would give a numerical score that would represent subjective scouting information.
  2. Create multipliers based on historical data that take into consideration the odds of a player reaching the majors and then reaching a specific success level, major league positional scarcity, and major league WAR imbalance between positions. (Substitute other metrics if you want, WAR was my metric of choice.)

There are a zillion different combinations just using these two steps, and it would be up to each organization to decide what metrics to value, what category weighting is appropriate, and what factors to apply.

Subjective Evaluation of the 2013 Draft

With point number 1 in mind, I sent a spreadsheet to our Crawfish Boxes minor league writers and asked them to fill it out. The batter weighted categories were Average, Power, Baserunning, Fielding, Arm, Body, Makeup, Age, Bust Potential, Star Potential, and Signability. For pitchers, the categories were Fastball, Breaking Pitches, Stamina, Mechanics Repeatability, Mechanics Soundess, Body, Age, Bust Potential, Star Potential, and Signability. The writers assigned percentage weights to each category (totalling 100%), then gave each of 20 draft prospects grades from 1 to 10 in each category. Here are the results of the averaged scores:


I then applied the factors that I created based on the research I performed on the 15 year sample of draft data. I’ll save space by not going through the gory details of how I calculated these factors, but here they are:


Multiplying the subjective scores from the other writers with my factors resulted in the following list of draft prospects:



I caution against using the ranking above for any difinitive statements. It is based on the subjective whims of baseball bloggers, not professional scouts, and on multiplier calculations of one writer who is limited by the amount and type of data that he could collect. The point of the exercise is to discuss how historical context may be used in conjunction with traditional scouting methods to create a working board that can help drive decision-making on draft day. There are an unlimited number of different ways such a ranking can be created based on what an organization or individual considers important, and these could result in vastly different results. The key would be agreeing on weightings and what factors are important so that everybody is comfortable with the process. I tried to do this by not participating in the subjective portion of this experiment and by letting the multipliers be driven by historical data.

Here is a summary of what I learned during this process:

  • College players drafted high have historically reached the majors at a vastly higher rate than High Schoolers.
  • Of those players who reach the majors, High School pitchers and College batters had the highest chance of staying there and reaching elite levels of performance.
  • Overall, the fact that elite hitters are more common and produce higher WAR than elite pitchers must be considered when drafting players.
  • Multipliers based on historical data may be applied to subjective scouting data to create a cohesive draft board, and the results will depend on how an organization or individual weights different grading categories.