The prospects we know and the prospects we don't

I like books.
What? You know that already? Okay.
Anyway, I do read e-books, but there are some books I still crave, and one of them is Baseball America’s annual Prospect Handbook. Which just arrived at my house last week, joining the dozen or so older editions.
The back cover of the 2015 Handbook features this headline:
WHY PROSPECTS MATTER
... followed by this text:
In the 2011 Prospect Handbook, we detailed the depth of the Royals’ top-ranked farm system, which we also featured on the cover of the March 2011 issue of Baseball America magazine. No team had ever placed nine players in Baseball America’s Top 100 prospects before, and group – both as big leaguers and through trades – helped form the core of the Royals’ 2014 American League pennant winners.
Turning a losing franchise into a winner – that’s why prospects matter.
Here are those nine guys who made the Top 100 list: Eric Hosmer, Wil Myers, Mike Moustakas, John Lamb, Mike Montgomery, Christian Colon, Danny Duffy, Chris Dwyer, Aaron Crow.
I will pardon you for being underwhelmed.
Here were the most valuable players on the Royals’ 2014 American League pennant winners, according to bWAR: Alex Gordon, Lorenzo Cain, Salvador Perez, Jarrod Dyson, Wade Davis, Danny Duffy, James Shields, Yordano Ventura, Kelvin Herrera.
Did you notice those two lists have exactly one name in common? Yup, Danny Duffy is the sole survivor.
Well, of course we have to mention Shields and Davis, for whom the Royals traded Myers and Montgomery (along with Jake Odorizzi and Patrick Leonard, who weren’t in their system in the spring of 2011) two years ago. So that’s three prospects from that 2011 list who played – “as big leaguers and through trades” – big roles in 2014, since Hosmer and (especially) Moustakas both struggled during the season.
And the other four Royals on the 2011 Top 100 list?
Crow pitched decently, though in a relatively low-leverage role, especially late in the season. Colon spent most of the season in Triple-A. John Lamb and Dwyer pitched in Triple-A, and neither got an inning with the big club.
Does this mean PROSPECTS DON’T MATTER? Or that PROSPECTS DON’T MATTER AS MUCH AS WE THINK?
Not necessarily. Here’s a list of nine guys Baseball America COULD have put on that cover in March, 2011:
1. Salvador Perez
2. Greg Holland
3. Yordano Ventura
4. Eric Hosmer
5. Kelvin Herrera
6. Wil Myers
7. Jarrod Dyson
8. Mike Moustakas
9. Aaron Crow
The Royals’ system was ABSOLUTELY LOADED four years ago. Among the prospects who did not make Baseball America’s Top 100 list in 2011 were Ventura (No. 12 on list of Royals’ prospects), Perez (17!), Jarrod Dyson (20), and Kelvin Herrera (30) … along with future superstar closer Greg Holland, who wasn’t then ranked among the Royals’ 30 best prospects.
Think about Greg Holland for just a few more seconds.
My point isn’t that prospects don’t matter, or that the Royals’ crop four years wasn’t tremendous and didn’t play a huge role last season.
My point is that it’s incredibly difficult to know which prospects will succeed and which will not. Four years ago, ranking the Royals’ farm system as tops in the majors now seems prescient. That magazine cover, though, with those nine guys? Not so much.
I hope you’ve not interpreted any of the above as a sort of criticism. It’s not that, at all. I do believe that we often fool ourselves into thinking this stuff’s easier than it is, and I do wonder if we – and by we, I mean the people who do this work for a living, publicly – are getting any better at it. Will this year’s prospect lists be more predictive than the prospect lists of five or 10 years ago? Probably, because we’ve got more information than ever before. But just marginally more predictive, I suspect.
What’s more, it seems to me there’s no great imperative for higher accuracy. If (for example) Baseball America is 10 percent more accurate, will they sell 10 percent more books? No, because (for one thing) as things stand now, nobody would even know they were 10 percent more accurate. It seems to me that there’s long been a sort of gentleman’s agreement, both in the private and public worlds of scouting, that nobody’s work will actually be checked. I’ve pointed this out before, and been ripped for it. Which is fine.
But checking someone’s work is the ultimate sign of respect. When you stop checking mine, I should start looking for something else to do.