The Shortfalls of Using Statistical Projections

[Migrant Redbird]

Last winter, when I talked about the potential of Ryan Ludwick as a major leaguer, there was a tendency to dismiss him as an aging "quad A" player. After all, he was 29 years old and had been getting "cups of coffee" in the majors since 2002, with no significant success until he was promoted mid season to the Cardinals in 2007.

However, statistics can not take into account how much misfortune has limited a player's performance. When the Reds took a flyer on Josh Hamilton in the 2007 rule 5 draft, there was even less of a statistical data base upon which to make a projection. The Reds based their long shot gamble on the glowing scouting reports on Hamilton before his drug addictions drove him from the game for three full seasons.

Ludwick was also a highly touted prospect, albeit not nearly so glowing as Hamilton had been. Ludwick's problems were injuries, not drugs, but they were nearly as much of a handicap to overcome.

Ludwick is savoring his arrival as All-Star

As a brief refresher, Ludwick, 30 years old on Sunday, was hurt four years in a row from 2002-2005 and still has a six-inch rod in his left hip that sends airport scanning wands into a tizzy.

.... "I don't think a lot of people realize how hard it is to be hurt year in and year out and not be able to get it back," he said. "To try to play at this level is tough. I really do think about how crazy a road it's been and here I am at an All-Star Game."

.... Ludwick, who has 20 homers and 64 runs batted in, said he hadn't thought much about being an All-Star until just recently. "That's not one of the goals you set as a player," he said. "This is a little perk, a little bonus. But if you're asking me, 'Did I think I was capable of being at this (big-league) level?' Yes. I thought of I was capable of it when I was 22."

Rick Hummel or his editor ought to have double checked his statistics; Ludwick's solo home run against the Pirates on Sunday gave him 21 home runs and 65 RBIs at the break.

I use statistics more than most fans to support my evaluations of players because (1) I don't get that many opportunities to watch them play in person, and (2) I'm not that good of a scout when it comes to evaluating players anyhow. Statistics don't lie; however, they can frequently misinform when we try to make them do more than they're capable of doing.

However, I do get a little annoyed when fans who've never seen a player on the field point to his statistics and contemptuously dismiss his prospects as a major league player. Statistics can only inform our projections; they don't actually determine a player's future performance. Some fans tend to forget the difference when they express their emphatic opinions.

[IamtheSkip]

However, I do get a little annoyed when fans who've never seen a player on the field point to his statistics and contemptuously dismiss his prospects as a major league player. Statistics can only inform our projections; they don't actually determine a player's future performance. Some fans tend to forget the difference when they express their emphatic opinions.

It's baseball. On occassion, the unexpected will happen.

Anyone that actually expected Ludwick to do what he did this year was either related to him, or just picking names out of a hat and saying "This guy's breaking out."

[Migrant Redbird]

It's baseball. On occassion, the unexpected will happen.

Anyone that actually expected Ludwick to do what he did this year was either related to him, or just picking names out of a hat and saying "This guy's breaking out."

Or had actually watched him play and recognized the extent of his ability.

Statistical projections are the best we can do when we don't have good scouting reports at our disposal, but even Bill James weights statistical evaluations at only 5 percent and scouting for the other 95 percent.

When players "break out", there's a reason. Scouting can help you predict a "break out"; statistics can't.

To simply dismiss it by saying "It's baseball. On occassion [sic], the unexpected will happen." is ignoring the limitations of statistical evaluations.

[mefogus]

Or had actually watched him play and recognized the extent of his ability.

Just about every scout in the universe thought Billy Beane was going to be the next Babe. When it comes to picking guys out of high-school and college it *is* better to scout because of the lack of valuable models equating HS/college play with MLB play. However, just seeing that a guy is really good doesn't mean that he will be good.

-m

[J.D.]

Just about every scout in the universe thought Billy Beane was going to be the next Babe. When it comes to picking guys out of high-school and college it *is* better to scout because of the lack of valuable models equating HS/college play with MLB play. However, just seeing that a guy is really good doesn't mean that he will be good.

-m

Statistics can be an indicator of future performance. So can scouting. You have to use both. Otherwise, you're going to have a completely incomplete view of a player's abilities.

With instances like Ludwick and Hamilton (especially Hamilton), you have to take scouting above anything because there just wasn't much numbers-wise to work with. You absolutely have to rely on scouting. In my opinion, if you have a stats-guy telling you that scouts are worthless, that person knows just as little as a "I use observation only, and stats are worthless" person. Most people lay somewhere in between.

Just by look at the numbers going into this season, what Hamilton and Ludwick are doing are outrageous. They're coming completely from nowhere. Scouts knew that these guys, especially Hamilton, were supremely talented. But scouts can't predict the future on their own, either. You have to use both.

[Migrant Redbird]

... just seeing that a guy is really good doesn't mean that he will be good.

No, but he has a lot better chance of continuing to be "good" than a guy who's unseen or who has been observed to be "not good".

[J.D.]

No, but he has a lot better chance of continuing to be "good" than a guy who's unseen or who has been observed to be "not good".

True.

This is why using both scouting and statistical analysis is the way to go. Play in the middle ground a bit.

[davearm]

However, I do get a little annoyed when fans who've never seen a player on the field point to his statistics and contemptuously dismiss his prospects as a major league player. Statistics can only inform our projections; they don't actually determine a player's future performance. Some fans tend to forget the difference when they express their emphatic opinions.

For every fan that actually does believe that stats determine a player's future performance, there are 1,000s that are falsely accused of believing this.

Proper application of rigorous statistical analysis is a perfectly reasonable basis to dismiss a guy's prospects as a major league player, even after acknowledging that nobody's able to predict the future.

It's just a lot easier to say, "this guy can't hit a lick" than it is to say, "from a preponderance of the statistical evidence, the only reasonable conclusion, and the overwhelmingly likely outcome (although not the only possible outcome), is that this guy will hit poorly as a major leaguer."

Quite frankly, the latter meaning ought to be understood from the former statement. Panties wind up bunched only when dismissive comments are taken too literally.

[DrungoHazewood]

For every fan that actually does believe that stats determine a player's future performance, there are 1,000s that are falsely accused of believing this.

Proper application of rigorous statistical analysis is a perfectly reasonable basis to dismiss a guy's prospects as a major league player, even after acknowledging that nobody's able to predict the future.

It's just a lot easier to say, "this guy can't hit a lick" than it is to say, "from a preponderance of the statistical evidence, the only reasonable conclusion, and the overwhelmingly likely outcome (although not the only possible outcome), is that this guy will hit poorly as a major leaguer."

Quite frankly, the latter meaning ought to be understood from the former statement. Panties wind up bunched only when dismissive comments are taken too literally.

Bravo. Agree 100%.

Those who say that statistical analysis stands on its own, and is more-or-less infallible, are almost completely inventions of those who don't like the increased emphasis on statistical analysis in modern baseball.

I'm not even sure what the point of using Ryan Ludwick as some example here is. There's nothing he's doing today that wasn't plausible based solely on the totality of his minor league and major league numbers. Using him as a reason to make sure you're weighting scouting far more heavily than stats is, well, odd, since his minor league stats were great. To me he's just another Ken Phelps All Star who didn't get an extended major league chance for a long while, for whatever reason. When he got that chance he's performed. Advocating for players like him is exactly what Bill James has done for decades.

[Eight]

And of course, seeing a guy perform at X level when the statistical record would predict he performs at Y level, does not invalidate the statistical analysis. Ludwick had a fairly good projection pre-season, and we know that there is some non-zero chance he performs at the level he has over a small sample of data.

Also, was there anyone that thought a purely statistical projection of Hamilton's career was appropriate? People knew that he had talent (#1 draft pick, highly touted prospect) and that he had performed well in the minors before leaving baseball. People also knew about his drug problem, and smart people knew that we had no idea of the toll that took on his body or whether or not he was fully clean and wouldn't relapse. So I don't think that anyone, on either side of the great statistical debate, claimed to know how his career would go.

[Migrant Redbird]

For every fan that actually does believe that stats determine a player's future performance, there are 1,000s that are falsely accused of believing this.

I did say some fans, didn't I? Oh yes, here it is.

Statistics can only inform our projections; they don't actually determine a player's future performance. Some fans tend to forget the difference when they express their emphatic opinions.

If the shoe doesn't fit you, don't insist upon being one of Cinderella's stepsisters.

Proper application of rigorous statistical analysis is a perfectly reasonable basis to dismiss a guy's prospects as a major league player, even after acknowledging that nobody's able to predict the future.

It's a reasonable basis for being pessimistic about a guy's prospects, but not for dismissing them. Unless you're requiring that proper application of stats includes subordinating them to informed scouting observations.

It's just a lot easier to say, "this guy can't hit a lick" than it is to say, "from a preponderance of the statistical evidence, the only reasonable conclusion, and the overwhelmingly likely outcome (although not the only possible outcome), is that this guy will hit poorly as a major leaguer."

Quite frankly, the latter meaning ought to be understood from the former statement. Panties wind up bunched only when dismissive comments are taken too literally.

No, only when those making the dismissive comments take themselves too seriously. It's a wise man who understands the limitations of his knowledge and a prudent man who avoids being too haughty in expressing his opinions.

[TGO]

MR, I think you're making a valid point but aiming it at the wrong crowd.

There are people who dismiss players as AAAA guys, but they fall on both lines of the supposed scouting/stats divide. This thread would be more aptly titled "The Shortfalls of Writing Off Minor League Prospects," but it seems you went out of your way to take a shot at statistical projection.

[Migrant Redbird]

I am not taking a shot at statistical projections. For someone like me, who doesn't get to watch many games and isn't a sufficiently informed observer that it would make any difference, statistical analysis is all there is. That, and the opinions of honest-to-goodness legitimate scouts, but I don't know too many of those. I have no access whatsoever to the scouting reports of major league teams. I know some forum posters whose scouting opinions seem to have some validity, but I recognize that I can't trust them too far, because I lack the expertise to evaluate them. Give me my stats; stats don't lie; they just misinform, which is my fault if I let them.

However, I also recognize the fans who regard statistical analysis as the final coup d'etat to any opposing opinion. They're the target of my angst, not Jon or Davearm. I didn't name any names; I'm only responding to the shots fired across my bow.

I'm not the only one who thinks some of the stats analysts get carried away. Kansas City Star Blog: Over The Top

Mom's Basement is a stat-obsessed guy who never played sports, or if he did, probably was embarrassingly bad at them. OK, you're right, that is a totally unfair generalization of the stat-lovers. But the guys I know who resemble this character firmly and smugly believe they are an expert on any sport because they think sports can easily be reduced to a series of mathematical equations, or acronyms that go on forever, like the OPSTS, which I think stands for "Oh please shove those stats..."

Mom's Basement is aptly nicknamed because I'm convinced that's where, at 54 years old, he still resides, crouched over his computer, wearing his Star Trek jammies, researching whether, statistically speaking, it is better to bunt with a runner on second and none out in the fifth inning of night games during the mid to late spring when there is a Democrat in office and the national ratings for "American Idol" are exceeding the ARF (average rainfall) of Moose Squat, Alberta.

But, we will concede this: Mom's Basement loves sports. He just loves them at a level unlike most others. But the sports world definitely needs more fans with his commitment and his passion for the games. Why? Because the truth is, without fans like him, I'm living in my Mom's Basement, with my Star Wars jammies on, waiting for my next shift at Mickey Dees, which is probably what I should be doing anyway.

OK, it's exaggerated, but it reflects how many of the average fans react to those of us who get too carried away with the importance of our statistical masturbation.

[davearm]

I did say some fans, didn't I? Oh yes, here it is.

You did say some.

And in response, I pointed out that for every correct conclusion (or should I say accusation) that someone else holds the flawed perspective that "stats determine a player's future performance", there are 1,000s of incorrect ones.

So in general, the perceived size of "some fans" is off by several orders of magnitude, and the problem is far more one of a listener's improper inference, than of a speaker's nonsensical implication.

If the shoe doesn't fit you, don't insist upon being one of Cinderella's stepsisters.

The point is, your shoe fits virtually noone, and the "problem" you're ranting about is far more imagined than real.

Nobody I know use stats as a de facto crystal ball, yet you're suggesting that such behavior is commonplace in fan communities such as this one.

It's a reasonable basis for being pessimistic about a guy's prospects, but not for dismissing them. Unless you're requiring that proper application of stats includes subordinating them to informed scouting observations.

I have no interest in debating the semantics of pessimistic vs. dismissive. The terms seem interchangeable enough to me, but if you'd like to lecture on their fundamental and profound differences, knock yourself out.

No, only when those making the dismissive comments take themselves too seriously. It's a wise man who understands the limitations of his knowledge and a prudent man who avoids being too haughty in expressing his opinions.

Read that again and then ask yourself who's taking themself too seriously here.

[J.D.]

I'm not the only one who thinks some of the stats analysts get carried away. Kansas City Star Blog: Over The Top

OK, it's exaggerated, but it reflects how many of the average fans react to those of us who get too carried away with the importance of our statistical masturbation.

The FJM response to this complete and utter throw-away article, by the way, is absolutely terrific.

Listen to me, friend. This character you are describing? This person doesn't exist. There is no such thing as "Mom's Basement." Yes, there are people of all ages who (foolishly, I guess? according to you?) attempt to learn things about the game they love. Those d***s. There are people who love statistics, who use spreadsheets, who like the back of the baseball card as much as the front. But they all love baseball. They all love watching baseball, and going to baseball games, and they all acknowledge that there are non-statistical factors at work. Mom's Basement, as you have described it, does not exist.

Mom's Basement is a boogeyman, invented by small-brained people with no aptitude for, or interest in, learning ****. It's the baseball equivalent of a dummy going to an art museum, looking at a Miró, and saying, "That ain't art! My 2 year-old could do that!!!!!!" It's a desperate, desperately unfunny, desperately boring, desperately hacky, desperately transparent attempt to make yourself feel better about not being smart enough to understand what OPS+ means, and because you don't know how to type "define:OPS+" into Google you invent this character and hope you can pick up a few cheap laughs from readers who are as uninventive as you are.

You write for a great paper, man. Act like it.

Yep. Pretty much.