Small sample sizes

[Conway12]

17 hours ago, Frobby said:

I’ve seen these kind of numbers, but I’m not really sure what “stabilize” really means in this context.   

I have to dust off the old statistics class, so a lot of this may be wrong. But, in the Fangraphs glossary article they say that "stabilization" is when the R^2 of the variable crosses .49 to .50. Said another way, the variable (for example strikeout rate) "stabilizes" at a certain sample size (60 PA). In this case, "stabilize" means that the variable (Strikeout Rate) is at least 50 percent determined by the sample size (60 PA). So, going forward, you would expect the variable (Strikeout Rate) to not go 50% higher or lower than when it crosses that "stabilization" point (60 PA). For example, if after 60 PA you had a strikeout rate of 10% you would only expect it to go up or down 5% (or less) moving forward. 

However, that is only if the sample (60 PA) is the exact same as the previous sample (60 PA), which of course it is not in baseball. Russel Carleton, the guy who popularized this stuff in baseball, has written on numerous occasions that his "stabilization point" was some what arbitrary and is used incorrectly nowadays to explain early season performances. He says it's really more like 150 PA to explain strikeout rate in a season to get to the "stabilization point." You should read the linked article and anything else he has written on the subject to understand it better. Here is quote from that article that I linked. 

Quote

The generally accepted "stability numbers" chart is a good chart for researchers who are doing retrospective research. I think it's also a good one to look at in terms of understanding which stats stabilize more quickly relative to others, which I think can show us some interesting truths about the game. However, I would kindly point out that they are not nearly as powerful in predicting future performance as people seem to believe that they are.

 

[Moose Milligan]

15 hours ago, now said:

This one stands out to me as the most surprising:

• 910 AB: AVG

In other words, nearly two full seasons to accurately predict the next...?

I can believe it.  A great rookie year, sophomore slump...3rd year probably tells you more about who a player really is.

[StottyByNature]

16 hours ago, now said:

This one stands out to me as the most surprising:

• 910 AB: AVG

In other words, nearly two full seasons to accurately predict the next...?

Baseball is a game of funny outcomes.  You can tap a ball in front of the plate and get a hit, or hit a screaming liner for an out.  In order for those outcomes to balance out it would take a large sample size.  It's also why hard-hit percentage is a better measure for performance, and why BABIP is a nice measure of a lucky or unlucky season.

[emmett16]

1 hour ago, Moose Milligan said:

I can believe it.  A great rookie year, sophomore slump...3rd year probably tells you more about who a player really is.

Thats why AVG EV and Barrel % are better than the counting stats for predictive measures.  They stabilize much quicker (50 balls put into play).