+ Reply to Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 24
  1. #1
    Aglets's Avatar
    Aglets is offline Plus Member since 12/10 Major League Starter Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Join Date
    Dec 2008
    Location
    Bel Air, MD
    Posts
    1,860

    Another thread about stats

    So all this talk about stats and which ones are the 'best' and the such as kind of got me thinking on my own.

    I'm not the kind of guy that takes a lot of things just based on what I've been told, I tend to like to do research on my own and test things out myself.

    I never really knew anything at all about wOBA before today, so thanks to eb45 for all of the work he did to educate us.

    I decided to take four different 'rate' stats from all MLB teams in 2010 (I looked at BA, OBP, OPS, and wOBA). I wanted to look for myself and see how these four correlate with run production.

    Not that complicated, but I graphed each team's run production in 2010 as a function of their BA, OBP, OPS, and wOBA. And I calculated r^2 for a linear regression each time. That just tells you how well a line 'fits' your data. the closer r^2 is to 1.......the better the fit.......herego the MORE likely that the given stat will be able to give you meaningful prediction power for run production.

    My results are below

    R^2 for AVG = 0.46 (pretty poor fit)
    R^2 for OBP = 0.77 (decent fit)
    R^2 for OPS = 0.89 (pretty good fit)
    R^2 for wOBA = 0.875*

    *This one I should probably ask for some further help on. I tried to calculate wOBA using FanGraph's formula, but the values i got for each team were slightly off for some teams compared to what FanGraphs had on their own. Only thing i can think of is maybe i used the wrong values for guys reaching base on an error? Where should I look for that? I used bb-reference, the "ROE" stat.

    CONCLUSIONS

    The findings pretty much confirm the suspicion of myself and most others that batting average really is a pretty poor judge of a team's offense and their ability to score runs. The findings confirm that .obp is a better choice, but they SEEM to show that OPS and wOBA may be the best available.

    I was surprised that OPS came out looking so good.

    Anyway, I'm wondering, do these results match what is accepted among most in the saber crowd? Is one season too small of a sample size? I was thinking about looking at another seaon or so just to make sure.

    But I thought there'd be enough saber guys on here that might find these results kind of interesting.

    NERDS.


  2. #2
    skanar is offline Plus Member Since 10/12 Major League Starter Reputation
    Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Join Date
    Aug 2008
    Posts
    1,705
    When you do a linear regression, the slope of the best fit line is also important. Could you report that as well as the R^2? We would be in better position to comment on your results.

  3. #3
    Say O! is offline Plus Member since 12/08 Major League Starter Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Join Date
    Nov 2003
    Location
    Chicago
    Posts
    1,896
    Quote Originally Posted by Aglets View Post
    So all this talk about stats and which ones are the 'best' and the such as kind of got me thinking on my own.

    I'm not the kind of guy that takes a lot of things just based on what I've been told, I tend to like to do research on my own and test things out myself.

    I never really knew anything at all about wOBA before today, so thanks to eb45 for all of the work he did to educate us.

    I decided to take four different 'rate' stats from all MLB teams in 2010 (I looked at BA, OBP, OPS, and wOBA). I wanted to look for myself and see how these four correlate with run production.

    Not that complicated, but I graphed each team's run production in 2010 as a function of their BA, OBP, OPS, and wOBA. And I calculated r^2 for a linear regression each time. That just tells you how well a line 'fits' your data. the closer r^2 is to 1.......the better the fit.......herego the MORE likely that the given stat will be able to give you meaningful prediction power for run production.

    My results are below

    R^2 for AVG = 0.46 (pretty poor fit)
    R^2 for OBP = 0.77 (decent fit)
    R^2 for OPS = 0.89 (pretty good fit)
    R^2 for wOBA = 0.875*

    *This one I should probably ask for some further help on. I tried to calculate wOBA using FanGraph's formula, but the values i got for each team were slightly off for some teams compared to what FanGraphs had on their own. Only thing i can think of is maybe i used the wrong values for guys reaching base on an error? Where should I look for that? I used bb-reference, the "ROE" stat.

    CONCLUSIONS

    The findings pretty much confirm the suspicion of myself and most others that batting average really is a pretty poor judge of a team's offense and their ability to score runs. The findings confirm that .obp is a better choice, but they SEEM to show that OPS and wOBA may be the best available.

    I was surprised that OPS came out looking so good.

    Anyway, I'm wondering, do these results match what is accepted among most in the saber crowd? Is one season too small of a sample size? I was thinking about looking at another seaon or so just to make sure.

    But I thought there'd be enough saber guys on here that might find these results kind of interesting.

    NERDS.
    OPS comes out so well because it captures the largest components of run production.

    But, the reason that wOBA looks relatively "worse" in terms of R^2 is because you are not comparing apples to apples. wOBA adjusts for park effects to "normalize" performance, whereas runs scored is just a raw unadjusted stat. Because your OPS figure is not adjusted for park effects, it looks relatively "better" in terms of predicting runs scored.

  4. #4
    Say O! is offline Plus Member since 12/08 Major League Starter Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Join Date
    Nov 2003
    Location
    Chicago
    Posts
    1,896
    Also would be interesting to understand which teams fall above/below the respective regression lines. TAM, I suspect, will show a strong above the line variance because of their baserunning effects and a much higher OPS with runners on base than their full year avg.

  5. #5
    Aglets's Avatar
    Aglets is offline Plus Member since 12/10 Major League Starter Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Join Date
    Dec 2008
    Location
    Bel Air, MD
    Posts
    1,860
    So because I wanted to see more......

    I did the same thing with all of the MLB teams in 2009.

    Performed a linear regression for all of the data that graphed Runs Scored as a function of OPS.

    Guess what, r^2 came out even better!

    for 2009 the value for r^2 was 0.919!

    I think reports of the death of OPS are greatly exaggerated, can someone answer my concerns earlier about wOBA? I want to make sure I was calculating it right.

  6. #6
    Frobby is online now Hangout Blogger Hall of Fame Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Reputation Reputation Reputation
    Join Date
    Dec 2003
    Location
    Bethesda MD
    Posts
    65,194
    Quote Originally Posted by Aglets View Post
    I think reports of the death of OPS are greatly exaggerated, can someone answer my concerns earlier about wOBA? I want to make sure I was calculating it right.
    I don't know how to calculate wOBA, which is part of the problem with that stat, it's just too complicated for a quick calculation. Your regression analysis is pretty interesting, because even if you are doing wOBA wrong, it shows that team OPS at least comes close to being just as good as a predicter of runs scored. Considering how easy it is to calculate, it will remain a very useful, accessible stat.

  7. #7
    Join Date
    Mar 2007
    Location
    Brooklyn, NY
    Posts
    5,184
    Quote Originally Posted by Aglets View Post
    So all this talk about stats and which ones are the 'best' and the such as kind of got me thinking on my own.

    I'm not the kind of guy that takes a lot of things just based on what I've been told, I tend to like to do research on my own and test things out myself.

    I never really knew anything at all about wOBA before today, so thanks to eb45 for all of the work he did to educate us.

    I decided to take four different 'rate' stats from all MLB teams in 2010 (I looked at BA, OBP, OPS, and wOBA). I wanted to look for myself and see how these four correlate with run production.

    Not that complicated, but I graphed each team's run production in 2010 as a function of their BA, OBP, OPS, and wOBA. And I calculated r^2 for a linear regression each time. That just tells you how well a line 'fits' your data. the closer r^2 is to 1.......the better the fit.......herego the MORE likely that the given stat will be able to give you meaningful prediction power for run production.

    My results are below

    R^2 for AVG = 0.46 (pretty poor fit)
    R^2 for OBP = 0.77 (decent fit)
    R^2 for OPS = 0.89 (pretty good fit)
    R^2 for wOBA = 0.875*

    *This one I should probably ask for some further help on. I tried to calculate wOBA using FanGraph's formula, but the values i got for each team were slightly off for some teams compared to what FanGraphs had on their own. Only thing i can think of is maybe i used the wrong values for guys reaching base on an error? Where should I look for that? I used bb-reference, the "ROE" stat.

    CONCLUSIONS

    The findings pretty much confirm the suspicion of myself and most others that batting average really is a pretty poor judge of a team's offense and their ability to score runs. The findings confirm that .obp is a better choice, but they SEEM to show that OPS and wOBA may be the best available.

    I was surprised that OPS came out looking so good.

    Anyway, I'm wondering, do these results match what is accepted among most in the saber crowd? Is one season too small of a sample size? I was thinking about looking at another seaon or so just to make sure.

    But I thought there'd be enough saber guys on here that might find these results kind of interesting.

    NERDS.
    Quote Originally Posted by Frobby View Post
    I don't know how to calculate wOBA, which is part of the problem with that stat, it's just too complicated for a quick calculation. Your regression analysis is pretty interesting, because even if you are doing wOBA wrong, it shows that team OPS at least comes close to being just as good as a predicter of runs scored. Considering how easy it is to calculate, it will remain a very useful, accessible stat.
    Here's a spreadsheet for wOBA calculation. I'm not sure if I've used this one, but there are a few floating around.

    http://tinyurl.com/6x3o83c

    Since WAR seems to be very popular on the site here, more attention should be given to wOBA, since oWAR is a function of wOBA.

    Once you have wOBA, the formula for offensive wins above average is ([wOBA-.338]/1.15) * (PA/10.5). (Note: .338 is a rough estimate of average wOBA, but it shifts from year to year.)

    Add about 20 runs (~2 wins) for replacement level if the player has accumulated ~600 PA.

  8. #8
    Join Date
    Jan 2006
    Location
    St. Michaels
    Posts
    6,399
    I had my friend take a look at this thread. He has a Masters in Statistics and a basic understanding of baseball in that he is American and read Moneyball. But, had didn't know much about OPS or wOBP. He made some very good points about this debate.

    1. Frobby is right. The simplicity of OPS makes it useful because if it parsimonious.

    Chat transcript:
    from a stats point of view
    parsimony
    is often important
    when predicting future outcomes
    its very easy to include EVERYTHING
    and say "yes, by knowing everything, we can predict what is most likely to come next"
    but EVERYTHING is often expensive to collect record clean and maintain
    so reducing it to the point where is quick and easy
    is often important

    ...

    Phil
    saving the more complicated model for performance
    for when its important
    so on the surface
    OPS seems much more parsimonious
    but, it could be that the other stat includes something really important
    but if the R-sq comes out the same most of the time
    you might question why you use the complicated one
    that said, there is probably a reason someone developed the complicated one
    and thats stats in a nutshelll- no clear answers
    2. Two good rate stats are better then 1. When wOPB and OPS tell you same thing, be happy we have more confidence in the predicative quality of the stats.

    3. At the end of the day, if the outcomes of OPS and wOBP aren't incredibly different, then choose the simpler one OPS.

    Parsimony.

    Phil
    both OPS and wOBP high...good
    both low
    bad
    signficant difference between the two
    proceed with caution
    if two players have similar, hire the one with the better personality
    boom

  9. #9
    Join Date
    Mar 2007
    Location
    Brooklyn, NY
    Posts
    5,184
    Quote Originally Posted by srock View Post
    I had my friend take a look at this thread. He has a Masters in Statistics and a basic understanding of baseball in that he is American and read Moneyball. But, had didn't know much about OPS or wOBP. He made some very good points about this debate.

    1. Frobby is right. The simplicity of OPS makes it useful because if it parsimonious.

    Chat transcript:


    2. Two good rate stats are better then 1. When wOPB and OPS tell you same thing, be happy we have more confidence in the predicative quality of the stats.

    3. At the end of the day, if the outcomes of OPS and wOBP aren't incredibly different, then choose the simpler one OPS.

    Parsimony.
    Good post, but here's an excellent discussion on wOBA and correlation to runs.

    The methodology for measuring that correlation gets a bit sticky. Working right now so I can't expand much at the moment, but check out Tom's comments:

    http://www.insidethebook.com/ee/inde...cked_up_to_be/

  10. #10
    Aglets's Avatar
    Aglets is offline Plus Member since 12/10 Major League Starter Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Join Date
    Dec 2008
    Location
    Bel Air, MD
    Posts
    1,860
    Alrighty then.

    So I decided to investigate the assumption that i miscalculated wOBA.

    Instead of using my 'homemade' wOBA values I simply took the given values provided at Fangraphs's website.

    Comparing OPS to wOBA vs. Runs Scored

    in 2010

    r^2 for OPS = .891
    r^2 for wOBA = .908

    in 2009

    r^2 for OPS = .919
    r^2 for wOBA = .923

    So this seems to make a bit more sense to me I suppose. wOBA looks like it's a 'slight' improvement over OPS, but very slight in the grand scheme of things.

    Also, I'm a bit concerned that I couldn't exactly replicate the FG values for wOBA, but anyway, all of this seems to show that OPS is a lot more interesting/useful than we might have previously anticipated.

    Pretty cool.

  11. #11
    Slappy's Avatar
    Slappy is offline Plus Member Since 07/09 All-Star Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation Reputation
    Join Date
    Oct 2006
    Posts
    3,428
    wOBA seems like an awful lot of work for not a whole lot of payoff.

  12. #12
    Join Date
    Jan 2006
    Location
    St. Michaels
    Posts
    6,399
    Quote Originally Posted by Slappy View Post
    wOBA seems like an awful lot of work for not a whole lot of payoff.
    What I'm getting from all this is that OPS may have flaws but they are largely academic.

    I'm going to use both stats from now on but only wOBP when I'm digging deep into details.

  13. #13
    Join Date
    Jul 2009
    Location
    Baltimore
    Posts
    324
    Very nice work so far. Here are a few comments that might help you.

    It is important to define the unit of observation you are assessing with your r2 statistic. So if the question is, how well does OPS correlate with runs scored, we must first ask the question, what is our unit of observation? It looks like you took a team's OPS for a season and regressed that against the total runs scored. That will give you one answer.

    If instead we created an OPS for each game and correlated that with runs scored in the game, we would get a different estimate. We would get a different one still if we used runs scored in each inning. And the differences are substantial:

    Code:
    Observation Unit	r2 between runs and OPS 
    team			0.889
    games			0.678
    inning			0.495

    The problem with using summary statistics for inference is that conclusions could fall prey to the ecological fallacy. Interpret with caution.

    In general, you want to use the finest level possible. But that's tricky in baseball because although some stats are based upon plate appearances (e.g. OBP), runs scored by plate appearances are not independent. Given that, it seems innings is the finest appropriate level, though the interpretation of a team's OBP or OPS in an inning is not often the parameter of interest (when, for example, you want ot evaluate players).

    Hope that helps.

  14. #14
    Join Date
    Sep 2003
    Location
    Maui, Hawaii
    Posts
    2,133
    Quote Originally Posted by Say O! View Post
    OPS comes out so well because it captures the largest components of run production.

    But, the reason that wOBA looks relatively "worse" in terms of R^2 is because you are not comparing apples to apples. wOBA adjusts for park effects to "normalize" performance, whereas runs scored is just a raw unadjusted stat. Because your OPS figure is not adjusted for park effects, it looks relatively "better" in terms of predicting runs scored.
    Aglets, very interesting so far. Why don't you take account of the above and simply use OPS+ instead of OPS - or at least add it to the selection of stats you're comparing?

  15. #15
    Catch 8's Avatar
    Catch 8 is offline Plus Member Since July 2009 Major League Starter Reputation Reputation Reputation Reputation Reputation
    Join Date
    Nov 2006
    Posts
    1,234
    Ahhhhhhhhhh! As a statistician, I find it my obligation to chime in. Do not blindly use regression!!! Check your assumptions:

    1) is your data linear? If not, is there a transformation that makes it linear (i.e, quadratic, inverse, square root, cubic, etc.)?
    2) Is your data normally distributed? Construct a histogram in Excel. The data should be roughly bell shaped, no skew character, roughly symmetric and unimodal.
    3) Independence - this you probably satisfied, since I presume you did not count the same player twice in any instance.
    4) Heteroscedasticity - Is there constant variation of the errors in your data? Note that a large error is an observation (data point) that is far away from the least square (regression) line. In contrast, a small error means that the data point lies very close to or on the regression line. What is important is that all the errors are roughly the same. If it is borderline, you proceed with regression. When you have a violation is upon plotting the residuals (errors), you notice any familiar, suspicious, cyclical, funneling out, funneling in, parabolic patterns.
    5) Outliers. Understand if the outliers "make sense" in the context of your study. Note that it is generally a BAD practice to throw out outliers, without a sensible reason (such as incorrect or incomplete information). For example, if you were interested in OPS in the early 2000s, Barry Bonds would be a big-time outlier. However, throwing him out because it is convenient is a terrible practice; he is a legitimate member of the population we are studying. So what do we do in the case of outliers/non-normality? A transformation! I would start off playing around with a log transformation, since that would bring Barry back closer to the rest of the data.
    6) Multicollinearity (same as collinearity) - Since you are performing a simple linear regression (meaning just one independent variable), this is not much a concern for us. In the multiple linear regression case, we would have to make sure that none of the independent variables are related to each other.

    I applaud you for your effort and interest in statistics! However, I just want you to know that regression makes certain assumptions and if you fail to verify them and they are violated, your conclusions will most likely be very inaccurate. If you have any questions, PM me (after Wednesday since I have a graduate course exam that day).

Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

OriolesHangout.com is an unofficial site and not associated with the Baltimore Orioles and part of Hangout Ventures LLC. Copyright ©2013 | Privacy Policy | Advertise with us