More hilarious than ever: PECOTA projects the Orioles at 71-91

[Number5]

5 minutes ago, Frobby said:

There's a difference between a prediction based on subjective opinion and a projection based on a mathematical model.    PECOTA and fangraphs are the latter.   But where are you finding the fangraphs projections?    The list I see has them at 79-83.  http://www.fangraphs.com/depthcharts.aspx?position=Standings

Perhaps there's less of a difference than you may think.  The computer won't make any errors in calculations, but the calculations it runs are based on human input.  The model (program) is designed by people.  PECOTA's track record speaks for itself, and it doesn't speak very highly.

[tntoriole]

1 hour ago, Frobby said:

There's a difference between a prediction based on subjective opinion and a projection based on a mathematical model.    PECOTA and fangraphs are the latter.   But where are you finding the fangraphs projections?    The list I see has them at 79-83.  http://www.fangraphs.com/depthcharts.aspx?position=Standings

There is not much science involved in using arbitrarily weighted statistics in a made up computer program. Science involves comparing one's theories and models with actual measurements.  If a scientific model was as inaccurate as this madeup collection of statistical biases, it would have been put in the category of pseudoscience long ago.  Just because someone uses numbers in a self created computer program does not make it scientific or any more predictive than those of panels of observers.  

http://www.fangraphs.com/blogs/here-are-your-expected-2017-standings/

[weams]

Did they change to 75 yet this morning?

[weams]

6 hours ago, tntoriole said:

There is not much science involved in using arbitrarily weighted statistics in a made up computer program. Science involves comparing one's theories and models with actual measurements.  If a scientific model was as inaccurate as this madeup collection of statistical biases, it would have been put in the category of pseudoscience long ago.  Just because someone uses numbers in a self created computer program does not make it scientific or any more predictive than those of panels of observers.  

http://www.fangraphs.com/blogs/here-are-your-expected-2017-standings/

You mean cause and effect shown in the model?

[Frobby]

Steve Melewski's take on the PECOTA projections: http://www.masnsports.com/steve-melewski/2017/02/a-look-at-the-pecota-projection-and-os-infield-depth.html

[Frobby]

8 hours ago, tntoriole said:

There is not much science involved in using arbitrarily weighted statistics in a made up computer program. Science involves comparing one's theories and models with actual measurements.  If a scientific model was as inaccurate as this madeup collection of statistical biases, it would have been put in the category of pseudoscience long ago.  Just because someone uses numbers in a self created computer program does not make it scientific or any more predictive than those of panels of observers.  

http://www.fangraphs.com/blogs/here-are-your-expected-2017-standings/

You are mistaken if you thought I was implying that PECOTA and the fangraphs projection systems were better than the subjective judgments of the other sources.   I'm just saying they're different.    

I do think people lose sight of the fact that PECOTA wasn't created to forecast team results.    It was created to forecast results for individual players.   Its not like it was designed to screw the Orioles.    But to the extent it's being used to forecast team results, it's obviously not done a good job.   

[LC_O's_87]

These debates about projections are long-running but I've always felt there's a human factor that's missing from these projections.

If you're up 8-2 in the 7th, you can trot out your 7th best bullpen guy for 2 innings and let him soak up innings and give up 3 runs.  You accepted a sub-optimal result under Pecota's projections because a win is a win, whether it's by 6 runs or 3 runs. 

Similarly, if you have Ubaldo going on a Sunday and a big series coming up against a division rival on Monday, you sacrifice him to the lambs and essentially punt the game.  Instead of using 5 guys out of the pen you let Ubaldo get shelled and then you let the next guy take it on the chin.  You lose by 10 runs when you could have, with optimal player usage, only lost by 5 runs.  1 loss is only 1 loss after all.

Human pragmatism in managing a team over the course of a 162 game is the element missing from the projections.  Managers make decisions based on winning, not on maximizing run differential.  While projecting run differential is probably the best way to approximate W-L record, it's inherently flawed imo.

[thezeroes]

1 hour ago, LC_O's_87 said:

These debates about projections are long-running but I've always felt there's a human factor that's missing from these projections.

If you're up 8-2 in the 7th, you can trot out your 7th best bullpen guy for 2 innings and let him soak up innings and give up 3 runs.  You accepted a sub-optimal result under Pecota's projections because a win is a win, whether it's by 6 runs or 3 runs. 

Similarly, if you have Ubaldo going on a Sunday and a big series coming up against a division rival on Monday, you sacrifice him to the lambs and essentially punt the game.  Instead of using 5 guys out of the pen you let Ubaldo get shelled and then you let the next guy take it on the chin.  You lose by 10 runs when you could have, with optimal player usage, only lost by 5 runs.  1 loss is only 1 loss after all.

Human pragmatism in managing a team over the course of a 162 game is the element missing from the projections.  Managers make decisions based on winning, not on maximizing run differential.  While projecting run differential is probably the best way to approximate W-L record, it's inherently flawed imo.

Very well said.  Thank You.

[Frobby]

1 hour ago, LC_O's_87 said:

These debates about projections are long-running but I've always felt there's a human factor that's missing from these projections.

If you're up 8-2 in the 7th, you can trot out your 7th best bullpen guy for 2 innings and let him soak up innings and give up 3 runs.  You accepted a sub-optimal result under Pecota's projections because a win is a win, whether it's by 6 runs or 3 runs. 

Similarly, if you have Ubaldo going on a Sunday and a big series coming up against a division rival on Monday, you sacrifice him to the lambs and essentially punt the game.  Instead of using 5 guys out of the pen you let Ubaldo get shelled and then you let the next guy take it on the chin.  You lose by 10 runs when you could have, with optimal player usage, only lost by 5 runs.  1 loss is only 1 loss after all.

Human pragmatism in managing a team over the course of a 162 game is the element missing from the projections.  Managers make decisions based on winning, not on maximizing run differential.  While projecting run differential is probably the best way to approximate W-L record, it's inherently flawed imo.

I understand what you're saying, but the biggest reason the PECOTA projections have been wrong about the Orioles is because they haven't projected the run differential well, not because run differential hasn't correlated well with our W-L.   We have outperformed the record you'd expect based on run differential by 16 wins over the last five years (11 of those coming in 2012), but we have outperformed PECOTA by 73 wins.    So, about 78% of the deviation relates to them underestimating our run differential.

[Spy Fox]

11 hours ago, tntoriole said:

There is not much science involved in using arbitrarily weighted statistics in a made up computer program. Science involves comparing one's theories and models with actual measurements.  If a scientific model was as inaccurate as this madeup collection of statistical biases, it would have been put in the category of pseudoscience long ago.  Just because someone uses numbers in a self created computer program does not make it scientific or any more predictive than those of panels of observers.  

http://www.fangraphs.com/blogs/here-are-your-expected-2017-standings/

That link is to the results of a Fangraphs poll asking fans last November how many games they expected each team to win in 2017, not the actual Fangraphs statistical projections. 

But I think you make some good points with regard to the fallibility of projections. 

[LC_O's_87]

1 hour ago, Frobby said:

I understand what you're saying, but the biggest reason the PECOTA projections have been wrong about the Orioles is because they haven't projected the run differential well, not because run differential hasn't correlated well with our W-L.   We have outperformed the record you'd expect based on run differential by 16 wins over the last five years (11 of those coming in 2012), but we have outperformed PECOTA by 73 wins.    So, about 78% of the deviation relates to them underestimating our run differential.

Fair point, and it does underscore your earlier point about issue with PECOTA's projections being this problem:

OK, I've gone back a few years.

2013: Projected 711 runs scored (745 actual), 782 allowed (709)

2014: Projected 686 runs scored (705 actual), 746 allowed (593)

2015: Projected 698 runs allowed (693 actual) (can't find a runs scored projection)

2016: Projected 697 runs scored (744 actual), 786 allowed (715).

So why has PECOTA struggled so mightily to accurately project the Orioles in runs allowed?

[Frobby]

1 hour ago, LC_O's_87 said:

So why has PECOTA struggled so mightily to accurately project the Orioles in runs allowed?

I don't know enough about how they identify the comparable pitchers to guess why their projections are so far off for our staff.

[SteveA]

4 hours ago, LC_O's_87 said:

Fair point, and it does underscore your earlier point about issue with PECOTA's projections being this problem:

OK, I've gone back a few years.

2013: Projected 711 runs scored (745 actual), 782 allowed (709)

2014: Projected 686 runs scored (705 actual), 746 allowed (593)

2015: Projected 698 runs allowed (693 actual) (can't find a runs scored projection)

2016: Projected 697 runs scored (744 actual), 786 allowed (715).

So why has PECOTA struggled so mightily to accurately project the Orioles in runs allowed?

In some of those years our excellent defense may have helped our pitching do better than expected.   I don't think our overall defense was that great last year, although the infield defense certainly is.

Looking at Tillman's projection this year and comparing it with his track record and age, it simply doesn't make sense.  And other projections seem similarly biased in a negative way.   To understand why they are making such projections, I would have to understand more about their system.

Trying to study their system to learn more about it would be a waste of my time, however.   Like learning to do multiplication and division with Roman numerals.  The results show it isn't very good so I'm not going to try to delve into understand why.   I'll just celebrate the annual "Orioles win total surpasses PECOTA prediction Day" which usually comes in late August or early September.

[25 Nuggets]

On 2/7/2017 at 6:26 PM, Frobby said:

I didn't mention it in the OP, but they projected Sisco at .713 OPS for 2017 and .750ish in the future.

750 from a catcher in today's game ain't bad at all.

Which brings us back to the point that Sisco needs to stick at catcher.

[George]

PECOTA only has the veneer of a scientific report because it uses numbers. In an absurd move, PECOTA reports numbers to three significant digits. At that point you know PECOTA is just messing with you.