PECOTA projections

[BohKnowsBmore]

PECOTA is as precise and accurate as today's forecasted high temperature. I am not mad at the weather forecast.

I'm not mad at Pecota, but it has been wrong on the Orioles the last four years. And it is often wrong on the Red Sox.

Same here. I'm not mad at PECOTA. If the weatherman were systematically off w/r/t the weather forecast in the same direction, I would question what is wrong with the predictive model, though.

[Frobby]

I'm not mad at Pecota, but it has been wrong on the Orioles the last four years. And it is often wrong on the Red Sox.

There is literally no reason to think that the system is intentionally slanted for or against a particular team. It's just a series of mathematical calculations based on player history and the history of comparable players. For the players, there's no subjective input once the parameters are set, as I understand it. I assume there is a small amount of subjectivity in determining which players are going to get the playing time on a particular team, but there's no reason to think there is some systemic bias there that would favor one team over another. So, if PECOTA has been wrong on a specific team, it's just one of those things.

That said, I think the 72-90 projection is ridiculous. Of course, it will change as the roster changes, but I doubt PECOTA will have us over 76 wins even if we add Gallardo and Fowler.

[Redskins Rick]

If PECOTA was around in 69, they probably have the Birds finishing in last with 73 wins.

[Number5]

There is literally no reason to think that the system is intentionally slanted for or against a particular team. It's just a series of mathematical calculations based on player history and the history of comparable players. For the players, there's no subjective input once the parameters are set, as I understand it. I assume there is a small amount of subjectivity in determining which players are going to get the playing time on a particular team, but there's no reason to think there is some systemic bias there that would favor one team over another. So, if PECOTA has been wrong on a specific team, it's just one of those things.

That said, I think the 72-90 projection is ridiculous. Of course, it will change as the roster changes, but I doubt PECOTA will have us over 76 wins even if we add Gallardo and Fowler.

I agree that the system is almost surely not slanted against any particular team, but I do believe that it favors certain types of teams disproportionately, and disfavors other types of teams. As a result teams constructed in certain ways tend to regularly outperform PECOTA, while other teams constructed in another way tend to regularly under-perform their projections. Despite this, PECOTA doesn't change their metrics, preferring to attribute these regular misses to luck. I would start worrying if PECOTA ever started to project the Orioles to do well.

[Frobby]

No, but at some point you have to start questioning the model. It's wrong so much that it's basically worthless at this point. And that's just in regards to one team. I'm sure it's wrong the majority of the time for the majority of the teams. So why is it even a thing that seems to have as much legitimacy in the baseball world as it does? Surely there are more accurate ways to model predictions.

PECOTA wasn't really designed to make team projections. It was designed to make player projections. I think it has done decently on that score, compared to other projection systems that are out there. But there is a lot of uncertainly in baseball projections. It's just the nature of the beast, and frankly, it's one of the main things that makes baseball so fun to follow.

[Frobby]

I agree that the system is almost surely not slanted against any particular team, but I do believe that it favors certain types of teams disproportionately, and disfavors other types of teams. As a result teams constructed in certain ways tend to regularly outperform PECOTA, while other teams constructed in another way tend to regularly under-perform their projections. Despite this, PECOTA doesn't change their metrics, preferring to attribute these regular misses to luck. I would start worrying if PECOTA ever started to project the Orioles to do well.

What types of teams does it favor or disfavor? I think it's dangerous to rely on the last four years involving the Orioles. PECOTA has been around much longer that that, and it covers pretty much all the players in the league.

Personally, I do not think the big flaw is with PECOTA. The big flaw is how the humans at BP use the individual PECOTA projections to construct team projections. I don't know how to do it any better than they do, though.

[Number5]

What types of teams does it favor or disfavor? I think it's dangerous to rely on the last four years involving the Orioles. PECOTA has been around much longer that that, and it covers pretty much all the players in the league.

Personally, I do not think the big flaw is with PECOTA. The big flaw is how the humans at BP use the individual PECOTA projections to construct team projections. I don't know how to do it any better than they do, though.

Seems like Boston and Detroit have tended to regularly be projected higher than they end up, and Baltimore and Kansas City seem to be regularly under-projected. That would lead me to think that perhaps the factors involving bull pen and defense may not be given enough weight. Hey, I don't know, but it sure seems that way to me. I certainly don't know how to do it, either. Just saying that if I were in that business and I saw these regular misses, I would look into what we might be doing wrong, rather than assume that certain teams are unbelievably lucky year after year.

[DrungoHazewood]

There is literally no reason to think that the system is intentionally slanted for or against a particular team. It's just a series of mathematical calculations based on player history and the history of comparable players. For the players, there's no subjective input once the parameters are set, as I understand it. I assume there is a small amount of subjectivity in determining which players are going to get the playing time on a particular team, but there's no reason to think there is some systemic bias there that would favor one team over another. So, if PECOTA has been wrong on a specific team, it's just one of those things.

That said, I think the 72-90 projection is ridiculous. Of course, it will change as the roster changes, but I doubt PECOTA will have us over 76 wins even if we add Gallardo and Fowler.

I would imagine that playing time forecasts are one of, if not the, major differentiator between projection models. Especially how they deal with the high attrition rate among pitchers.

[DrungoHazewood]

I don't know how to do it any better than they do, though.

Crowdsourcing it using fans, similar to the Fan's Scouting Report. Or using the fan projections from Fangraphs, just for playing time. But that will still miss Matt Wieters missing four months. No way to project that.

[Frobby]

Crowdsourcing it using fans, similar to the Fan's Scouting Report. Or using the fan projections from Fangraphs, just for playing time. But that will still miss Matt Wieters missing four months. No way to project that.

You can't forecast injuries, but perhaps one could determine which teams are best positioned to adjust in case of injuries, in a way better than BP does. On the other hand, maybe not -- I don't think anyone could have foreseen how well we adjusted when Wieters went down, for example.

[Phantom]

Crowdsourcing it using fans, similar to the Fan's Scouting Report. Or using the fan projections from Fangraphs, just for playing time. But that will still miss Matt Wieters missing four months. No way to project that.

That's what fangraphs does. They have Streamer projections and the crowdsource projections.

And you know what? Streamer is more accurate every single year.

[DrungoHazewood]

That's what fangraphs does. They have Streamer projections and the crowdsource projections.And you know what? Streamer is more accurate every single year.

What does Steamer use for playing time projections? Of course they're more accurate than the fans. The fans always, always, always are too optimistic. Fans usually rate their hometown players, and overrate them by 10-20%. They've documented that on Fangraphs. I think if you took the fan forecasts and cut 10 wins a team off you'd do pretty well.

[Stotle]

What does Steamer use for playing time projections? Of course they're more accurate than the fans. The fans always, always, always are too optimistic. Fans usually rate their hometown players, and overrate them by 10-20%. They've documented that on Fangraphs. I think if you took the fan forecasts and cut 10 wins a team off you'd do pretty well.

That would make the Orioles around a 77 win team?

[DrungoHazewood]

That would make the Orioles around a 77 win team?

Do the fan forecasts have them at 87?

[Stotle]

Do the fan forecasts have them at 87?

I was going with forecasts on this board. Somewhere around mid- to upper-80 wins.