FWIW, Fangraphs Projects O's in Last Place in 2015

[Number5]

Fair enough. But since we are talking about assigning value to defense, and then using that value to make predictions, variance in skill and variance in the best numbers we have to measure that skill are basically the same.

In other words, my bad. Should have been more clear.

Well, the fact that Fangraph's projections are consistently wrong year after year by a large number for the Orioles and other teams that have either clearly above average defenses, as well as those that have clearly below average defenses, seems to be clear evidence, but go ahead and believe what you want. There is variance in every factor, including FIP. That is all part of making projections. Of course the exact defensive numbers aren't going to be identical every season. Neither is FIP, for crying out loud. But we know that Baltimore has a good defensive team. That is not in dispute by Fangraphs (they even have a defensive matrix, as I'm sure you are aware)... it is simply completely ignored. To intentionally ignore a known element is just poor on Fangraph's part.

[DrungoHazewood]

I guess I just feel FIP is very overrated. If I can go back to basics, pitching results have three components: skill of the pitcher, skill of the defense, and luck. I don't think FIP fully captures skill of the pitcher. Not all fly balls are alike. Not all ground balls are alike. I believe some pitchers are better than others at eliciting the kind of contact that leads to outs, and I don't think FIP captures it well. Maybe in another five years, with the advent of hit f/x and StatCast, someone will come up with a more sophisticated system than FIP. For now, I just use it as a tool to see if I should have any concerns about whether ERA results are sustainable, but I don't see it as a great predicter of future results.

If FIP is very overrated (at least when using it to project future performances), then ERA is even moreso. FIP has a higher correlation to future ERA than present ERA does.

[Phantom]

I guess I just feel FIP is very overrated. If I can go back to basics, pitching results have three components: skill of the pitcher, skill of the defense, and luck. I don't think FIP fully captures skill of the pitcher. Not all fly balls are alike. Not all ground balls are alike. I believe some pitchers are better than others at eliciting the kind of contact that leads to outs, and I don't think FIP captures it well. Maybe in another five years, with the advent of hit f/x and StatCast, someone will come up with a more sophisticated system than FIP. For now, I just use it as a tool to see if I should have any concerns about whether ERA results are sustainable, but I don't see it as a great predicter of future results.

You are letting the perfect be the enemy of the good here.

[LoyalBirdFan]

I guess I just feel FIP is very overrated. If I can go back to basics, pitching results have three components: skill of the pitcher, skill of the defense, and luck. I don't think FIP fully captures skill of the pitcher. Not all fly balls are alike. Not all ground balls are alike. I believe some pitchers are better than others at eliciting the kind of contact that leads to outs, and I don't think FIP captures it well. Maybe in another five years, with the advent of hit f/x and StatCast, someone will come up with a more sophisticated system than FIP. For now, I just use it as a tool to see if I should have any concerns about whether ERA results are sustainable, but I don't see it as a great predicter of future results.

Good post, FIP is just a stat. If you want to find out what a pitcher is going to do next year you have to look at PERA.

PERA (Projected ERA) is easily derived....

ERA+FIP*3.14 (Pi)/7

This doesn't account for the number of playoff appearances or WS rings the manager has though.

[Frobby]

You are letting the perfect be the enemy of the good here.

Meh. I've already told you I consider FIP. If I had to Vegas today and bet on whether Tillman and Gonzo would be over or under the Steamer projections that are based on FIP, I'd gladly take the under. I can't watch every pitcher in MLB, but I can watch the Orioles' pitchers, and I have confidence that these are two pretty good pitchers.

[MarylandAaron]

Yeah they are the sharpest tools in the shed. How many times can you be wrong about something before you take your worthless simple stat and realize it is meaningless and try and come up with a new theory?

I wonder what place they had the O's in last season. I am guessing it wasn't first or even second.

As soon as someone start mentioning FIP I know they are clueless and don't know the first thing about statistics or baseball.

Seeing how consistently wrong they are doesn't seem to deter anyone at Fangraphs from quoting those stats religiously.

[Phantom]

Seeing how consistently wrong they are doesn't seem to deter anyone at Fangraphs from quoting those stats religiously.

This is like describing the best hitter in the league as "consistently bad at hitting" because he gets out 6 out of 10 times. When you don't add context, any projections are going to seem poor.

So I ask you, Fangraphs are consistently wrong compared to whom?

[MarylandAaron]

This is like describing the best hitter in the league as "consistently bad at hitting" because he gets out 6 out of 10 times. When you don't add context, any projections are going to seem poor.

So I ask you, Fangraphs are consistently wrong compared to whom?

They seem to be consistently wrong on the O's, and as another poster pointed out, just about all their team projections from last year were way off the mark. I don't know how they are in comparison to other projection systems, but I just find it funny that they don't ever consider that maybe their model is flawed. If the projections are off, it's always an "outlier", which makes their projections right no matter what.

[stomperspc]

I realize that, but at some point you have to confront the reality that these two pitchers outperform their FIP every single year, and factor that in to your projections if you want the projections to be as good as they can be. I don't think the creators of Steamer are inclined to make individual exceptions to their methodology. That's fine, and I don't necessarily blame them, but in these two cases the chances are very good that their projections will be far off base.

I have done quite a bit of work with financial/business plan projections in the course of my career. It would be borderline laughable to suggest that the same projection model can be applied to thirty different companies even in the same industry. Almost no two companies are alike and they all present their own unique challenges when it comes to forecasting earnings. Perhaps the thirty MLB teams are all operated similarly enough that you can get by applying one projection model to all thirty teams. However, we have seen first-hand over the past three years that using FIP to estimate runs allowed can give you wonky results when forecasting a team that is relies of non-strike out pitchers and a strong defense. I am sure there are other hiccups to the model(s) that might not present themselves when FanGraphs or other groups evaluate the results by looking how the projections did on a league wide basis (ie. it might average out to an acceptable variance). I get there are time constraints, integrity issues, and knowledge issues with tweaking the model for a team's particular situation but as long as that isn't done, they are opening up the possibility that their projections are going to be way off for certain teams when the issue can be fixed.

If the actual numbers missed our forecast substantially three years in a row AND I acknowledged that I knew part of the reason why, I would be expected to fix the model. FanGraphs knows (or should know) that the using FIP to calculate an estimate of runs allowed leaves out the impact of defense and that this can have a huge impact on the projections of teams built a certain way. If they want to present the most accurate projections possible (within reason), I would think they'd address that.

[SteveA]

This is like describing the best hitter in the league as "consistently bad at hitting" because he gets out 6 out of 10 times. When you don't add context, any projections are going to seem poor.

So I ask you, Fangraphs are consistently wrong compared to whom?

And earlier in the thread, when you stated pretty definitifely that:

Fangraphs and the other advanced stats projections systems are a lot more accurate than predicting every team at .500. They are also more accurate than any other prediction system.

I asked you for the basis of what seems to be a pretty strong, definitive statement. I assume there must have been some sort of study that compares predictive systems' results after the fact and that Fangraphs has proven to be the best? I asked you to point to that. Could you please provide the link?

I would be curious how much better they do in predicting team's W/L records than a variety of sources:

-- sports writers and preview magazines who do it by the seat of their pants each year

-- the Vegas over/under for win totals which is basically crowd sourcing because Vegas wants to set a line that will get even money on both sides of the over/under bet

etc.

Since you stated definitively that they ARE better, I assume some type of study must have been done of the results. Because it wouldn't be very analytical to just assume that they have performed better because you believe their methodology is better. No one of an analytical frame of mind would ever make such an assumption, they would actually base a statement like that on solid evidence.

Please provide the link to the study that is the basis of your definitive statement that the advanced stats based systems, most specifically Fangraphs, have outperformed other predictors over a reasonably large sample size of time.

[wildcard]

While the method I use to project next year stats is about as far away from what Fangraphs does as anyone could imagine, the three year average method shows a regression by the O's pitching staff as well. Several of the O's pitchers took big steps forward last year and a three year method smoothes those results.

The O's pitching staff in 2014 had a 3.43 ERA. It was 3rd in the league.

The three year average I worked out has O's with a 3.67 ERA. That would have been 7th last year.

So though I don't agree with the Fangraphs numbers the concept of a regression is supported.

[atomic]

While the method I use to project next year stats is about as far away from what Fangraphs does as anyone could imagine, the three year average method shows a regression by the O's pitching staff as well. Several of the O's pitchers took big steps forward last year and a three year method smoothes those increases.

The O's pitching staff in 2014 had a 3.43 ERA. It was 3rd in the league.

The three year average I worked out has O's with a 3.67 ERA. That would have been 7th last year.

So though I don't agree with the Fangraphs numbers the concept of a regression is supported.

You don't think the new pitching and bullpen coaches had anything to do with the changes last year?

[Frobby]

I have done quite a bit of work with financial/business plan projections in the course of my career. It would be borderline laughable to suggest that the same projection model can be applied to thirty different companies even in the same industry. Almost no two companies are alike and they all present their own unique challenges when it comes to forecasting earnings. Perhaps the thirty MLB teams are all operated similarly enough that you can get by applying one projection model to all thirty teams. However, we have seen first-hand over the past three years that using FIP to estimate runs allowed can give you wonky results when forecasting a team that is relies of non-strike out pitchers and a strong defense. I am sure there are other hiccups to the model(s) that might not present themselves when FanGraphs or other groups evaluate the results by looking how the projections did on a league wide basis (ie. it might average out to an acceptable variance). I get there are time constraints, integrity issues, and knowledge issues with tweaking the model for a team's particular situation but as long as that isn't done, they are opening up the possibility that their projections are going to be way off for certain teams when the issue can be fixed.

If the actual numbers missed our forecast substantially three years in a row AND I acknowledged that I knew part of the reason why, I would be expected to fix the model. FanGraphs knows (or should know) that the using FIP to calculate an estimate of runs allowed leaves out the impact of defense and that this can have a huge impact on the projections of teams built a certain way. If they want to present the most accurate projections possible (within reason), I would think they'd address that.

There's probably a few reasons they don't. First, it would be a lot of work. Second, they don't want to fool with it every time a position player changes (which then alters the defensive expectations).

I am sure that fans of every team always think the projections for certain players on their team are too negative. I've done my own OPS projections for Oriole players each of the last 6 years or so and compared them to the various systems (but using my projections about playing time). I can't say I've been particularly better or worse. Last year Steamer said .744 and I said .756. Actual was .735, so Steamer beat me. Of course, I didn't know the league OPS was going to drop by 19 points, and neither did Steamer.

[Phantom]

They seem to be consistently wrong on the O's, and as another poster pointed out, just about all their team projections from last year were way off the mark. I don't know how they are in comparison to other projection systems, but I just find it funny that they don't ever consider that maybe their model is flawed. If the projections are off, it's always an "outlier", which makes their projections right no matter what.

I don't know where to begin. Please define "consistently wrong", and show me the prediction system that is in comparison "consistently right". Are you basing this off of how they have projected one teams wins over the last three years?

[AlbionHero]

I think regression should be expected. I don't think even the most optimistic of us could say we have one of the best pitching rotations in baseball.

But its just a matter of how much regression there should be. 3.65-3.70 ERA sounds pretty realistic to me.