Defense?

[Frobby]

I have sent Michael Lichtman some specific questions about park effects and UZR. In the meantime, I will leave you with the following quotes:

In the OF, each section, LF, CF, and RF, is divided into two zones, shallow and deep, for park adjustment purposes. Each of those 6 zones per park has their own adjustment factor. For example, the deep zone in LF at Fenway has an adjustment factor of .5, meaning that of all balls hit past a certain distance in LF at Fenway, the overall “catch rate” is only half that of the average major league park. Similarly, in Houston’s LF “short porch,” it is .86. In Seattle, fly balls in all sections of all fields are easier to catch than at an average MLB park, presumably because of the altitude, the cold weather, and the large but not too-large outfield dimensions, and thus have a park factor above 1.0.

* * *

Park factors are very difficult to do when constructing defensive metrics. They are much more difficult than offensive park factors, and those are no walk in the park either. The saving grace is that most parks do not have major defensive park factors. In any case, I think that the current version of UZR handles those factors pretty well, even in parks like Fenway and Coors. As well, Fangraphs now has home/road UZR splits so if you still don’t trust a certain player’s UZR because of the park factors issue, you can check out his road numbers.

* * *

[Jason Bay] had terrible UZR’s in Pittsburgh in 2007 and 2008, prior to his trade to Boston, so it should not have been surprising that he would have bad numbers in Boston as well, even without any park adjustment problems. Now, whether he is indeed a terrible, bad, slightly below average, or average (or even above average) fielder, is another story. Just because UZR, or any other defensive metric “says” that someone is X, even if that X is based on many years of data, does not make it so. When you are dealing with sample data, as we almost always are with every metric in baseball that we encounter, there is a certain chance that the metric is going to be “wrong.” Sometimes, you can use other information (such as scouting and observation, or physical attributes like size and speed) in order to adjust your “conclusions” and decrease your chance of being “wrong” and sometimes you can’t (because the requisite information is not available).

* * *

A player’s UZR does not necessarily tell you how he actually played just as it does not necessarily tell you what his true talent is. That is a very important point.

* * *

Now, that being said, there is still a potentially large gap between what you might see on the field if you were to watch every play of every game and what UZR “says” happened on the field. And that is one of several reasons why one year or even 10 years of UZR (or any other sample metric) does not give us a perfect estimate of a player’s true talent or even an accurate picture of what actually happened on the field. The reason for that is that the data is imperfect. For example, UZR might put a certain batted ball in a certain bucket and determine that that batted ball was extremely difficult for the CF’er to catch, based on the recorded (by the BIS “stringers”) qualities of the ball and other data. We don’t, of course, know for sure whether it was indeed a difficult to field batted ball. We don’t know exactly where each fielder was stationed, we certainly don’t know the exact location of the batted ball to the nearest square inch on the field, and we definitely don’t know how long the ball was in the air or on the ground. In reality, it might have been an easy ball to catch or it might have been a difficult one to catch, or somewhere in between. We can only hope that in the long run, those balls were indeed hard to catch, on the average, for each individual player.

http://www.fangraphs.com/blogs/index.php/the-fangraphs-uzr-primer/#9

[Lucky Jim]

The issue here is the accuracy of Nick and AJ's defensive metrics. Fans of these players point to their home way splits as evidence to doubt the validity of all defensive metrics, which I find absurd. In the absence of any hard data showing negative park effects, I see no reason to doubt their defensive metrics. I don't need to answer your surmise with my own, in order to proove these numbers accurate. I offered a possible explanation for Nick's difficulties. It doesn't matter what the reasons could be (I'm pretty sure it's not the grass) the fact is, Marakais is a below average RF.

This sounds like the certainty of someone who knows just enough to fool himself into certainty.

[El Gordo]

This sounds like the certainty of someone who knows just enough to fool himself into certainty.

I am not certain of anything except that I have seen no hard data to indicate that there are negative park factors effecting Markakis' and Jones' defensive numbers. My eyes tell me that they have flaws in their game, and their numbers supoport that notion. Until some one can demonstrate other wise, I trust the combination of the two. Perhaps I should have said, the fact is, as far as any thing can be a fact when measuring defense, Markakis is a below average RF.

[primetime]

I am not certain of anything except that I have seen no hard data to indicate that there are negative park factors effecting Markakis' and Jones' defensive numbers. My eyes tell me that they have flaws in their game, and their numbers supoport that notion. Until some one can demonstrate other wise, I trust the combination of the two. Perhaps I should have said, the fact is, as far as any thing can be a fact when measuring defense, Markakis is a below average RF.

Did your eyes tell you this before or after you saw their defensive metrics?

[CA-ORIOLE]

I have sent Michael Lichtman some specific questions about park effects and UZR. In the meantime, I will leave you with the following quotes:

http://www.fangraphs.com/blogs/index.php/the-fangraphs-uzr-primer/#9

Thanks, I found this part pretty interesting. Don't recall reading it before. But it gives some insight how UZR approaches this over FB.

In the OF, each section, LF, CF, and RF, is divided into two zones, shallow and deep, for park adjustment purposes. Each of those 6 zones per park has their own adjustment factor. For example, the deep zone in LF at Fenway has an adjustment factor of .5, meaning that of all balls hit past a certain distance in LF at Fenway, the overall “catch rate” is only half that of the average major league park. Similarly, in Houston’s LF “short porch,” it is .86. In Seattle, fly balls in all sections of all fields are easier to catch than at an average MLB park, presumably because of the altitude, the cold weather, and the large but not too-large outfield dimensions, and thus have a park factor above 1.0.

[El Gordo]

Did your eyes tell you this before or after you saw their defensive metrics?

I remember being impressed with Markakis' defense in 2008, especially his arm. About that same time I became interested in the Fielding Bible and their methods for measuring defense. That year Markakis got a number of first place votes as a top defensive RF in the Fielding Bible awards. I was looking forward towards his continued progress, but the next season I noticed Markakis seemed to be slower, particulrly in the field. His SB numbers had dropped off from his 18 in 2007 as well. I was looking for them to increase. So in 2009 I started tracking Nick's FB numbers on the Bill James site and was shocked to see how far he had fallen off from one year to the next. I kept waiting for him to rebound but his numbers continued to be below average with each year and have remained so. As I continue to watch him it seems to me he has difficulty coming in on balls, running them down in the gap, and down the line. At the same time his arm which I have always thought was strong and accurate has remained that way, and is reflected in the numbers.

[Hallas]

It does appear to reward Nick (and Jones) for limiting advance (arm component) though.

Arm component applies to baserunners only. If you limit a ball off the wall to a single, you get no credit, positive or negative. If you prevent a runner from advancing to 3rd on a single, or scoring on a double, you get credit. This limits the amount of reward given to Nick and Jones. I also don't know how much credit it gives to Nick and Jones for hitters being dumb and trying to stretch singles into doubles; one report said that those were disregarded, but those seem to be a high percentage of Nick's (and Jones') assists. If you have info on this, feel free to enlighten me.

[CA-ORIOLE]

It does appear to reward Nick (and Jones) for limiting advance (arm component) though.

Arm component applies to baserunners only. If you limit a ball off the wall to a single, you get no credit, positive or negative. If you prevent a runner from advancing to 3rd on a single, or scoring on a double, you get credit. This limits the amount of reward given to Nick and Jones. I also don't know how much credit it gives to Nick and Jones for hitters being dumb and trying to stretch singles into doubles; one report said that those were disregarded, but those seem to be a high percentage of Nick's (and Jones') assists. If you have info on this, feel free to enlighten me.

No I actually don't. You sound more informed than me on some of this. Giving no credit for balls of the wall (limiting to a single) intuitively makes sense to me though. If not for the wall, they'd be doubles anyways. No credit could imply a reward in that sense. Of course it's more complicated than that. I just think it's very hard problem to overcome.

[Lucky Jim]

No I actually don't. You sound more informed than me on some of this. Giving no credit for balls of the wall (limiting to a single) intuitively makes sense to me though. If not for the wall, they'd be doubles anyways. No credit could imply a reward in that sense. Of course it's more complicated than that. I just think it's very hard problem to overcome.

Or outs.

[primetime]

I remember being impressed with Markakis' defense in 2008, especially his arm. About that same time I became interested in the Fielding Bible and their methods for measuring defense. That year Markakis got a number of first place votes as a top defensive RF in the Fielding Bible awards. I was looking forward towards his continued progress, but the next season I noticed Markakis seemed to be slower, particulrly in the field. His SB numbers had dropped off from his 18 in 2007 as well. I was looking for them to increase. So in 2009 I started tracking Nick's FB numbers on the Bill James site and was shocked to see how far he had fallen off from one year to the next. I kept waiting for him to rebound but his numbers continued to be below average with each year and have remained so. As I continue to watch him it seems to me he has difficulty coming in on balls, running them down in the gap, and down the line. At the same time his arm which I have always thought was strong and accurate has remained that way, and is reflected in the numbers.

I'll readily admit to being a novice when it comes to much of the advanced metrics. I've learned a great deal from the hangout but there's a ton that I don't grasp yet.

I thought Frobby's post about UZR on this page was pretty interesting. What I got from that post was that UZR is something to look at but we don't even know if the data is worthwhile or reliable, but it might be. Perhaps with the advent and improvement of Hit F/X, that data can be used and applied to defensive metrics.

Having watched a lot of baseball in my 33 years and a ton of Orioles baseball at that, I have a really, really hard time buying into the notion that any metric could show that Jay Gibbons has ever had 1 season better in RF than even Nick Markakis' worst season in RF.

One more thing, when it comes to home/road splits on UZR, does Fangraphs break down the road numbers by individual park and then provide a cummulative number for a player's entire season on the road? Something like that could explain a difference in home/road splits because the most road games that any player will have in an opposing ball park in any given season is 9. Based on what some posters are saying about the reliability of defensive metrics in regards to sample size could be explained by small sample size.

Again though, I'm just scratching the surface of the advanced metrics so I'll defer to some of the more knowledgable posters on the matter.

[Hallas]

No I actually don't. You sound more informed than me on some of this. Giving no credit for balls of the wall (limiting to a single) intuitively makes sense to me though. If not for the wall, they'd be doubles anyways. No credit could imply a reward in that sense. Of course it's more complicated than that. I just think it's very hard problem to overcome.

I think you can divide the wall into zones the same way you divide the field up into zones, and assign a run value based on the trajectory and the average result of all wall balls in that zone. That way you can give (or take) credit based on the result of the play and the expected result.

Not terribly easy, but doable, and it would probably lead to more accurate results in stadiums such as Fenway and OPACY.

[El Gordo]

Arm component applies to baserunners only. If you limit a ball off the wall to a single, you get no credit, positive or negative. If you prevent a runner from advancing to 3rd on a single, or scoring on a double, you get credit. This limits the amount of reward given to Nick and Jones. I also don't know how much credit it gives to Nick and Jones for hitters being dumb and trying to stretch singles into doubles; one report said that those were disregarded, but those seem to be a high percentage of Nick's (and Jones') assists. If you have info on this, feel free to enlighten me.

The FB gives a % of runners with what they percieve as an opportunity to advance, who do. Less than .500 is good. I don't think they disclude runners from 1B.

[Frobby]

I'll readily admit to being a novice when it comes to much of the advanced metrics. I've learned a great deal from the hangout but there's a ton that I don't grasp yet.

I thought Frobby's post about UZR on this page was pretty interesting. What I got from that post was that UZR is something to look at but we don't even know if the data is worthwhile or reliable, but it might be. Perhaps with the advent and improvement of Hit F/X, that data can be used and applied to defensive metrics.

Having watched a lot of baseball in my 33 years and a ton of Orioles baseball at that, I have a really, really hard time buying into the notion that any metric could show that Jay Gibbons has ever had 1 season better in RF than even Nick Markakis' worst season in RF.

One more thing, when it comes to home/road splits on UZR, does Fangraphs break down the road numbers by individual park and then provide a cummulative number for a player's entire season on the road? Something like that could explain a difference in home/road splits because the most road games that any player will have in an opposing ball park in any given season is 9. Based on what some posters are saying about the reliability of defensive metrics in regards to sample size could be explained by small sample size.

Again though, I'm just scratching the surface of the advanced metrics so I'll defer to some of the more knowledgable posters on the matter.

Like you, I'm hoping that Hit f/x will lead to some improvements in the defensive metrics that lead me to feel more comfortable with them. As to your question about ballpark splits, I have posed a similar question to Michael Lichtman (creator of UZR), and if he answers me I will post it here.

[Hallas]

The FB gives a % of runners with what they percieve as an opportunity to advance, who do. Less than .500 is good. I don't think they disclude runners from 1B.

"exclude"

I'm almost certain that they exclude the batter-runner.

Another area of concern is if balls that drop in a particular "zone" have specific park effects: for example, if a ball in deep RF that falls in for a hit has a higher expected run value overall than it does in a particular stadium. They only park-adjust based the number of plays made/not made, not the expected run values for the various situations. I would imagine that a ball down the line in RF has a higher run value in OPACY due to the nook.

edit: Also, while they exclude wall balls, that could negatively affect the park's handling because those balls might produce a generally positive fielder run expectancy at other parks.

[CA-ORIOLE]

I think you can divide the wall into zones the same way you divide the field up into zones, and assign a run value based on the trajectory and the average result of all wall balls in that zone. That way you can give (or take) credit based on the result of the play and the expected result.

Not terribly easy, but doable, and it would probably lead to more accurate results in stadiums such as Fenway and OPACY.

Yeah, that would be the next logical extension, but I see a danger in making too many adjustments and ruining the integrity of the data. Sounds to me (as I suspected) that UZR has a more comprehensive data/analytical approach and FB a simpler one on this wall issue. I don't think FB adjusts at all for IF defense/groundballs and believe UZR does.