HHP: Hard Data on Ball/Strike Calls - How Good/Bad are the Umpires

[osfan83]

The closest analog in my opinion is tennis: would anyone claim that it's okay for line-judges to simply give Federer three extra inches because he's proven he can locate? Of course not. Because a tennis court is an objective map.

Everything CA-O says is true (his diagnosis is 100% apt, and I agree it's not "cheating," or likely "conscious") but the fact that umps are using such shoddy heuristics for what is supposed to be an objective map is a serious problem. If the issue is ump location, then that needs to change. If the issue is ump skill, then that needs to change. If the issue is a kind of bounded rationality (i.e., assumptions making up for what they can't catch) then that needs to change.

Great analogy! How about the three point line in basketball..should should Steve Novak of the Knicks be givin threes when his foot is clearly over the line by 2 inches just because he has the best 3 point shooting percentage in the league this year? Should Tom Brady be allowed to throw a forward pass after scrambling and passing the line of scrimmage by a yard?

[FuManchu]

http://www.wired.com/playbook/2011/0...-bias-umpires/

[srock]

I just did it manually: looked at the pitch chart for each game and counted balls and called strikes in and out of the PitchF/X strike zone. That's one reason the analysis is so rudimentary; I'm just making a quick in/out decision for each pitch and counting them up.

Brooks Baseball does provide data from each pitcher/game as a table, including (I think) horizontal and vertical distances, but I don't know if you can make calls on their database.

Some interesting things at Brooks Baseball. The Strike Zone Map tool is generating the images charting pitches. But the response does not contain the raw data. The images are generated server side. So its good for eyeballing the data but not for analyzing with a script.

The Pitch F/X tool is quite interesting. Take a look at the data generated for Chen yesterday.

Now that is computable data! But not for balls/strikes, that is still in graphical format.

Clearly, Brooks Baseball has pitch f/x data loaded into some sort of database they can write apps against. Which is awesome. But they don't appear to expose the data.

I just joined the forum there, maybe it contains more info on how to get out the data.

Edit: Another thought. Because the data is parameterized by date and a specific game, they might be hitting the shared data on MLB directly and just processing the XML files. This means no database. And explains why generating the images is a bit slow.

[srock]

http://www.wired.com/playbook/2011/0...-bias-umpires/

Excellent article:

umpires were correct only 85.6 percent of the time, which the authors contend is not that bad, “considering that the average pitch starts out at 92 mph, crosses the plate at more than 85 mph, and usually has been garnished with all sorts of spin and movement.”

when you drill down and look at called pitches that were around the corners of home plate, umpires only correctly called 49.9 percent of pitches that were strikes according to PitchFX

[DrungoHazewood]

Excellent article:

when you drill down and look at called pitches that were around the corners of home plate, umpires only correctly called 49.9 percent of pitches that were strikes according to PitchFX

I don't think it's particularly surprising that umps are basically guessing around the margins. What's good is that we finally have the tools to do something about it, and we no longer have to accept random guesses as a proxy for the ground truth.

[SrMeowMeow]

See my point re: tennis. I understand the strategy of using a weakness in the game to your advantage, that doesn't mean that baseball shouldn't alleviate that weakness.

The fact is, we're dealing with really shoddy technology. And when better technology exists, shoddiness should not be countenanced.

I have no objection to using better ball/strike tech. But our analysis of a situation should be within existing conditions, and the strike zone in baseball right now is very much not objective. It'd be like trying to end a debate on the subtleties of race relations in America by saying that the Constitution guarantees equal rights.

[crawjo]

Just a process question that might have been asked earlier...how do we know that the strike zone as shown on TV and in these spray charts is accurate? Shouldn't the height of the strike zone change from batter to batter?

[skanar]

Just a process question that might have been asked earlier...how do we know that the strike zone as shown on TV and in these spray charts is accurate? Shouldn't the height of the strike zone change from batter to batter?

The short answer is: we don't.

Brooks Baseball does attempt to correct for the height of each batter's individual strike zone (they store a top height value and bottom height value for each hitter), but they say that their methods for doing so are limited.

[Can_of_corn]

Just a process question that might have been asked earlier...how do we know that the strike zone as shown on TV and in these spray charts is accurate? Shouldn't the height of the strike zone change from batter to batter?

The short answer is: we don't.

Brooks Baseball does attempt to correct for the height of each batter's individual strike zone (they store a top height value and bottom height value for each hitter), but they say that their methods for doing so are limited.

Of course a lot of the problems are with the width not the height on the zone.

[Lucky Jim]

I have no objection to using better ball/strike tech. But our analysis of a situation should be within existing conditions, and the strike zone in baseball right now is very much not objective. It'd be like trying to end a debate on the subtleties of race relations in America by saying that the Constitution guarantees equal rights.

I don't buy this at all. It's wildly inapt, frankly.

The fact that our technology is so shoddy that it commits enormous, repeated rounding errors doesn't mess with the objectivity of what it's trying to interpret.

[CA-ORIOLE]

Of course a lot of the problems are with the width not the height on the zone.

That's true imo as well, but from the few charts I have seen it does appear we are not getting a lot of low/high calls. Though I'm not convinced we are not getting a relative percentage of these types of calls. We just appear to have more disputable balls in the upper/lower quadrants. In that sense, crawjo's point is a pretty good one.

[SrMeowMeow]

I don't buy this at all. It's wildly inapt, frankly.

The fact that our technology is so shoddy that it commits enormous, repeated rounding errors doesn't mess with the objectivity of what it's trying to interpret.

It's not like someone broke the pitch meter this season. The "shoddy technology" has been part of the game for over a hundred years. I agree that we should take steps to improve it (although there are legitimate arguments against doing so). But pitchers have succeeded and failed within that paradigm since forever. And if we're getting "cheated" out of calls and strikes not because of some unconscious umpire bias but because, say, our pitchers don't know how to work within a strike zone that may change umpire to umpire, or because they haven't learned the skill of expanding the strike zone, or because the catcher can't frame pitches as well as other teams, then we're just doing things poorly. We're certainly not getting "cheated" by crooked umpires.

If every car's steering in a NASCAR event starts pulling to the right, you hold the wheel to the left on the straightaways, not slam into the right-hand wall while complaining that things need to be fixed.

You master the game you're playing, flawed or not.

[Can_of_corn]

It's not like someone broke the pitch meter this season. The "shoddy technology" has been part of the game for over a hundred years. I agree that we should take steps to improve it (although there are legitimate arguments against doing so). But pitchers have succeeded and failed within that paradigm since forever. And if we're getting "cheated" out of calls and strikes not because of some unconscious umpire bias but because, say, our pitchers don't know how to work within a strike zone that may change umpire to umpire, or because they haven't learned the skill of expanding the strike zone, or because the catcher can't frame pitches as well as other teams, then we're just doing things poorly. We're certainly not getting "cheated" by crooked umpires.

If every car's steering in a NASCAR event starts pulling to the right, you hold the wheel to the left on the straightaways, not slam into the right-hand wall while complaining that things need to be fixed.

You master the game you're playing, flawed or not.

But if your opponents steering is fine and some races yours pulls a bit to the right, or a bit to the left, or hard to the right. Then sometimes it starts off fine but the last few laps it starts pulling. Then by the time you can adjust the other car is past you.

[Lucky Jim]

We're certainly not getting "cheated" by crooked umpires..

Please identify one instance where I refered to being cheated or anyone being crooked.

[SrMeowMeow]

Please identify one instance where I refered to being cheated or anyone being crooked.

You're running a little hot, baby.

I'm responding, not just to you, but to one general argument: if a) the strike zone is an objective thing, and b) we're getting more missed calls than other teams, then it's logical to conclude that our pitchers are getting "cheated" out of some good pitches because of flaws in the system. I'm saying that there's another possibility - that the strike zone is not objective and that we're doing a poor job of working within the confines of a flawed system.

And that's really the one and only line you choose to respond to?