HHP: Hard Data on Ball/Strike Calls - How Good/Bad are the Umpires

[sangar]

After thinking about this a bit more, I think it brings up some significant issues. Most businessses will implement and incur the cost of new technology to gain productivity, but what real incentive does MLB have here? They don't make more money by being more accurate. In fact, the cost of equipment and labor (operators/technicians) could be fairly significant. Theoretically you'd probably have teams of operators and technicians in addition to the umpire crews.

In any industry you'll find that quality control has a negative impact on cost per item, and a corresponding increase in customer satisfaction. You will also find that customer satisfaction is the #1 revenue driver.

[CA-ORIOLE]

In any industry you'll find that quality control has a negative impact on cost per item, and a corresponding increase in customer satisfaction. You will also find that customer satisfaction is the #1 revenue driver.

I agree with this in general, but I'm not sure of the relavance here. This has been going on for a long, long time and I'm not sure that many people have really voiced any significant displeasure with it, or are even cognizant of it for that matter. In many ways fans are purists and accepting of this kind of stuff....to some level. Now it may be that some of the calls have been so bad this season and it's become more to the forefront.

[sangar]

I agree with this in general, but I'm not sure of the relavance here. This has been going on for a long, long time and I'm not sure that many people have really voiced any significant displeasure with it, or are even cognizant of it for that matter. In many ways fans are purists and accepting of this kind of stuff....to some level. Now it may be that some of the calls have been so bad this season and it's become more to the forefront.

From what I can remember "Kill the umpire" has been a customary mantra for a long time. People may like to know that the calls are actually fair and accurate.

[CA-ORIOLE]

From what I can remember "Kill the umpire" has been a customary mantra for a long time. People may like to know that the calls are actually fair and accurate.

That may be, but I bet if you took a poll of all fans in the year 2012 more than not would still prefer an human umpire over a machine to call balls and strikes.

[sangar]

That may be, but I bet if you took a poll of all fans in the year 2012 more than not would still prefer an human umpire over a machine to call balls and strikes.

Haha, you're probably right.
I think as the casual fan becomes more exposed to the available data, and they realize that their long-held disdain for the "Human element" is indeed correct and having a significant impact on the outcome of games, technological additions to the national pastime will become inevitable.

[Lucky Jim]

Haha, you're probably right.
I think as the casual fan becomes more exposed to the available data, and they realize that their long-held disdain for the "Human element" is indeed correct and having a significant impact on the outcome of games, technological additions to the national pastime will become inevitable.

People resist and adapt. It's not a big deal, I don't think.

[skanar]

April 25 vs Blue Jays
Jason Hammel: 0 gifts, 7 squeezes (-7)
Kyle Drabek: 8 gifts, 0 squeezes (+8)

Game score: -15.

Wow. 2nd-worst umpired game of the season.

[Lucky Jim]

April 25 vs Blue Jays
Jason Hammel: 0 gifts, 7 squeezes (-7)
Kyle Drabek: 8 gifts, 0 squeezes (+8)

Game score: -15.

Wow. 2nd-worst umpired game of the season.

It looked pretty bad while watching, but it's amazing how having filthy stuff can make up for that. Throw in a few errors and a botched call at first, and we can say, if nothing else, that Hammel sure knows how to overcome adversity.

[allquixotic]

Been done for 100 years.

See my point re: tennis. I understand the strategy of using a weakness in the game to your advantage, that doesn't mean that baseball shouldn't alleviate that weakness.

The fact is, we're dealing with really shoddy technology. And when better technology exists, shoddiness should not be countenanced.

Exactly. Bring on the "post-modern era" of baseball, or whatever you want to call it. If we do that, the eras can almost be divided cleanly among century borders: 1800's, 1900's and 2010's (assuming that we actually do move to computerized ball/strike calls in this decade). It only makes sense that something which has so fundamentally changed our society (and indeed, something I've built my profession around) would come to impact one of our most traditional and conservative (resistant to change) sports, even if the change does come a decade or two late.

You pretty much have to call it a new era because, each time you change the rules of the game or the way the game is judged (such as replacing subjectivity with objectivity or vice versa, even if the rules on paper don't change), it's difficult to compare stats from before that change to stats after that change.

For instance, imagine the 2012 Orioles with a Bad% of Zero in every single game. How many games that we won, would we have lost? How many games that we lost, would we have won? It's hard to say. Everything changes. Even the pitcher's approach to the game changes. And as we've already established that "control pitchers" -- those who can absolutely paint the catcher's mitt with the baseball as if they shot a laser at it -- get very favorable calls due to "framing bias". You take away framing bias, and pitchers that rely heavily on it, such as Mo Rivera, have to reinvent themselves. Pitchers that held it as a weakness, such as Arrieta and Matusz, start to look good.

Just to restate the point that you said in my own interpretation, I think that, at face value, we needn't interpret the PitchFX Umpire as inherently better for the Orioles than human umpires. After all, we could have a fire sale at the end of this year and pick up a staff full of starters with astounding control who would benefit hugely from framing bias on the corners of the strike zone. What we should interpret the PitchFX Umpire as offering, is, a fairer game. And I think we all like fair... everyone except the Yankees

Anyway, rep handed out appropriately... a lot of brain power in this thread. If any of you know some of the folks who can "get things done" in the central MLB governance and administration silos, please, please ping them and ask them if they can look into this.

[callahan09]

This may seem stupid, but I was wondering how the squeeze/gift tally is being researched? Are you just looking at the PitchFX graphs and doing it by eye, looking for the appropriately colored markings in position relative to the strike zone box? I've had difficulty doing this myself, because sometimes the markings overlap, or you can't tell which color it is (maybe I'm colorblind?). If you are doing this, which map do you use? Normalized? Non-normalized? Do you look at versus left-handed batters and right-handed batters and add the tally up, or do you look at the "all batters" map? Do you go by looking at every at-bat individually (seems like an obscene amount of work, so I doubt it)? Is there some way to make the determination other than using the maps? Do you use the spreadsheet output and compare each pitch's vector position with the vectors for the strikezone (maybe math gurus have worked out the formula for this? It would seem the most efficient way to make the determination, if you've got the formula right)?

What I'm curious about, is coming up with some way to record the gift/squeeze tally for every pitcher, for every game, for every team, for every umpire, etc. so that we can see useful information like who's getting hurt/benefit from the calls both on a per team basis and a per player basis. Going truly overboard, I'd be curious to do it at an at-bat level so we can see how it effects individual batters, though this may be overkill, I don't know.

Seem to me that with all the baseball data being combed through and analyzed every day, it's odd that it's so tough to find comprehensive data on ball/strike calls relative to what we're calling the "objective strikezone".

For the time being, I'd just be satisfied to see the Orioles gift/squeeze differential for each game, laid right out in a nice table, and then of course the tally for the season-so-far.

Anybody who might be inclined to do work on this or reveal some answers, if you need help let me know. I'm pretty good with researching and with numbers and math, so I'm not just going to throw it out there that I want to see this data then rest on my laurels until it's handed to me on a silver platter (unless it's all done already and I just haven't found it yet). I just don't know where the start, and I don't think it's a task for a single person.

[srock]

This may seem stupid, but I was wondering how the squeeze/gift tally is being researched? Are you just looking at the PitchFX graphs and doing it by eye, looking for the appropriately colored markings in position relative to the strike zone box? I've had difficulty doing this myself, because sometimes the markings overlap, or you can't tell which color it is (maybe I'm colorblind?). If you are doing this, which map do you use? Normalized? Non-normalized? Do you look at versus left-handed batters and right-handed batters and add the tally up, or do you look at the "all batters" map? Do you go by looking at every at-bat individually (seems like an obscene amount of work, so I doubt it)? Is there some way to make the determination other than using the maps? Do you use the spreadsheet output and compare each pitch's vector position with the vectors for the strikezone (maybe math gurus have worked out the formula for this? It would seem the most efficient way to make the determination, if you've got the formula right)?

What I'm curious about, is coming up with some way to record the gift/squeeze tally for every pitcher, for every game, for every team, for every umpire, etc. so that we can see useful information like who's getting hurt/benefit from the calls both on a per team basis and a per player basis. Going truly overboard, I'd be curious to do it at an at-bat level so we can see how it effects individual batters, though this may be overkill, I don't know.

Seem to me that with all the baseball data being combed through and analyzed every day, it's odd that it's so tough to find comprehensive data on ball/strike calls relative to what we're calling the "objective strikezone".

For the time being, I'd just be satisfied to see the Orioles gift/squeeze differential for each game, laid right out in a nice table, and then of course the tally for the season-so-far.

Anybody who might be inclined to do work on this or reveal some answers, if you need help let me know. I'm pretty good with researching and with numbers and math, so I'm not just going to throw it out there that I want to see this data then rest on my laurels until it's handed to me on a silver platter (unless it's all done already and I just haven't found it yet). I just don't know where the start, and I don't think it's a task for a single person.

Need access to a pitch f/x database. Then it is quite doable, but it would take some time. People who put the time into importing pitch f/x into a custom DB seem to be pretty protective when it comes to access. I have not found a public facing data source other then the flat files provided by MLB. Parsing those files everyday is also doable, but requires more programming then having the data in a friendly database.

So yes this can be done. If you want to pay me enough so I can quite my job, I would code it up for you

[skanar]

This may seem stupid, but I was wondering how the squeeze/gift tally is being researched? Are you just looking at the PitchFX graphs and doing it by eye, looking for the appropriately colored markings in position relative to the strike zone box? I've had difficulty doing this myself, because sometimes the markings overlap, or you can't tell which color it is (maybe I'm colorblind?).

I'm doing a very simplistic analysis, taking the pitcher's PitchFX "Strikezone Plot" for the entire game and counting (total balls, total called strikes, gifts, and squeezes) by eye. I believe that the data for this plot is corrected for each batter's handedness, height, etc. before being graphed. I haven't had a problem with overlapping pitches; it's never been more than one or two pitches each game, and usually I can see the color of the rear pitch peeking out. Color has not been an issue either.

Here's an example:


It would certainly be better to use the hard data and to work with a full dataset, rather than just looking at Orioles games this year. There's a lot of potential here. I just don't have the time to work on such a in-depth project; baseball stats is what I do to avoid actual work. Feel free to do your own analysis - I'd be excited to read what you come up with!

You can find my initial table with the team gift/squeeze differential for each game in the OP. I'll make a more detailed one for the entire month of April once all the games have been played.

[callahan09]

Need access to a pitch f/x database. Then it is quite doable, but it would take some time. People who put the time into importing pitch f/x into a custom DB seem to be pretty protective when it comes to access. I have not found a public facing data source other then the flat files provided by MLB. Parsing those files everyday is also doable, but requires more programming then having the data in a friendly database.

So yes this can be done. If you want to pay me enough so I can quite my job, I would code it up for you

How can I get this MLB flat file? I'm not familiar with that...

[callahan09]

I'm doing a very simplistic analysis, taking the pitcher's PitchFX "Strikezone Plot" for the entire game and counting (total balls, total called strikes, gifts, and squeezes) by eye. I believe that the data for this plot is corrected for each batter's handedness, height, etc. before being graphed. I haven't had a problem with overlapping pitches; it's never been more than one or two pitches each game, and usually I can see the color of the rear pitch peeking out. Color has not been an issue either.

Here's an example:


It would certainly be better to use the hard data and to work with a full dataset, rather than just looking at Orioles games this year. There's a lot of potential here. I just don't have the time to work on such a in-depth project; baseball stats is what I do to avoid actual work. Feel free to do your own analysis - I'd be excited to read what you come up with!

You can find my initial table with the team gift/squeeze differential for each game in the OP. I'll make a more detailed one for the entire month of April once all the games have been played.

Thanks for the explanation! I would need to come up with a more foolproof way of doing the analysis because like I said my eyes don't see the colors well enough to differentiate called strikes, bunts, and foul bunts from one another, and balls and in-play no outs confound me as well. I guess I'd have to look at the vector data for the strike zone and the pitch location and determine if the pitch is in or out of the strike zone if I wanted to do my own analyses on this subject.

Edit: In addition, which criteria are you using for the strike zone... must the ball be located entirely inside the box, or a majority of the ball inside the box, or just any piece at all inside the box? A lot of pitches overlap the line, so I was wondering how you are tallying those. My philosophy is that if any portion of the ball overlaps any portion of the strikezone box, it's a strike, but some disagree. Does the rulebook mention this specifically? My memory is foggy.

[skanar]

Edit: In addition, which criteria are you using for the strike zone... must the ball be located entirely inside the box, or a majority of the ball inside the box, or just any piece at all inside the box? A lot of pitches overlap the line, so I was wondering how you are tallying those. My philosophy is that if any portion of the ball overlaps any portion of the strikezone box, it's a strike, but some disagree. Does the rulebook mention this specifically? My memory is foggy.

I'm looking at the dead center of the pitch box vs the line. This is the same as looking at the majority of the box. If the pitch is exactly evenly split by the line (this has happened maybe 2 or 3 times in 20 games), I've been giving the umpire the benefit of the doubt.

I don't think worrying about what the rulebook says is all that relevant; honestly, the analysis is just not precise enough to worry about this sort of fine distinction. It's important to remember that, as a result, differences of +2 or +3 pitches are almost certainly too small to really mean anything. CA-ORIOLE was suggesting that splitting things up into zones and then counting pitches in those zones would be better than relying on the PitchFX strikezone, and I agree, but initially I wanted to get through the O's first 15 games or so quickly to get some (preliminary, approximate) results.