HHP: Hard Data on Ball/Strike Calls - How Good/Bad are the Umpires

[square634]

If the computer system were imperfect, but better than a human umpire, would we call it "subjective," or simply the poorly-implemented arbiter of an objective strike zone?

What do you think?

Would the computer be consistently imperfect in the same systemic way, or would it be influenced by externalities, such as a catcher's frame of the pitch, the pitcher's general wildness, etc.?

I would suggest that anything that is subject to human interpretation is tautologically subjective. For example, I would say that the words in our country's laws are "objective", while their actual practical interpretation and implementation are "subjective." I would similarly say that the physical dimensions of the strike zone are objectively stated, but the implementation and interpretation of the strike zone by umpires is subjective.

[CA-ORIOLE]

Would the computer be consistently imperfect in the same systemic way, or would it be influenced by externalities, such as a catcher's frame of the pitch, the pitcher's general wildness, etc.?

It may not be perfect, but it would obviously not be influenced by framing and command/wildness.

[Lucky Jim]

Would the computer be consistently imperfect in the same systemic way, or would it be influenced by externalities, such as a catcher's frame of the pitch, the pitcher's general wildness, etc.?

I would suggest that anything that is subject to human interpretation is tautologically subjective. For example, I would say that the words in our country's laws are "objective", while their actual practical interpretation and implementation are "subjective."

Laws are a a fundamentally terrible example, because words are, also fundamentally, "heuristics" which function only at a single point of agreement between speaker and audience - a single point, for both speaker and audience, that requires that they exclude all other possible definitions (as well personal information informing definitions). Much of this work is easily done by context, but not all. That's why being a judge is difficult. Some laws are easier than others, but they exist on a scale.

On the other hand, if I commit a crime for which there is a $25,000 fine, and a judge checks his cheat-sheet and misreads it instead as $2,500 and fines me 1/10 of my due, that's not the law being subjective, even though it's "subject to human interpretation" (myopia is a form/limitation of interpretation). It's bad technology (human judge w/ human eyes) offering a flawed interpretation of an objective rule.

In a sports context, by your definition, because a human in an umpire chair called balls "in" or "out" in a tennis match prior to (whenever), the tennis court, though exactly measured and painted, was "subjective."

[Lucky Jim]

It may not be perfect, but it would obviously not be influenced by framing and command/wildness.

Or probability/confirmation bias, which I also think plays a part. I.e., good pitchers get calls because good pitches are expected. It's understandable that our expectations would back-fill what we didn't see but are charged with interpreting.

[srock]

It may not be perfect, but it would obviously not be influenced by framing and command/wildness.

More difficult then accuracy, is reliability and objectivity. By objectivity I mean, the hometeam can't squeeze the strikezone a bit when they are at bat!

This would require complete buy in form the umpires. They are the 3rd party arbitrator trusted with calling a fair game. Therefore, they would have to be in charge of calibrating and verifying the pitch f/x data.

And this system would not replace the behind the plate umpire. Instead, it would supplement. Put a buzzer or something on the umpire so when pitch f/x registers a strike it buzzes.

Issues:

• Will Umpires buy in? Money should keep them happy.
• Is the system reliable enough?
• Consistent 1 second lag time needed for the system to register a strike and inform the ump.
• Fault tolerance: Umpire must be sure system is working on every pitch.
• Security - can calls be tampered with (hackers, hometeam rigging strikezone)?
• High/low: Need definitive way to determine high and low for each batter.

[square634]

Laws are a a fundamentally terrible example, because words are, also fundamentally, "heuristics" which function only at a single point of agreement between speaker and audience - a single point, for both speaker and audience, that requires that they exclude all other possible definitions (as well personal information informing definitions). Much of this work is easily done by context, but not all. That's why being a judge is difficult. Some laws are easier than others, but they exist on a scale.

On the other hand, if I commit a crime for which there is a $25,000 fine, and a judge checks his cheat-sheet and misreads it instead as $2,500 and fines me 1/10 of my due, that's not the law being subjective, even though it's "subject to human interpretation" (myopia is a form/limitation of interpretation). It's bad technology (human judge w/ human eyes) offering a flawed interpretation of an objective rule.

In a sports context, by your definition, because a human in an umpire chair called balls "in" or "out" in a tennis match prior to (whenever), the tennis court, though exactly measured and painted, was "subjective."

I knew I'd get nailed on the law example, but I had to try for it anyway to see why it was flawed.

In terms of the example of the judge's misreading the fine, I'd argue that a misreading could be caused by subjective factors beyond a shortcoming in the mechanical processes of the eye. For example, that error could be partially caused by a faulty memory of what the fine "should be" or a subconscious opinion about the severity of the crime/the nature of the criminal. I think the word "subjective" implies that something is subject to certain biases, and I find it hard to believe that any human ruling/decision can be devoid of internal biases.

In baseball and tennis, I agree that some "mechanical" errors are mostly due to limitations of human sensory perception. But there are other "subjective" errors caused by extraneous factors which influence the umpire's perception of reality. A computer system would probably limit the first class of error (due to increased precision) and eliminate the second class. And regardless of this rhetorical argument, if we want an accurate analysis of actual strike zone outcomes under the current system, we need to take into consideration all of those human factors.

In other words, I believe a "strike" has two separate definitions, the objective definition stated in the rulebook, and the partially subjective definition of the umpire on each and every pitch. When we make a statement after a start such as "Chen threw 100 pitches, 60 of them for strikes", which definition are we using?

[CA-ORIOLE]

Or probability/confirmation bias, which I also think plays a part. I.e., good pitchers get calls because good pitches are expected. It's understandable that our expectations would back-fill what we didn't see but are charged with interpreting.

srock: More difficult then accuracy, is reliability and objectivity. By objectivity I mean, the hometeam can't squeeze the strikezone a bit when they are at bat!

Again, it's my opinion that good pitchers get more calls primarily because they have better command, the ability to exploit the umpires given strike zone, and the skill to execute pitches within the current system of evaluation..............not just because of their reputation as good pitchers. It is a skill by both the pitcher and the catcher. I don't discount vet/hometown bias (or other possible biases), but generally speaking, I think they are lesser and secondary factors. I also don't discount some level of inconsistencies in a single game, but in general think the in-game inconsistencies are usually lesser/secondary factors as well.

[Lucky Jim]

I think the word "subjective" implies that something is subject to certain biases, and I find it hard to believe that any human ruling/decision can be devoid of internal biases.

You're making the mistake of conflating "ruling/decision" with the law. A "ruling/decision" is a separate entity - it's the result of "technology" applying a "law" to specific "facts." If you improve the technology, you get rid of the biases. But you don't change the objectivity of what the decision is supposed to be based upon.

The other part of your post is somewhat tautological. Obviously, if "Chen threw 100 pitches, 60 of them for strikes," you're talking about a historical fact, separate-and-apart from what Chen's status would be under a proper application of the strike zone. The game is over. And that is the outcome. Just as the fact that "Lucky Jim got fined $2500 for being a message board elitist" when he should have been fined $25,000 doesn't mean that there are "two separate definitions" of the law mandating a $25,000 fine for being a message board elitist.

We're not disagreeing about all that much. My point is that you need to work hard to minimize "human ruling/decision"-based biases, and if eliminating umpire-based balls and strikes is the method, then so be it. So when you write:

But there are other "subjective" errors caused by extraneous factors which influence the umpire's perception of reality. A computer system would probably limit the first class of error (due to increased precision) and eliminate the second class. And regardless of this rhetorical argument, if we want an accurate analysis of actual strike zone outcomes under the current system, we need to take into consideration all of those human factors.

...I'm not sure what you're getting at. I've listed bounded rationality and the limits of human perceptions as the very reason why we live with the flawed system we do (I find the arguments about "cheating" and "crookedness" wholly irrelevant). I think it's well-known why these things happen -though the interaction of the various factors for various umpires probably differs.

But that doesn't mean that we should apologize for them, anymore than we would apologize for any piece of technology that does a #$&*ty version of its job.

You're not wrong about the definition of "subjective," but your take is incomplete.

SUBJECTIVE

a : characteristic of or belonging to reality as perceived rather than as independent of mind : phenomenal — compare objective 1b b : relating to or being experience or knowledge as conditioned by personal mental characteristics or states

4

a (1) : peculiar to a particular individual : personal <subjective judgments> (2) : modified or affected by personal views, experience, or background / a subjective account of the incident b : arising from conditions within the brain or sense organs and not directly caused by external stimuli <subjective sensations> c : arising out of or identified by means of one's perception of one's own states and processes / a subjective symptom of disease — compare objective 1c

OBJECTIVE

b : of, relating to, or being an object, phenomenon, or condition in the realm of sensible experience independent of individual thought and perceptible by all observers : having reality independent of the mind

In other words, the "ruling/decision" of a "strike" or "ball" may well be subjective. That changes nothing, however, regarding the objectivity of the strike zone as something fixed, extrinsic, independent, and preceding the call of a strike or ball. To the extent that umpires present a shoddy technology that interferes with the application of that objective criteria (whether by bias, myopia, laziness, or anything else) they should be replaced with a technology that functions better, whether it be fully-automated (PitchFX-based), cyborgean (Ump+technological aid) or simply involve a better quality of umpire (more regulated, and position-adjusted, perhaps?). Any of those could work, theoretically.

In the end, acknowledging the myriad ways that umps get it wrong, and why, is neither here-nor-there. It's obvious. Fixing it is the issue. And any "fix" will involve a return to that objective strike zone.

[Lucky Jim]

Again, it's my opinion that good pitchers get more calls primarily because they have better command, the ability to exploit the umpires given strike zone, and the skill to execute pitches within the current system of evaluation..............not just because of their reputation as good pitchers. It is a skill by both the pitcher and the catcher. I don't discount vet/hometown bias (or other possible biases), but generally speaking, I think they are lesser and secondary factors. I also don't discount some level of inconsistencies in a single game, but in general think the in-game inconsistencies are usually lesser/secondary factors as well.

That's fine. I'm adding one more possibility for other readers to consider. I know that what I'm talking about exists in general as a means of coping with incomplete information, and can't see any reason it wouldn't apply here. It's no different than the likelihood that, at times, good hitters get the same kind of benefit of the doubt. If you believe umpires are different than pretty much every other human being, I'm sure they'll appreciate that.

[CA-ORIOLE]

Introducing ROBO-UMP.

Arguing with ROBO-UMP will result in a painful phaser shot to the groin.

[CA-ORIOLE]

If you believe umpires are different than pretty much every other human being, I'm sure they'll appreciate that.

Since I have openly admitted that umps have this bias (and probably other biases), why was this comment needed? It's just my opinion that the biases you're talking about here are a lesser/secondary factors in the bigger picture of things. The point being, that the primary biases are in fact more systematic, fair, and based on player skills.

[square634]

You're making the mistake of conflating "ruling/decision" with the law. A "ruling/decision" is a separate entity - it's the result of "technology" applying a "law" to specific "facts." If you improve the technology, you get rid of the biases. But you don't change the objectivity of what the decision is supposed to be based upon.

The other part of your post is somewhat tautological. Obviously, if "Chen threw 100 pitches, 60 of them for strikes," you're talking about a historical fact, separate-and-apart from what Chen's status would be under a proper application of the strike zone. The game is over. And that is the outcome. Just as the fact that "Lucky Jim got fined $2500 for being a message board elitist" when he should have been fined $25,000 doesn't mean that there are "two separate definitions" of the law mandating a $25,000 fine for being a message board elitist.

We're not disagreeing about all that much. My point is that you need to work hard to minimize "human ruling/decision"-based biases, and if eliminating umpire-based balls and strikes is the method, then so be it. So when you write:

...I'm not sure what you're getting at. I've listed bounded rationality and the limits of human perceptions as the very reason why we live with the flawed system we do (I find the arguments about "cheating" and "crookedness" wholly irrelevant). I think it's well-known why these things happen -though the interaction of the various factors for various umpires probably differs.

But that doesn't mean that we should apologize for them, anymore than we would apologize for any piece of technology that does a #$&*ty version of its job.

You're not wrong about the definition of "subjective," but your take is incomplete.

In other words, the "ruling/decision" of a "strike" or "ball" may well be subjective. That changes nothing, however, regarding the objectivity of the strike zone as something fixed, extrinsic, independent, and preceding the call of a strike or ball. To the extent that umpires present a shoddy technology that interferes with the application of that objective criteria (whether by bias, myopia, laziness, or anything else) they should be replaced with a technology that functions better, whether it be fully-automated (PitchFX-based), cyborgean (Ump+technological aid) or simply involve a better quality of umpire (more regulated, and position-adjusted, perhaps?). Any of those could work, theoretically.

In the end, acknowledging the myriad ways that umps get it wrong, and why, is neither here-nor-there. It's obvious. Fixing it is the issue. And any "fix" will involve a return to that objective strike zone.

Believe it or not, I pretty much agree with you completely, until the last line. If it is possible to accurately "acknowledge the myriad ways that umps get it wrong, and why", the Orioles might be able to take advantage so that they are the beneficiaries of more calls. We agree that a better system should be instituted to determine balls and strikes. But until that time, is it better for the Orioles to throw their hands up in the air and say "What can we do, we are disproportionately screwed over by umpire rulings according to PitchFx?", or to change their behavior to better take advantage of the current system?

[CA-ORIOLE]

Believe it or not, I pretty much agree with you completely, until the last line. If it is possible to accurately "acknowledge the myriad ways that umps get it wrong, and why", the Orioles might be able to take advantage so that they are the beneficiaries of more calls. We agree that a better system should be instituted to determine balls and strikes. But until that time, is it better for the Orioles to throw their hands up in the air and say "What can we do, we are disproportionately screwed over by umpire rulings according to PitchFx?", or to change their behavior to better take advantage of the current system?

............................Yes.

[Lucky Jim]

Since I have openly admitted that umps have this bias (and probably other biases), why was this comment needed? It's just my opinion that the biases you're talking about here are a lesser/secondary factors in the bigger picture of things. The point being, that the primary biases are in fact more systematic, fair, and based on player skills.

How are you defining fact? And I thought you said earlier that it was merely your opinion? Anyway...

If we are dealing with "fact," it seems to me that your hierarchy of values regarding what influences these things is, while important to you, of less significance to the bigger picture - unless there's something empirical backing it up. Determining what's primary or secondary seems to bite off more than we have the data/information to chew right now. That said, I think it's a fair intuition. (Not that you need my validation.)

Your initial comment, that I piggy-backed on, identified things that automated-umping would eliminate. To the extent that it exists, the bias I identified would also be eliminated. Which is why I added it. You responded by saying that you didn't think it existed/played a role. I merely defended my position by explaining that - in my attempt at an agnostic, "objective," grab-all assessment - I see no reason to eliminate something that is very real, even if you intuitively don't think it matters much.

[Lucky Jim]

Believe it or not, I pretty much agree with you completely, until the last line. If it is possible to accurately "acknowledge the myriad ways that umps get it wrong, and why", the Orioles might be able to take advantage so that they are the beneficiaries of more calls. We agree that a better system should be instituted to determine balls and strikes. But until that time, is it better for the Orioles to throw their hands up in the air and say "What can we do, we are disproportionately screwed over by umpire rulings according to PitchFx?", or to change their behavior to better take advantage of the current system?

It's a strawman. Of course the Orioles should "get better" and "learn to beat the system." But that's not the point of the discussion. No one here's offering ideas as an alternative to having pitchers talented enough to manipulate a faulty system.