Jump to content

Crasnick article on Defensive Stats


JTrea81

Recommended Posts

There's a great Crasnick article on ESPN.com regarding the validity of defensive stats and how they compare in relation to actual scouting.

And apparently the folks that did Pitch/FX are going to do something similar for defenders:

Baseball executives clamoring for more foolproof analysis might soon get their wish. Sportvision, the company that pioneered the Pitch/FX technology used to monitor the speed and movement of pitches on television and the Internet, is working to apply similar technology to defense. This year, Sportvision installed special cameras at AT&T Park in San Francisco to track everything from fielders' jumps to the velocity of batted balls. A more widespread rollout of the product is still under discussion.

Ryan Zander, Sportvision's general manager for baseball products, said the new application will serve three purposes -- as a coaching tool, a scouting tool and a means for teams to more accurately judge defensive performance. As Christmas gifts go, it's the equivalent of "The Beatles: Rock Band" for the modern-day baseball executive.

"With this system, we'll be able to get things like running speed, throwing speed, reaction times and 'glove to throw' time -- all sorts of information," Zander says. "We're just scratching the surface right now. There's going to be an overwhelming amount of data that will give teams the ability to look at trends over time."

Link to comment
Share on other sites

Good, balanced article, and it certainly doesn't dissuade me from the view that even the most sophisticated defensive stats somehow don't capture defense as accurately as you'd like them to. I did love the ending quote from Billy Beane as to why he relies heavily on stats:

"You have to be careful when you see something with your eyes, because sometimes your emotions tend to dictate your viewpoint," Beane says. "I can't explain why a magician looks like he pulls a rabbit out of his hat. I just know the rabbit wasn't in the hat."
Link to comment
Share on other sites

The article seems fair. Didn't go very deep into the issues, but it seems a fair summary.

The only thing I found iffy was one statement by Beane: "There's always going to be some form of subjectivity. The closer you can get to minimizing it, the more accurate you're going to be." If he's talking about measurement, then I agree with him. But if you're talking about reaching the right conclusions then I think the trick is to have the right guys whose subjectivity is trustworthy. The whole issue of "being objective" is quite tricky when you're talking about studying things. One risk is that you trust a measurement over an insightful scout because the measurement has an aura of being objective. But the scout might be right. It's not the case that subjectivity is always wrong. Some subjectivity is better than others, and sometimes the right subjectivity is the most-right thing there is.

As for the system they're talking about setting up, it sounds like they're gonna video everything at once so they can measure everything that matters on a D-play. I think that's a fine aspiration, but we should not expect this to produce usesful trustworthy results any time soon. This is because of all the things they need to sort out:

  • They're doing it in 1 ballpark. They would need to do it in all ballparks.
  • They need to do it right, which is probably harder than we think. Maybe it makes sense to do it in just 1 ballpark until they can sort out exactly how to get all the measurements right. What they're doing will be based on math-based analysis of video images, and I have no doubt that can be done properly, but there's gonna be a ton of seemingly-little things they're gonna have to sort out.
  • Once they get it right, and get it working everyplace, so they've got the data they're trying to get, they're still gonna have to figure out what to do with it all. IMO, this will be hard to do. I'm not saying it's impossible to do, just saying it will be hard to do. Knowing how to weigh various things in a multi-factor problem is gonna take a while to do to anybody's satisfaction.
  • It will take way, way longer to do to everybody's satisfaction. They're gonna have all kinds of arguments, and many of them will be perfectly valid arguments that are hard to resolve.
  • I think the hard part is gonna be tying it to runs. Let's say that they find out that This Guy gets a better jump than That Guy, and throws better too, but That Guy is better at knowing where the ball is going. Turning those judgments into runs is gonna be a very non-trivial problem. I think this might be one of those things where we have valid arguments about that part forever.

I'm all in favor of them trying to do this. The problem of coming up with proper D-stats requires new forms of data acquisition. Unlike hitting and pitching stats, you just can't do it based on scorecard data, which is pretty much all we use for hitting and pitching stats. The explosion in SABR stuff is because PC's let people slice-and-dice the same old scorecard data we've always had, it's just that PC's and the intertubes permit people to look the same old data in a zillion different ways. Doing D-stats right requires something completely new, and what they're trying to capture seems like the right stuff to try to get. But nobody should expect this to produce new magic numbers any time soon. This is simply because the problem is way harder. The difficulty difference between P-stats and O-stats vs. D-stats is like the diff between aiming artillery right vs. trying to shoot down a missle with another missile. What these guys are trying to do is like SDI. So, we should be patient about getting good results. The only thing we can pretty much count on is that the early versions of results will be wrong. There will be a significant phase of that, and during that phase some folks will trust the wrong results because they're based on numbers and are supposedly objective. But the interpretation and judgment based on them won't be, there will be subjective value judgments inside the equations, but some folks will miss that. It's just the nature of the beast.

Once they get the basics right about turning video into measurements, it will be interesting to see if-and-how the new data sets might filter into P-stats and hitting-stats. The new datastream might end up taking those things beyond their current basis in age-old scorecard data. I don't know how it's gonna turn out, but one possibility is that we'll still have fuzzy pictures about D-issues, accompanied by a surprising improvement in hitting-stats and P-stats that we cannot foresee right now.

Link to comment
Share on other sites

Beane says. "But when you're using 'objective' data, you have to be careful to ferret out how it was collected. There's always going to be some form of subjectivity. The closer you can get to minimizing it, the more accurate you're going to be."

Even Beane who is an advocate of stats thinks that the collection of defensive stats is subjective. This is what we have been talking about concerning Markakis and his UZR/150 rating.

Good article.

Interesting that many scouts think Bay is better defensively but the stats say Holliday is better.

Link to comment
Share on other sites

Talk about taking a couple of statements and twisting them to meet your preconceived notions.

Beane didn’t say (all) defensive stats are subjective, he said there’s some subjectivity involved in them. I don't know of anyone who knows what he or she is talking about who would say different. The same can be said about any offensive stats that count hits. Whether a ball is an error or a hit is about as subjective as you get. That alone impacts AVG, OBP, SLG, OPS, and wOBA (as a minimum) on the offensive side and ERA (as a minimum) on the pitching side. Actually I was wrong when I said the H vs E was about as subjective as you get. Whether a pitch is a strike or a ball is even more subjective and has an even greater impact.

Also, you have conveniently excluded the other item attributed to Beane:

Reference Bay, unless I missed it I see one line where it is stated that one team’s scouts prefer Bay to Holliday. Yet you were somehow able to turn that into most scouts feel that way? Pretty impressive leap there.

I have no doubt that that Beane likes stats. There was a book written about that. It is not news.

That Beane considers the the defensive stats subjective is news IMO.

About Bay I used the word "many", you twisted that to "most". That isn't what I said.

The article is making a point and it is stated right up front:

Much gray area in defensive analysis

Trust the eyes or the numbers?

Defensive ratings still highly subjective

By Jerry Crasnick

ESPN.com

I didn't write it, Crasnick did.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.



×
×
  • Create New...