Go to any Pittsburgh Pirates forum and you’ll see some variation of the following conversation:
Fan #1: Ronny Cedeno is a horrible shortstop.
Fan #2: According to his UZR/150, Cedeno is the ninth best defensive shortstop in the league this year.
Fan #1: That just shows how wrong UZR is. I’ve seen Cedeno, and he’s nowhere near the best defensive shortstops in the game.
If you’re unfamiliar with UZR, it stands for Ultimate Zone Rating, and in my opinion it is the top statistical method of evaluating defense. For a great breakdown of how the stat works, check out the FanGraphs UZR primer. Plenty of people share my opinion, but don’t let that confuse you in to thinking that UZR is the consensus guide to evaluating defense. There are plenty of people on the other side of the debate who think UZR is totally worthless.
In most cases I encounter, the anti-UZR people don’t like the stat because it goes against a belief they have on a certain player. You probably won’t get many people arguing over the fact that Pedro Alvarez has a -10.9 UZR/150, putting him well below average on defense. However, you will get a lot of skepticism over Andrew McCutchen’s -12.5 UZR/150 or Cedeno’s 2.9 UZR/150 this year. The reason for this is simple. People don’t believe Pedro Alvarez is good defensively, so it’s not a huge shock to hear that a defensive stat has Alvarez well below average defensively. On the other hand, people think of McCutchen as good defensively, and Cedeno as too inconsistent to be a top shortstop in the league, putting their UZR numbers in question.
Part of that is due to the nature of the modern stats. For the longest time, baseball stats were very basic. Amount of hits divided by the amount of at-bats told who was a good hitter. Amount of runners batted in told who was a good middle of the lineup guy. Wins and losses, plus the amount of runs allowed per nine innings told who was a good pitcher. Saves told who was a good closer. Errors divided by fielding attempts told you who was a good fielder.
Now we look at strikeout and walk ratios to determine who is a good pitcher, regardless of their role on the pitching staff. We look at on-base percentage and slugging percentage to determine who the good hitters are. We look at batting average per balls in play to determine whether a hitter or a pitcher is lucky or unlucky. We look at Wins Above Replacement level, or WAR, to determine how good or bad a player is. And despite the controversy, the best statistical method to evaluate defense is UZR.
Dejan Kovacevic took a look at the UZR stat, in relation to Andrew McCutchen and McCutchen’s total this year. The stat gets put in question in these situations, in part because the stat is new, but also because most people don’t understand UZR. First of all, like any stat, sample size is a huge issue. One season isn’t enough to determine the skill level of a player, regardless of the stat. That’s true whether it’s batting average, home run totals, strikeouts, or UZR. So when people say that Cedeno has been the eighth best defensive shortstop this year, that’s all they’re saying. Cedeno could play horrible next year and end up rated lower, just like a hitter who bats .300 one year, and drops down to .250 the next year.
UZR isn’t perfect, and I’ve never seen anyone say it is perfect. For example, Dejan mentions a lot of the alternative positioning used with McCutchen this year. That’s definitely something that can affect his UZR rating. UZR factors in starting field position on some plays, and it factors in outfield wall factors, but those are still being developed as the sample sizes grow. That’s part of the reason why I give more value to infield UZR, as every infield is shaped the same, and there’s not a lot of different alignments that an infielder can have, with most of those being covered by UZR.
There are two problems I see with the criticism of UZR. First of all, it’s held to a higher standard than any other stats. Fielding percentage is an accepted stat to give an indication of a player’s defensive value, despite the fact that fielding percentage is incredibly flawed. Fielding percentage only factors in errors and chances, and those rulings can be highly subjective. An error in one play might be called a base hit in a totally different play. A fielder with a lot of range could be charged with an error on a play that a lesser fielder wouldn’t even be able to get to. Fielding percentage also gives no credit to those fielders with extra range on difficult plays.
At the same time, UZR isn’t perfect, but it’s far better than fielding percentage. However, many critics of UZR will simply point out a few flaws, and be content to stick with fielding percentage, which has even more flaws, all due to familiarity. That shouldn’t be a surprise. The same situation exists with the replacement of RBIs for hitters, and won-loss records for pitchers.
The biggest challenge to UZR is the nature of defense. It’s hard to have major differences on offense. A hit is a hit. A home run is a home run. The same is true for pitchers. A strikeout is a strikeout. A walk is a walk. Defense is entirely subjective, which is why many people rely on “the eye test”. The problem with the eye test is that no one can truly be objective when using it.
As I always say, people are usually looking for affirmation, not information. If a fan thinks that a certain player is a bad fielder, then any bad play is going to stick out to that fan as reason why the player is a bad fielder. Any good play is going to be appreciated, but viewed more as a pleasant surprise. Likewise, if a fan thinks a certain player is a good fielder, any good play is proof, while any bad play is an exception.
Take Ronny Cedeno, for example. No fielder is perfect, and every fielder will make mistakes. Likewise, every fielder will make above average plays. The way to judge how good or bad a fielder plays would be to add up the mistakes and above average plays, and see which side the player ends up falling on. No fan sits in the stands and does this. Instead, fans take mental notes. The problem with mental notes is that they come with biases. If Cedeno makes a bad play, and a fan thinks Cedeno is bad defensively, then that fan is more likely to remember the bad play Cedeno made. The same fan is going to chalk up any above average plays as exceptions, rather than keeping track of them in the mental notes.
Aside from being subjected to personal biases, the eye test is completely based on what the person sees in front of them, with no other factors considered. When Cedeno makes a mistake, people only see that mistake. What they don’t see, or consider, is how often other players around the league make or miss the same play. Did Cedeno miss a play that is commonly missed by other shortstops, or did he miss a play that is an easy out by other shortstops? The eye test doesn’t care about that. Cedeno missed the play, so chalk one up for him being a bad defender.
When you really think about the eye test, what does it boil down to? A fan is watching a player, and taking mental notes on every play the player makes or misses. There are three problems here. One, the fan probably isn’t going to see every play the player is a part of during the season. Even a season ticket holder who attends all 81 games isn’t going to see the player during every road game. Two, there’s the personal bias that I mentioned above. Three, the fan only cares about this one player in front of them.
Let’s think about what UZR is. It’s a scorekeeper who takes notes on every play made or missed by a player. So right there, it’s the same approach as the eye test. The UZR scorekeeper is actually jotting down information on each play, such as where the ball was hit, how hard or soft it was hit, the defensive alignment, the side of the plate the batter was hitting from, how many outs, how many runners on base, etc. All of these recordings are then taken and compared to every other player around the league, to see how common a similar play is made by other players at that position.
Basically, UZR is a more advanced eye test, only it actually takes notes on every single play, something that no fan does, and compares those notes to other players around the league, which gives a true indication of whether a player is above or below average at a position.
I don’t think the eye test is totally worthless. As with any stat, I believe UZR and the eye test should be used together to form a complete opinion. Cedeno is the perfect example of that. I have seen Cedeno play, and I believe he has all the tools to be a top defensive shortstop. The knock on Cedeno is that he’s inconsistent, something I have also observed. That doesn’t really say what type of defender he is. An inconsistent player can still be above average, which is what UZR says Cedeno is. In fact, UZR even indicates that Cedeno is inconsistent.
At one point this season, Cedeno was the best defensive shortstop in the league. When Dejan wrote his piece on UZR above, Cedeno was third in the league. Cedeno now ranks eighth. Cedeno is still above average, but he’s not at the top of the league. To me, that says that he’s an above average defender, but the lack of consistency holds him back from being a top defensive shortstop. UZR is obviously showing that Cedeno is inconsistent, otherwise he’d still be at the top of the rankings.
There are currently advancements being made to improve the ability to grade defense on a statistical level. Eventually we will be able to consider where the fielder started, the path the fielder took to the ball, the arch of the ball, how hard the ball was hit, the weather factors on the play, and pretty much everything you’d need to know to grade a player defensively. For now, the best measure of grading defense we have is UZR. Is it perfect? Of course not. However, neither is fielding percentage, and neither is the eye test.
To suggest that UZR should be discredited because it’s not perfect seems unfair to the stat. No method of evaluating defense is perfect, and right now UZR is the most advanced of any method available. If I’ve got someone telling me that Ronny Cedeno hasn’t been an above average defensive shortstop this year, and I’ve got UZR telling me Cedeno has been above average, I’m trusting UZR. The person performing the eye test isn’t comparing Cedeno to other players in the league. Instead, Cedeno is being compared to a hypothetical shortstop who makes whatever play Cedeno misses, out of convenience to the personal bias that Cedeno is a bad defensive shortstop.
UZR has no biases. UZR simply compares the play Cedeno made to similar plays by every other actual shortstop in the league, and uses this information to determine whether Cedeno is above or below average. UZR also factors in every play, which is something that the eye test fails to accomplish in almost every case. So if you’re saying Cedeno is poor defensively, and UZR is saying Cedeno is above average, maybe you should consider your view. Should UZR be discredited because you disagree? Or is there a chance that you might be wrong, simply because you only see a limited amount of plays Cedeno makes, only remember the bad ones that fit your existing bias, and never consider what the other 29 shortstops in the majors would do during similar plays?
