Do MLB Teams Undervalue Defense — Or Just Value It Differently?
By Rob Arthur
Link: https://fivethirtyeight.com/features/do-mlb-teams-undervalue-defense-or-just-value-it-differently/
What’s Dexter Fowler’s defense worth to the Cardinals?
Absent a helpful general manager opening up his computer system — or letting you hack in, if that’s more your style — it’s tough to know what baseball teams think of different players. But one place GMs leave clues about their preferences is in free agency. Since each team can bid on every available player, and the competition to acquire the most valuable talent is fierce, the free-agent sweepstakes is baseball’s closest answer to an open market; accordingly, the cash that teams deal out tells us how much they’re willing to pay for each area of on-field expertise. And for all the strides made in evaluating defense (plus convincing clubs to buy in), my analysis of recent offseasons suggests that MLB teams still don’t value defense the same way as sabermetricians do — though it might not be because they don’t value it enough.
To estimate how much teams pay for offense relative to defense, I looked at the average annual value of every non-catcher position-player contract signed since the 2006 offseason and compared those dollar figures to players’ offensive and defensive runs above average (according to FanGraphs.com) in the previous three years. I found that, from the front-office perspective, a run saved just isn’t worth as much as a run scored.
For every offensive run a player generated above average in the season before he inked a new deal, he was paid an extra $215,000. An offensive run two years back was worth $113,000, and there was even value — $93,000 per run — in stats from three years in the past. By contrast, each defensive run was worth only $84,000 one year back, with the benefit even lower in earlier years.
As far as teams are concerned, then, offense is what drives a player’s value, with defense a secondary priority. That’s not to say that front offices ignore fielding entirely when signing players, but they do appear to take defensive statistics with a grain of salt.
One potential reason is obvious: Teams may be late to understanding the value of good defense. Mitchel Lichtman, the sabermetrician who created the defensive metric Ultimate Zone Rating, told me in an email that “it will probably be a long time before teams fully appreciate the proper mathematical role of defense in evaluating players and making transactions.”
Of course, analytics experts have been arguing that teams undervalue defense since the days of Moneyball. And in the years since, sabermetrics has gone mainstream. Teams now employ huge research and development departments — many staffed directly from the ranks of baseball writers who used to criticize teams for undervaluing defense. So it seems unlikely that those analysts forgot about glovework when they walked through the front-office doors.
Moreover, teams have shown that they can rapidly adjust to new information about player value when it emerges. For instance, pitch-framing skills — wherein a catcher boosts the chance that a pitch will be called a strike — weren’t quantified until about 2009. Before that offseason, there was no significant relationship between a catcher’s framing ability and the average annual value of his free-agent contract; afterward, the correlation spiked.6 In other words, teams quickly learned about framing skill and calibrated their contract offers to adjust for it.
So another potential explanation for the seeming disconnect between defense and dollars is that teams do properly quantify defense, but in a different way than our publicly available metrics. In other words, if we replaced the public metrics in my previous analysis with proprietary ones cooked up by front offices, there’s a possibility that the inconsistency between defensive performance and pay would disappear.
I spoke with a handful of former and current front-office analysts about how teams value defensive metrics, none of whom would speak on the record. They mentioned that some teams have proprietary systems to measure the value of defense, sometimes adding input from scouts or other non-public data sources. If even a handful of teams have systems that produce substantially different fielding valuations than the public statistics, it could appear as though they are disregarding defense — when, in fact, they’re measuring it better than we know.
At a minimum, teams have access to much better data with which to construct defensive metrics than the public. With the advent of Statcast, MLB’s radar-based tracking system, team analysts can quantify the location and movement of every player on the field. Statcast also provides raw data on the running speed and reaction times of fielders, allowing front offices to break defense down into its individual components.
MLB is providing only a fraction of that data to the public. But if we’ve learned anything from early attempts to model fielding using Statcast — such as Catch Probability, which measures the likelihood that any given batted ball will be caught — it’s that new fielding metrics can disagree significantly with conventional ones.
Depending on the analysis being run, Statcast-based Catch Probabilities correlate with Ultimate Zone Rating either moderately (r=.47) or strongly (r=.71). Some have taken the latter as confirmation that UZR was correct all along. But consider as well that the correlation between batting average and on-base percentage is also 0.71.8 The realization that OBP was a better hitting metric than batting average, you may recall, formed one of the cornerstones of the Moneyball revolution.
Similarly, if Statcast-fueled metrics represent as much of an upgrade on public defensive stats as OBP was on batting average, it’s no wonder that teams don’t seem to value defense. Those clubs might just be so far beyond currently available fielding statistics that sabermetricians can no longer criticize them.
It’s likely that not every team has its own defensive metric, and some teams’ metrics are probably no better than the public’s. But occasionally, it’s obvious that a team is onto something we don’t know about. Take the case of outfielder Dexter Fowler: Before he joined the Cubs for the 2015 season, writers raised questions about his defense based on poor public metrics; that year, he ended up earning 3.3 wins above replacement, fueled largely by a 20-run improvement in his UZR. Then in 2016, after more skepticism about Fowler’s fielding, he put up the best defensive numbers of his career to date. That, in turn, led to his signing by the St. Louis Cardinals this past offseason for the staggering total of $82.5 million over five years, questionable defense be damned.
Whether because of Statcast or scouting, the Cubs and now the Cardinals have seen something in Fowler’s performance that current fielding valuations don’t seem to capture. And when two of the smartest front offices in baseball appear to be discarding defensive metrics, it makes you stop and wonder whether the metrics might just be wrong.