Objectivity, In A Subjective World
This week ESPN gave us a snapshot of what’s become one of their standard summer projects, #NBARank
. As an end-of-season treat they put together an early preview, giving their panel of 111 contributors (myself included) the standard task of ranking each player on a scale of 1-to-10 in terms of ‘current quality.’ This time instead of evaluating the whole league we were just asked to assess a core group, assembling a list of the top 30 players in the league. There were no surprises at the top, with LeBron nailing a perfect 10
I always find this project fascinating, not because of the order the players eventually settle into, but because of what it reveals about the distribution of talent in the league, how that talent is perceived and what scale is created by this sort of mass evaluation. When this project was first run, in the summer of 2011, I looked at how the results reflected the top-heavy talent distribution of today’s NBA. This past summer, I looked at what appeared to be a severe undervaluing of the rookie class. With these latest round of results, I thought it might be interesting to compare the subjective voting to some objective, statistical measures of player quality.
The table below shows each player included in the NBA Rank Top 30. The blue line shows their score from ESPN’s project, the average rating given by the 111 panelists. The red line shows each player’s Win Shares from this season, taken from Basketball-Reference. The green line shows each player’s Wins Produced, taken from Patrick Minton’s site, The NBA Geek. The purple line shows each player’s Value Over Replacement Player (VORP), taken from Daniel Myers’ site, DStats.
One of the things I always struggle with as a voter in this project is how to create separation between tiers of talent, with only the 1-to-10 scale to work with. This season, that meant giving LeBron James, and only LeBron James, a perfect 10. Win Shares and Wins Produced both rate Kevin Durant‘s season as fairly comparable to LeBron’s, but all three measures show an enormous drop-off after those two players. Depending on how you evaluate Durant, leaving the two of them alone in the 9-to-10 range may not even be enough. Making the 1-to-10 scale fully reflect the difference in production between them and their peers may require leaving the score of 8 completely empty as a spacer of excellence.
Looking at the graph, we can see a few other places where the wisdom of the masses differed significantly from the numbers. James Harden was ranked 8th by the voters, but measures out no lower than 5th in any of the statistical systems. All three systems also seem to indicate that Marc Gasol, Blake Griffin, Stephen Curry and Paul George may be undervalued. Although VORP doesn’t like him quite as much as the other two, Tyson Chandler is another player who’s NBA Rank is much lower than his statistics might indicate.
Looking at players who are overvalued by the group is a little bit trickier. These metrics are cumulative which means values are naturally lower for those players who have missed significant time with injury this season. This makes it difficult to realistically compare the ratings for players like Dirk Nowitzki, Derrick Rose, John Wall and Rajon Rondo. However there are some healthy players on which the numbers seem to agree. None of the three statistical systems ranks Carmelo Anthony as a top-ten player, although he was ranked 9th by the ESPN voters. LaMarcus Aldridge, Chris Bosh and Dwight Howard are other players who the numbers seem to agree may have been placed a little too high.
Besides the fact that the statistical systems order these specific 30 players from NBA Rank differently, they don’t even agree on which players make up the top 30. Here are the players from the Win Shares Top 30, missing from this round of NBA Rank:
Those nine players would bump out Dwight Howard, Kyrie Irving, Dirk Nowitzki, John Wall, Rajon Rondo, Derrick Rose, LaMarcus Aldridge, Kevin Love and Paul Pierce.
Here are the players from the Wins Produced Top 30, missing from this round of NBA Rank:
Wins Produced disagrees about more than half of the Top 30 including players like Carmelo Anthony, Deron Williams, LaMarcus Aldridge, Brook Lopez and Chris Bosh.
Here are the players from the VORP Top 30, missing from this round of NBA Rank:
Those nine players would bump out Zach Randolph, Kevin Love, Brook Lopez, LaMarcus Aldridge, Derrick Rose, John Wall, Rajon Rondo, Chris Bosh and Dirk Nowitzki.
Three names showed up in the top 30s from all three statistical systems, but were absent from NBA Rank – George Hill, Mike Conley and David Lee. Showing up on two of the three lists were David West, Paul Millsap, Serge Ibaka, Kenneth Faried and Jose Calderon.
Ultimately, these differences are all academic. ESPN’s goal is not to create a definitive scale of NBA talent, it’s to stimulate and extend discussion. The criteria given to panelists is purposefully structured in an open-ended way, allowing each respondent to make decisions however they see fit. It’s clear though from the results that those factors weighed by each individual are wide-reaching, stretching well beyond pure statistical production.