Box Plus/Minus (BPM)

Hey everyone,

I’ve really enjoyed doing this series so far and hopefully you’ve enjoyed it as well. In case you missed the first two posts, I’m doing a series that deep dives into different advanced stats that are used as all-in-one measures of player performance. Previously I looked at PER and RAPM, and this week we’ll be looking at Box Plus/Minus, or BPM.

In addition to these, I’m also doing a podcast series by the same name that summarizes and discusses what I’m breaking down here. I won’t go through all the nuts and bolts in the pod, but check it out if you want a slightly higher level version.

But with that, let’s get into the breakdown of BPM!

The Basics

Despite its name, Box Plus/Minus is not really in the same family of Plus/Minus stats that we looked at with simple Plus/Minus, APM, and RAPM. These Plus/Minus stats generally have simple Plus/Minus at the heart of them, whereas BPM is a stat derived from a player’s box score stats, such as points, rebounds, and assists, combined with team performance. Perhaps the best way to explain the difference is this: APM and RAPM use the team’s performance depending on players on the court to find a player’s value, but BPM uses RAPM to find the value of box score stats, and then translates that into player value.

BPM was created by Dan Myers, and one of the interesting choices he made in the creation of the stat was to intentionally limit it to stats that were tracked as of  the 1973-74 season. His article about the stat on Basketball Reference suggests that he could have improved the accuracy of the stat had he not made this choice, but he prioritized something that could be historically useful over improved accuracy.

The Foundation

BPM is closely akin to a stat called Statistical Plus/Minus (SPM), which we should probably discuss first. If you recall from our breakdown of RAPM, Dan Rosenbaum didn’t stop once he found Pure APM. He recognized that the stat was noisy and had too much statistical error, so he continued to tinker with it in his definitive article on the stat. (Here’s a one-sentence refresh on APM: A play-by-play linear regression is run on the margin of stints of play to solve for player coefficients that equate to each player’s Pure APM.)

So from here, Dan runs another regression, but this time, rather than finding the values for each  player, he’s looking for the value of the standard box score stats. He sets Pure APM equal to an equation that consists of box score stats multiplied by coefficients. He then solves for these coefficients that set the value for each box score stat, such as points per 40 minutes, DRB per 40 minutes, ORB per 40 minutes, 3PA per 40 minutes, etc. He uses 14 of these stats in total, and solves for the coefficient for each of them. So rather than relying on APM to dictate player value, he uses APM to assign values to box score stats.  He then applies those values to each player’s box score stats to calculate SPM, where he found a much smaller standard error compared to APM.

So how does SPM relate to BPM? BPM is a variation of SPM that uses advanced box score stats wherever possible. For example, where SPM used DRB per 40 minutes, BPM uses DRBD% instead. In addition, BPM’s statistical regression is built using RAPM values rather than APM, and RAPM has proven to be a much more reliable measurement than APM (If you’re unfamiliar with RAPM vs APM, please refer to my previous article). Essentially, BPM is an improvement on almost every aspect of SPM.

The Formula

So let’s talk about exactly what all goes into BPM! There is a simple formula that we can just plug numbers into and generate a BPM number, but if you’ve been following along, the purpose of this is to understand the “why” of the formula. So we’ll get there, but let’s talk about what BPM is made of first.

As with SPM, the formula for BPM is going to consist of coefficients that are multiplied by players’ box score stats, and that will give us BPM. But what are these coefficients and where do they come from? We start with RAPM. Dan Myers used a 14-year RAPM sample from Jeremias Engelmann and used 961 players from that sample. These 961 players were the highest “weighted” players in the sample, meaning they played a substantial amount of possessions as to not negatively impact the model.

For these 961 RAPM numbers spread over a 14-year period, Dan then tested many different iterations of different combinations of advanced statistics to find the combination that provided the least error in his regression. He then set all of these advanced stats and their coefficients equal to the various RAPM values and ran the regression, looking at the error compared to the 14-year RAPM sample. In the end, he settled on a combination of statistics that satisfied a few things: 1) low error, 2) a reasonable amount of variables (not a crazy, 50-term equation), and 3) few logical nonlinear terms. So with that understanding, let’s look at the equation that Myers gives us:

Raw BPM = a*ReMPG + b*ORB% + c*DRB% + d*STL% + e*BLK% + f*AST% – g*USG%*TO% + h*USG%*(1-TOV%)*[2*(TS% – TmTS%) + i*AST% + j*(3PAr – Lg3PAr) – k] + l*sqrt(AST%*TRB%)

So now that we see all stats he settled on, let’s go ahead and define them (and note that a-l are all coefficients that he gives value to, and we’ll look at those values later):

  • ReMPG = Regressed Minutes per Game (a slightly altered MPG to avoid instances where a player plays very few games but with high minutes per game)
  • ORB% = Offensive rebounds divided by offensive rebounding opportunities
  • DRB% = Defensive rebounds divided by defensive rebounding opportunities
  • STL% = Steals divided by number of opponent possessions while player is on the floor
  • BLK% = Blocks divided by number of 2-point FGA while player is on the floor
  • AST% = Assists divided by teammate FGs (so excluding FGs made by the player)
  • USG% = Number of possessions “used” by a player divided by number of possessions while he’s on the court (“used” meaning ends in a shot attempt, free throws, or turnover by the player)
  • TOV% = Turnovers divided by possessions “used” by the player
  • TS% = Points / ( 2 * TSA) where TSA = True Shooting Attempts (FG + 0.44 * FTA)
  • TmTS% = Team’s true shooting (same as immediately previous formula, but for entire team instead of the player)
  • 3PAr = 3PA / FGA
  • Lg3PAr = 3PA / FGA (for entire league)
  • TRB% = Total rebounds divided by total rebounding opportunities

Alright, now let’s take a look at what each of these terms is doing. And let me say up front that Myers explains this best in his Basketball Reference article that I mentioned earlier, but for my own sake at least, I’m going to walk through it myself as well.

ReMPG, ORB%, DRB%, STL%, BLK%, and AST% are all very straightforward, linear terms. Those terms are individually multiplied by their corresponding coefficients to add in to the player’s overall value. And remember that we’re using “advanced” stats here, so everything is possession or rate based. But a few notes on some specific stats here:

  • The fact that MPG was factored in is an interesting tidbit on its own. Myers found that including it improved the accuracy of BPM, and he accounts for it by noting two things: 1) Coaches often know more and give better players more playing time, even if it’s for reasons that stats or the general public can’t see and 2) it serves as an indicator for strength of opponent, meaning that higher minute players are likely facing stronger competition.
  • STL% and BLK% are simple, linear terms that only appear here.
  • ORB% and DRB% are also simple, linear terms, but note that TRB% appears later in the equation, so these will ultimately impact each other to completely account for rebounding.
  • AST% makes several appearances throughout the formula, so this AST% term is merely an adjustment to more accurately measure the value of AST%.

USG%*TOV% is our first multi-variable term, but it equals out to turnovers committed by the player divided by possessions that he’s on the court, which is still a simple, linear term.

Our next term, which is Myers’ scoring term, starts to get very interesting. Let’s look at it as a whole first, then break it down: h*USG%*(1-TOV%)*[2*(TS% – TmTS%) + i*AST% + j*(3PAr – Lg3PAr) – k]

  • USG%*(1-TOV%) can be translated as, “Shooting USG%.” It is the percentage of the team’s possessions in which the player is shooting, in other words, USG% with turnovers factored out. He multiplies this by the entire following scoring term.
  • 2*(TS% – TmTS%) accomplishes a couple things: 1) it translates TS% and TmTS% into points per shooting possession by multiplying by two, and 2) it compares the player’s TS% to the team’s average TS% (average without the player). So if our player has a higher TS% than his team’s average without him, this is a positive term. If it’s lower, it’s a negative term, which makes sense, as his less efficient shots would theoretically be hurting his team, since they would otherwise get a higher efficiency shot had the player not taken the less efficient shots.
  • AST% is factored in again here, which is interesting considering Myers referred to this as a scoring stat. But he explains that a player’s assists are worth more if he’s also contributing to scoring, which is why it’s factored in here.
  • 3PAr – Lg3PAr is considering whether the player is shooting threes more often than the league average. Given that we already factored in TS% and that this is the rate of 3PA and not a scoring stat, this is accounting for the value of stretching the floor. I absolutely love that he does this! Spacing is an extremely valuable factor on offensive and this was one of the first (if not the first) advanced all-in-one stat to account for this.
  • And finally, k is a constant that we’ll give value to shortly.

And that brings us to our last term: sqrt(AST%*TRB%). I love the way Myers explains this stat, so I’ll just share what he wrote about it:

Finally, a positive interaction term between rebounding and assists is included. This can be interpreted a number of ways – athleticism interacting with basketball awareness, size interacting with basketball skills, etc. This term was highly significant, and helped the overall fit of the regression quite a bit. Using the square root of the interaction both makes sense theoretically (maintaining a denominator of opportunities or possessions) and empirically (it is more significant and helps the overall fit of the regression more).

And that gives us our Raw BPM! Afterwards, he adjusts at the team level, and we will get there in a second, but let’s first look at the value of the coefficients.

Coeff.TermBPM ValueVariable Format
aRegr.MPG0.12339148.0
bORB%0.119597100.0
cDRB%-0.151287100.0
dSTL%1.255644100.0
eBLK%0.531838100.0
fAST%-0.305868100.0
gTO%*USG%0.921292.000*100.0
hScoring0.711217

USG%
100.0

TO%
.000

TS% & TmTS%
.000
AST Interaction0.017022100.0
3PAr Interaction0.297639.000
Threshold Scoring0.213485
lsqrt(AST%*TRB%)0.725930100.0*100.0

A quick note on two coefficients: AST% and DRB%. You’ll notice they’re both negative, and that might leave you scratching your head as to why, considering those are positive contributions. Remember that AST% is factored in multiple places, and the latter positive terms outweigh the initial negative. The same is true for DRB% when you look at TRB% factored in at the end.

So with that, let’s look at the team adjustment that Myers makes:

BPM_Team_Adjustment = [Team_Rating*120% – S(Player_%Min*Player_RawBPM)]/5

There’s really two big terms at work here:

  • The Team_Rating term is the net rating of the team, adjusted for strength of schedule.
  • S(Player_%Min*Player_RawBPM) is the total of a team’s player’s Raw BPMs, but adjusted for the percentage of minutes played. So if a player has a Raw BPM of 20, but played 10% of the team’s overall minutes, his value here is going to be two. Then you add in that value for each player on the team.

Once you have the BPM_Team_Adjustment for a team, that number is then added to every player’s Raw BPM on that team. So let’s say we have a Team_Rating of +10, and the minute-weighted sum of the players’ Raw BPM on the team is +6, that would give us an equation of BPM_Team_Adjustment = [10*1.2 – 6]/5 = 1.2. We would then add 1.2 to every player’s Raw BPM that’s on the team we just looked at.

Let’s take a slight detour to talk about the 120% number, because it stands out here as a somewhat arbitrary number at first. When watching a game where a good team falls behind, you may often find yourself assuming that the trailing team is going to make a comeback. It seems like it happens all of the time. Well, Jeremias Engelmann did a study and found that assumption to be well-based! He found that teams that are usually ahead are generally 20% better than their final results indicate. Conversely, teams that are usually trailing are generally 20% worse than their results. So this 120% accounts for that fact.

And that is how BPM is calculated! But there’s one other thing we can do here if we want to go a step further. Myers’ RAPM dataset from Engelmann had both ORAPM and DRAPM, so he ran another regression just for ORAPM to get another set of coefficients. So you can take the same Raw BPM formula, and use a separate set of coefficients to solve for OBPM. Here are the OBPM coefficients:

Coeff.TermOBPM ValueVariable Format
aRegr.MPG0.06444848.0
bORB%0.211125100.0
cDRB%-0.107545100.0
dSTL%0.346513100.0
eBLK%-0.052476100.0
fAST%-0.041787100.0
gTO%*USG%0.932965.000*100.0
hScoring0.687359

USG%
100.0

TO%
.000

TS% & TmTS%
.000
AST Interaction0.007952100.0
3PAr Interaction0.374706.000
Threshold Scoring-0.181891
lsqrt(AST%*TRB%)0.239862100.0*100.0

From there, you perform the same team adjustment to find OBPM. Then, DBPM is simply equal to BPM – OBPM. And now we’re really finished with the formula!

Bonus Round! – Value Over Replacement Player

As we mentioned in the beginning, even though BPM uses MPG as part of the formula, it is a per 100 possessions based stat, so it doesn’t account for playing time. However, Myers doesn’t leave us hanging! He also presents us with another step we can take in calculating Value Over Replacement Player, or VORP.

VORP = [BPM – (-2.0)] * (% of minutes played)*(team games/82)

VORP is supposed to measure how many points per 100 possessions a player is valued at over an average player at that position. It’s a good approximation for how much value a player is providing to his team during a season, and it’s a number that you can compare linearly. As Myers points out, a player with a VORP of 4.0 contributed twice as much to his team compared to a player with a VORP of 2.0.

Historical Examples

Given that BPM is still a box score based stat, I wanted to do something similar to what we looked at with PER. So let’s first take a look at the top 100 BPM seasons since the 1973-74, with a minimum of 40 games played.

The top 10 looks really good. There are multiple LeBron and Jordan seasons, which is usually a good sign at the top. Westbrook, Curry, Harden, and interestingly Chris Paul round out the rest of the top 10 seasons. Overall, I think this is a great representation. All of these individual seasons were some of the most incredible regular season performances of all time. The Chris Paul season is probably the one that most people would take issue with, but in general, I think this is a good list.

When we expand to the top 100 however, we start to see some cracks. Similarly to PER, we have no sign of Stockton or Nash in the top 100. I don’t believe that alone is a deal-breaker, but when you consider there are four Westbrook seasons and five Chris Paul seasons in the top 100, having Stockton and Nash missing seems to be an issue. At least we have Magic here this time, so not all is lost!

With PER, we saw defensive players severely underrated. Myers told us up front that he believe BPM doesn’t accurately reflect defensive contribution, but I expect DBPM to do a much better job than PER. So let’s take a look at DBPM historically.

The top 10 is…interesting. We have some very great signs though, with Ben Wallace and Marcus Camby having multiple appearances. There’s no way we would expect either of them to appear in the top anything for PER, but they’re clearly elite defensive players, and DBPM captures that. However, the rest of the list is a little more suspect with Ekpe Udoh and Greg Stiemsma both making appearances.

The rest of the top 100 has some notable absences, such as Kawhi, Bruce Bowen, Rodman, and Pippen, while Shawn Bradley and Myles Turner both make appearances. Myers mentioned that DRPM tended to favor big men, and that seems to clearly be the case here.

Strengths and Weaknesses

Overall, going through this exercise gave me a new respect for BPM. I was aware of its defensive deficiencies beforehand, but the method by which Myers created BPM was extremely well thought out and well tested. It truly is an enormous advancement for box-score-based stats. While it’s not a true successor of PER, it’s a better alternative in almost every respect.

Still, BPM suffers from the common box score stat deficiencies, namely defense. It absolutely is an improvement over prior box score based statistics as it captures, if not overstates, big man defense. This is a big step forward, but perimeter defense still seems to be largely undervalued here.

We also see some of the assist-first guards undervalued, such as Nash and Stockton. Even though Magic had multiple top 100 seasons, BPM tended to favor Westbrook-type play comparatively, and most basketball minds would argue Magic was a much better player.

Conclusion

As I mentioned above, I have a renewed respect for BPM after doing this exercise. As an overall stat, it’s useful to look at and should definitely take precedent over PER. I even think it’s valuable to compare players using BPM, particularly when converting to VORP in a season. But big men will be overrated on defense, and pass-first guards will be seemingly underrated. Still, BPM was an enormous advancement on box score stats!

PER: Player Efficiency Rating

https://www.podbean.com/media/share/pb-zzcxg-bcf2b1

During this episode, we take a look at PER, its place in history, how it’s calculated, and what it looks like when you look at past seasons with it. 

 

My article breaking down PER: https://basketballstat.home.blog/2019/08/07/player-efficiency-rating-per/

 

Twitter: https://twitter.com/RobertAntle

Instagram: https://www.instagram.com/robertantle/

Blog: https://basketballstat.home.blog/

Regularized Adjusted Plus/Minus (RAPM)

Welcome back to another edition of Basketball, Stat. Hopefully you enjoyed the last post I did on PER. As a follow up, I wanted to do a post covering Regularized Adjusted Plus/Minus (RAPM). For those of you who might have missed the first post or introduction, I wanted to do a series covering many of the advanced stats used in the NBA. We see these stats used all the time to compare players, but I know for myself at least, I didn’t have a firm grasp of what went into these numbers. So given that, I wanted to build that understanding for myself and share what I find!

The Basics

To get to RAPM, we first have to get to Adjusted Plus/Minus, or APM. APM is a variation of the standard Plus/Minus stat. If you’re unfamiliar with Plus/Minus, it is a very simple concept: the net score while a given player is on the court is defined as his Plus/Minus. In other words, if Player A enters the game with a score of 50-50, then exits the game with his team winning 75-70, he would have a Plus/Minus of +5 for that “stint.” This stat is cumulative. You can use it for certain stretches of a game (e.g. the Lakers were +10 when LeBron was on the court in the first half), for an entire game (e.g. even though the Lakers lost, they were +3 when LeBron was on the court), or even throughout the season (e.g. the Lakers are only 4th in the West, but are +100 with LeBron on the court so far this season).

I’m sure you can already see the potential flaws in using standard Plus/Minus to measure player performance. For example, it doesn’t account for competition or other players on your team. You can put nearly any player as the fifth in a lineup of Steph, LeBron, AD, and Giannis, and that player will have a positive Plus/Minus. Similarly, if your minutes only come against garbage-time lineups, you’re Plus/Minus will be higher than it should be. 

This also raises some questions: Does a player with an unexpectedly high Plus/Minus benefit from circumstances, or is he doing something that creates value that we might not be seeing? Is a player with an unexpectedly low Plus/Minus not as good as we thought, or is he just in an unfortunate situation?  To try and answer some of these questions, many people have tweaked the stat to try account for various things, but Dan Rosenbaum was the one who really articulated and defined Adjusted Plus/Minus in 2004. Here’s part of what he wrote:

However, these “unadjusted” plus/minus ratings do not measure the value of a player per se; they measure the value of the player relative to the players that substitute in for him. In addition, there are differences in the quality of players that players play with and against. A weak starter on a team with exceptionally good starters (relative to bench players) will generally get a very good unadjusted plus/minus rating – regardless of their actual contribution to the team.

Thus, a better measure of player value would “adjust” these plus/minus ratings to account for the quality of players that a given player plays with and against. In addition, it would account for home court advantage and for clutch time/garbage time play. Thus, unlike in unadjusted plus/minus ratings, these “adjusted” plus/minus ratings do not reward players simply for being fortunate to being playing with teammates better than their opponents.

With that said, there’s still a lot of ambiguity around APM, as there are many different versions by different people with slight tweaks. But we will specifically focus on the work Dan did for APM  which will then launch us into RAPM!

The “Formula” – APM

APM is a little more dense than a statistic like PER, as it’s not simply calculated by plugging numbers into a formula and getting a number back. To get APM, we must return to our college math classes and remember how to do a linear regression. But fear not! This is not a subpar attempt to teach you linear algebra, so I will do my best to explain it in a way the average basketball fan can understand.

Returning to our Plus/Minus example, let’s take the idea of stints. We can define a stint as a period of play in which no substitutions were made and that will have a margin (e.g. +2 if Team A wins that stint by two points). In his original article, Rosenbaum observed over 60,000 stints over two seasons in order to calculate his first public set of APM stats.

For each stint, we have 10 players on a court, five players for Team A, the home team, and five players for Team B, the away team. Let’s call the players for Team A as follows: P₁, P₂, P₃, P₄, P₅. And for Team B, we will have: P₆, P₇, P₈, P₉, P₁₀.

If a player is playing on the home team, P = 1. If a player is playing on the away team, P = -1. And if a player is not in the game for that stint, P = 0.

APM tries to figure out how these players work together to reach the margin on the court during that time, while also accounting for home court advantage (b₀). So let’s lay that out:

Margin = b₀ + b₁P₁ + b₂P₂ + b₃P₃ + b₄P₄ + b₅P₅ -+ b₆P₆ + b₇P₇ + b₈P₈ + b₉P₉ + b₁₀P₁₀

“Margin” is the offensive rating (points per 100 possessions) of Team A minus the offensive rating of Team B during that stint. Note that we use offensive rating difference instead of actual score difference in case one team gets more possessions than the other. This makes it a “per 100 possessions” margin.

If we were actually in a college linear algebra class, this is the part where we would start using matrices with lots of 1s, 0s, and -1s to solve for the “b” coefficients. But alas, we’re not, so I’ll save you the eye-sore and just assume we run a linear regression here.

But in simple terms, what the regression is doing is trying to find the values of all of the “b”s that correspond to each player that makes this equation true over all of the stints that we are evaluating.

So this regression is solving for all of the “b” coefficients, and that is ultimately what is each player’s Pure APM (this is how Dan refers to it). You can interpret each player’s Pure APM as the value a player is adding per 100 possessions according to the regression. It uses thousands of stints for data points to see how all of the “players” impact each other and minimize standard error of the regression. So we solve (to an extent) the issue of players playing with either elite or subpar lineups that will impact their standard Plus/Minus, and we get a closer representation of how much value they are adding to their team.

If you look at these raw values, you will absolutely see some outliers, and even small adjustments to the numbers will cause huge swings in the results. So Dan does a few things to try to adjust his results and turn Pure APM into APM, but this is where we will part ways with Dan’s APM and introduce RAPM!

The “Formula” – RAPM

As we begin our discussion on RAPM, this is where things can get extremely math-heavy! Again, I won’t deep-dive into what would be covered in a college linear algebra class, but if you wish to do so yourself, here is a great article that gets into that level of detail. I also want to mention Jermias Engelmann, since he’s largely credited with the creation of RAPM. He did an awesome lecture on it you can find here.

With that said, let’s talk about what RAPM is compared to APM. With APM, we will inevitably have instances where there are players with small playing times that result in giant outliers. Since APM is accounting for all of the relationships of how the players impact each other, this will affect the whole model, and you will end up with some extremely “off” values that don’t make sense. On top of that, even the numbers that do make sense will have a large mathematical margin of error according to the model. And again, small tweaks to any of the numbers in the model can impact things drastically.

So RAPM applies a filter, to put it simply. The filter is a technique called ridge regression that attempts to filter out the “bad” pieces of data without changing the ones that are valid. For those interested, here’s the equation:

β = ( XTX + ƛI )-1XTY

This isn’t a perfect analogy by any means, but for the less-mathematically inclined, you can think of it like a water faucet that hinges on the value of ƛ (lambda) in the filter equation. If lambda is zero, the faucet is turned all the way on, meaning every piece of data gets through untouched, so at lamba = 0, RAPM is the same as APM since the filter isn’t actually restricting any of the data. Conversely, as lambda approaches infinity, it’s as if the faucet is turned completely off, so all of the data is restricted, and RAPM = 0 for everyone! So we need to find the sweet spot for lambda so that we’re controlling the data at the right level. It’s not extremely important for us here, but if you’re curious at all, Joseph Sill wrote a paper for the Sloan Conference and found that the ideal lambda is close to 2,000 for one year of data, or close to 3,000 for three years of data.

So when you apply this filter, it gives us a much better balance in terms of which data is used. You won’t have the issue of “overfitting,” where it takes every piece of data, including the guy who played five minutes and went 4/4 from three and had a Plus-Minus of +10 in those random minutes. This kind of data throws off normal APM and would likely overrate that player because it doesn’t necessarily recognize this as an outlier automatically. But RAPM does!

So that is how RAPM is calculated! There really is a lot of room for minor variations, so if you go looking at various RAPM calculations that are available online, you might notice they’re all different. Again, given the nature of how it’s calculated, that’s just going to be part of the package with RAPM. But in general, the numbers should all be relatively close and ultimately tell you the same information with only slight nuances. For example, some people will calculate ORAPM and DRAPM separately, then add the two together for overall RAPM. Some won’t separate them. Some will use two seasons of data at a time, some will use just one. Some will have slightly different values of lambda. So it’s important when looking at RAPM measurements to realize what the model is using.

Taking a Step Back: Strengths and Weaknesses

You might have realized we haven’t used a single basketball value in calculating RAPM other than the score of the game. That’s one of my favorite things about this stat! Ultimately, the point of basketball is to outscore your opponent, and you do that by maximizing the difference between the rate at which you score and the rate at which your opponent scores. And RAPM focuses on players doing that exclusively. Your points scored, rebounds, and assists absolutely play a part in how well you do that, but RAPM looks below the surface of those numbers to see how you’re impacting the game based on the scores while you’re in the game, while accounting for who else is on the floor.

With that said, there’s still some head-scratching results at times. And given the nature of how it’s calculated, RAPM makes it extremely difficult to go back and find where the errors come from. That is perhaps my biggest frustration with the stat. Going through people’s RAPM data and seeing an extremely random player in the top 10 raises lots of questions: Is this player extremely underrated? Do we need to adjust the model because we’re getting an outlier result? Are we going to get this outlier result no matter how we adjust and we just need to ignore it?

I think we’ll see something that’s a common theme with every advanced stat out there, we can’t rely on RAPM blindly. RAPM is something that helps us uncover the players that traditional box score stats miss, like the Shane Battiers of the world. But even when we see an unexpected player, RAPM isn’t reliable enough alone to know that a player is a positive impact player. So ultimately I treat it as a clue or a piece of the puzzle, as should be the case with any stat or player evaluation. But we can be more specific here and say that this is a tool that is looking specifically for impact on the scoring difference, regardless of the box score. So RAPM is a stat in which a player that is creating value on defense, via screen assists, or through “hockey assists” (a pass leading to an assist) would shine. Executing to a well-designed scheme and being in the right spot that enables your teammate to get an open shot is something that is recognized by RAPM (not specifically, but through scoring margin).

Historical Examples

We want to look back at actual data from past years to compare historical data to what we expect to see from RAPM. But as we discussed earlier, RAPM does not necessarily have a single, specific way to calculate, as there are different values of lambda, multi-year vs single-year datasets, etc. So we need to choose one data source, and I will be looking at this website and their calculation of RAPM. In their “About” section, they detail how they’ve calculated RAPM, but for simplicity, they are using a single-year RAPM calculation, where ORAPM and DRAPM are both calculated and added together.

We can really make our points about RAPM just by looking at one data set, so let’s take a look at the top 10 from the 2018-19 regular season:

RankPlayerTeamPossORAPMDRAPMRAPM
1Kevin DurantGSW12,3155.26471.41936.684
2Danny GreenTOR9.8304.2972.17496.4719
3Stephen CurryGSW10,6694.39161.04925.4409
4Paul GeorgeOKC13,3101.82713.52965.3567
5Jrue HolidayNOP11,0973.26861.76955.0382
6Derrick WhiteSAS7,4242.51762.28224.7998
7Jayson TatumBOS10,8202.00521.95433.9595
8Joel EmbiidPHI9,8862.4061.40513.8111
9Al HorfordBOS8,6443.1450.58913.734
10Jimmy ButlerPHI8,2252.71890.90493.6238

Well that’s interesting! With our first small set of data, we already have quite a few unexpected results! Durant at 1 makes sense, and Steph Curry at 3 being an example of someone who plays often with him and is an elite player in his own right also makes a lot of sense. Paul George had an incredible season last year, so him at 4 looks solid as well. But Danny Green at 2, Jrue Holiday at 5, and Derrick White at 6 are huge surprises! Again, we don’t want to take these numbers as prescriptive, so our takeaway shouldn’t be “Danny Green was better than Steph last year.” That’s clearly not the case for anyone that follows basketball.

But it should prompt us to ask, “Did Danny Green, Jrue Holiday, and Derrick White provide a lot of value that we’re not seeing in the box score?” I would guess the answer is yes, but to know, we really need to dig in to each of those player’s seasons. We need to watch film to see what they’re doing on defense and off the ball. We need to look at spacing data to see if maybe Danny Green’s shooting is drawing players to him and increasing the spacing on the floor while he’s there. We need to look at the other players on their teams and see if they have similar RAPMs that could be positively impacting these numbers.

On the opposite end of the spectrum, we see a number of “star” players that might surprise us far down the rankings:

  • Kemba Walker ranked 460, with an RAPM of -0.5252
  • Ben Simmons ranked 445, with an RAPM of -0.4669
  • Devin Booker ranked 435, with an RAPM of -0.4186
  • Blake Griffin ranked 425, with an RAPM of -0.3952
  • D’Angelo Russell ranked 390, with an RAPM of -0.2611

Several of these I expected, while others were big surprises. Devin Booker is often someone talked about who puts up “empty stats.” I don’t fully believe that’s the case, but I do think his point scoring output has caused him to be overrated, and he hasn’t impacted winning at a high level. But does this mean Devin Booker is a bad player? Absolutely not! It means that if it’s assumed Devin Booker is going to be a future All-Star, we should rethink that opinion and look more closely at the areas of his game that we might be missing. Is he missing many defensive rotations? Is he over-shooting and dragging down his and/or his team’s scoring efficiency? Again, it’s something we have to watch out for, but RAPM isn’t going to tell us; it simply alerts us to a potential player misevaluation.

Conclusion

Essentially, RAPM is a “jumping off” point. It’s a signal that maybe we’re missing something. Looking back at this list, I would say that Al Horford in the top 10 makes a lot of sense, but it’s not an opinion that would have been widely accepted 5 years ago. He’s another player in the Shane Battier mold that impacts the game without filling up the box score. RAPM tells us that. But that doesn’t mean we should blindly accept it! I would be hard pressed to believe that Danny Green was the best player on the Raptors, much less the 2nd best player in the league last year. I would venture to guess, however, that his impact was underrated by the average fan. Similar things can be said for Jrue Holiday and Derrick White (and I’m especially interested in digging into how White ended up there!).

Since RAPM was invented, there have also been several new metrics created that are improvements on RAPM, such as RPM (Real Plus/Minus) and PIPM (Player Impact Plus/Minus). I will likely visit both of those metrics in a future post.

But hopefully this was helpful for you and educational! I know that for me, learning how APM and RAPM were calculated was extremely helpful in terms of understanding what I’m really looking at when I see these numbers. Thank you all again for your time reading this! If you have any thoughts, feedback, or requests, please let me know as I would love to hear them!

Player Efficiency Rating (PER)

I wanted to do a quick introduction to this series I’ve been wanting to do. As I’ve mentioned, I love looking for hidden value, and the most common place that happens in the NBA is with advanced stats and analytics. So there are a large number of stats out there trying to evaluate players in different ways. So I wanted to take a look at some of those different stats and break them down.

For the first post in this series, I wanted to look at a stat that is more commonly used but still may seem a little confusing to someone that’s new to basketball. And for those of us familiar with it, we probably still don’t know exactly what ingredients go into pumping out the numbers we end up seeing. So with that, let’s take a look at Player Efficiency Rating, or PER.The Basics

PER is a stat created by John Hollinger (formerly of ESPN, currently with the Memphis Grizzlies), that attempts to encapsulate the entirety of a player’s performance per minute into one single number, relative to the rest of the league (league average PER is always 15), while adjusting for pace. It is a series of terms (some positive, some negative) that are added together, resulting in one number that represents that player’s contribution to his team.The Formula

So with that, let’s dive into the nitty gritty. I’ll list out the formula (as found on basketball-reference.com), and below that I will define each of the variables and take a closer look at what it’s doing.

uPER = ( 1 / MP ) * [ 3P + ( 2 / 3 ) * AST + ( 2 – factor * ( team_AST / team_FG ) ) * FG + ( FT * 0.5 * ( 1 + ( 1 – ( team_AST / team_FG ) ) + ( 2 / 3 ) * ( team_AST / team_FG ) ) ) – VOP * TOV – VOP * DRB% * ( FGA – FG ) – VOP * 0.44 * ( 0.44 + ( 0.56 * DRB% ) ) * ( FTA – FT ) + VOP * ( 1 – DRB% ) * ( TRB – ORB ) + VOP * DRB% * ORB + VOP * STL + VOP * DRB% * BLK – PF * ( ( lg_FT / lg_PF ) – 0.44 * ( lg_FTA / lg_PF ) * VOP ) ]

WHEW! That, my friends, is a long equation. BUT WAIT! THERE’S MORE! First of all, some of you probably noticed that this formula hasn’t accounted for pace anywhere. What’s more, you might have noticed that absolutely nowhere in the formula is the number 15. “But Rob, didn’t you tell me this was a pace-adjusted formula and didn’t you say that it’s normalized to a league average of 15?” Congrats! You’d be correct. What is listed above is actually “unadjusted PER,” meaning that we haven’t accounted for pace yet, or normalized it. So to get there we need to take two more steps: first we adjust for pace, which we can call aPER:

aPER = uPER * lg_Pace / tm_Pace

Then, we normalize it at 15 being the league average to get our final PER number, which is what you see when you look up the stat online:

PER = aPER * (15 / lg_aPER )

Congratulations on making it this far! Now, you might have realized that in the above paragraph, I said “First of all…” but had no following “second of all…” in the paragraph. Well here it is! Second of all, that formula has at least one variable that is completely unrecognizable. For example, what on earth is “factor?” Well a couple of these are small formulas on their own, while the others are standard NBA variables. So let me just list out and explain each piece.

  • MP = Minutes played (remember, this is a per minute stat)
  • 3P = 3-point field goals made
  • AST = Assists
  • FG = Field goals made
  • FT = Free throws made
  • FGA = Field goals attempted
  • TRB = Total rebounds
  • ORB = Offensive rebounds
  • TOV = Turnovers
  • factor = ( 2 / 3 ) – ( 0.5 * ( lg_AST / lg_FG ) ) / (2 * ( lg_FG / lg_FT ) )
    • This is admittedly the strangest part of the whole equation… I will have a section below discussing this
  • VOP = lg_PTS / ( lg_FGA – lg_ORB + lg_TOV + 0.44 * lg_FTA )
    • This piece of the formula is essentially estimating the value of a possession, or the average points per possession across the league, with the numerator being league points and the denominator being an estimate of league possessions. The constant 0.44 is an estimate of the percentage of free throws that end a possession. However, it should be noted VOP is not a highly accurate estimate.
  • DRB% = League average defensive rebounding percentage ( ( lg_TRB – lg_ORB ) / lg_TRB )
  • STL = Steals
  • BLK = Blocks
  • PF = Personal fouls

There we go! Now we have all the pieces to the puzzle. Now let’s go through this monster of an equation and try and talk through what it’s doing.Parsing the Equation

We’ve already explained the parts about adjusting for pace and normalizing to a league average of 15, so here we will focus on what’s going into uPER. Let’s take it term by term:

  • 1/MP * the rest of the formula
    • Dividing by minutes played to make it a per minute stat. Easy enough.
  • 3P
    • 3P made is it’s own part of the equation, weighted at 1.
  • + ( 2 / 3 ) * AST
    • Weighting an assist as 2/3. This fact combined with the weighting of 3P is a red flag for me. Yes there are additional factors below that don’t make it that simple, but already I’m starting to question the reasoning behind how this is calculated.
  • + ( 2 – factor * ( team_AST / team_FG) ) * FG
    • Alright, starting to get fun now! Let me first state that “factor” is an interesting piece all on it’s own, so I’ll give it its own paragraph below. But with that in mind, let’s simplify it by not deep-diving into “factor” yet. So that leaves us with 2 * FG as a piece of the equation, then you subtract out factor * FG * the percentage of team’s FGs that are assisted. This seems to be weighting FGs as 2, but also subtracting some value based on how many of those FGs were assisted, which makes some sense (though it’s not a perfect assumption by any means). When you consider 3P was weighted at 1, this is essentially giving us total points scored outside of FT ( 2 * FG + 3P ) while making a slight tweak for if they were assisted.
  • + ( FT * 0.5 * ( 1 + ( 1 – ( team_AST / team_FG ) ) + ( 2 / 3 ) * ( team_AST / team_FG ) ) )
    • Here we’re starting to factor FTs. But Hollinger is also again attempting to differentiate between assisted vs. unassisted. As above, t’s pretty simplified by taking the team’s unassisted FG rate ( 1 – ( team_AST / team_FG) ) and then a 2/3 weighting of the team’s assisted FG rate. So here, the player’s FT is weighted between 0.83 and 1 according to the team’s assist rate (.83 if every FG is assisted; 1 if no FG is assisted).
  • – VOP * TOV
    • This is our first negative term. Again, VOP is an estimate of the league average points per possession. So here we’re subtracting out the average points per possession across the league that this player lost out on due to turnovers.
  • – VOP * DRB% * ( FGA – FG )
    • Here we have another negative term, but this time we’re subtracting out the average points per possession (across the league) lost due to missed FG. Notice we’re only accounting for missed points that we assume aren’t rebounded by the offensive, by taking the league average DRB% and multiplying by missed FG and again by VOP.
  • – VOP * 0.44 * ( 0.44 + ( 0.56 * DRB% ) ) * ( FTA – FT )
    • This piece of the formula looks complicated, but I’ll give you the simplified version first: we are calculating expected missed points due to missed FTs. Remember that 0.44 is a constant that is the rate at which a FT shot ends a possession (in other words, 44% of FTs are the last shot of a possession, generally speaking). So that middle term is looking for the rate at which a free throw not ending the possession does not result in an offensive rebound. So ultimately here we’re taking VOP * .44 * (an estimate of the rate at with your team does not secure an offensive rebound on a FT) * FT missed.
  • + VOP * ( 1 – DRB% ) * ( TRB – ORB )
    • This looks to be an estimation of how many points the player “contributed” by rebounding, thus reducing the other team’s opportunity to score. So we have VOP * the league’s average offensive rebound rate * the number of the player’s defensive rebounds.
  • + VOP * DRB% * ORB
    • Here we have the VOP created by offensive rebounds, weighted by the DRB% of the league (so if they league as a whole doesn’t defensive rebound as well, offensive rebounds aren’t as valuable)
  • + VOP * STL
    • This is straightforward enough: VOP created by steals
  • + VOP * DRB% * BLK
    • Again, fairly straightforward: we have VOP created by the number of blocks expected to be rebounded by his team
  • – PF * ( ( lg_FT / lg_PF ) – 0.44 * ( lg_FTA / lg_PF ) * VOP )
    • Finally we get to our last term: fouls. We have the number of fouls committed by our player first, and again this is a negative term so we’re taking out value that our player has cost the team by fouling. Then we have the rate at which PF results in made FTs in the league (lg_FT / lg_PF); so far we’ve subtracted out the number of FT we expected our player to have caused by fouling ( PF * (lg_FT / lg_PF )). Then we’re adding back in (the double negative) the VOP that comes of the free throw attempts that were expected to have ended that possession (which is why we have that .44 again). To be honest, I don’t love the second part of what is factored in here for fouls, as I feel like it gives too much positive value. But that said, I think the first part of this term on its own would be too harsh for penalizing fouls.

And that covers all of the different terms going into PER! Some of it is pretty straightforward. Some of it has me scratching my head quite a bit. So with that said, let’s take a closer look at “factor.” And let me say up-front that I don’t fully understand this part of the equation, but I’ll do my best to reason out what purpose it serves. We won’t get a clear answer, but I’ll give my best guess.

Our first observation we can make is that Hollinger is not a believer in simplifying equations. It could much more easily be written as factor = ( 2 / 3 ) – ( ( 1 / 4 ) * ( lg_AST / lg_FG ) * ( lg_FT / lg_FG ) ). But let’s take a look at the way Hollinger wrote it, as I suspect he did it for a reason. Without equation language, let’s try and say what we’re calculating here: two-thirds, minus ( ( half of the percentage of FG that are assisted ) divided by ( 2 times the ratio of FG to FT ) ). I think it’s extremely important to note that “factor” is applied to the team’s percentage of FGs that are assisted here. So it seems like he’s trying to fine tune the relationship between assists, FGs, and FTs, and make sure they are weighted in a way that is fair to the value the player is created, but it really just gets a bit too convoluted to understand without asking Hollinger himself why he did some of the things he did. So we really do hit a bit of a roadblock here unfortunately.Strengths and Weaknesses

So now that we understand most of the ins and outs of PER, let’s talk about how well we expect it to do its intended job.

Strengths

  • Easily digestible and comparable across league
  • Adjusts for the norms of a particular season by using lots of league average values for that seasons and normalizing to 15; ends up being an adjustment for different eras of the game
  • Adjusts for pace and against league averages

Weaknesses

  • Some of the constants (e.g. 0.44) are outdated and don’t accurately reflect what they should
  • Use of league averages don’t always reflect true value; i.e. a possession lost due to TO by the 2015-16 Warriors should be more impactful than one by the 2011-12 Bobcats
  • We’re mainly looking at offensive performance; the two defensive stats incorporated (STL and BLK) don’t necessarily translate to overall defensive impact, so that is a HUGE missing piece
  • It doesn’t account for who the player is playing with or against
  • PER tends to favor volume regardless of efficiency
  • Doesn’t capture any non-box score stat that impacts the game

Historical Examples

Now let’s take a look at some historical seasons and the corresponding PER to see if we can find some examples of how it translates. I went to basketball-reference.com to find all PERs of players that played at least 60 games in a season in the 3-point era. Here are some of the findings:

  • Historical top 100 PER seasons
    • The top 100 looks pretty good! Better than I expected
    • The top 5 consists of 2 LeBron seasons, 2 MJ seasons, and Steph’s best season.
    • Rounding out the top 10, we have some more MJ and LeBron, as well as Giannis last season (not crazy) and an Anthony Davis season (this is my first issue… AD is great, but at no point can we say he had one of the best 10 seasons of all time).
    • Kobe doesn’t show up until 46, and he only has 2 seasons in the top 100.
    • I would have expected to see Harden higher, when you can make the case for arguably the best offensive season in history for both last season and the year before, but those seasons show up at 13 and 21.
    • If you change the criteria to a minimum of 54 games played, Boban shows up at 58!!!
    • Two off-the-top-of-my-head notable absences: Steve Nash and John Stockton… They aren’t even in the top 250! So playmaking PGs are clearly underrated by the stat.
  • Role players on great teams (I believe this is an area where PER falls very short in adequately translating player value, especially defensive specialists)
    • 2015-16 Warriors
      • Draymond – 19.3
      • Andre Iguodala – 12.2 (12th on the team!)
      • Harrison Barnes – 12.3
    • 1995-96 Bulls
      • Dennis Rodman – 13.6
      • Luc Longley – 11.9
      • Ron Harper – 14.4
      • Steve Kerr – 15.2
    • 1985-86 Celtics
      • Robert Parish – 18.7
      • Bill Walton – 17.0
      • Danny Ainge – 13.6
    • 1999-00 Lakers
      • Robert Horry -. 14.3
      • Ron Harper – 12.0
    • 2012-13 Heat
      • Mario Chalmers – 13.3
      • Shane Battier – 10.7

Conclusion

There we have it! Hopefully this either refreshed you or gave you a new understanding of what PER is, how it works, where it falls short, and provides some context for you next time you compare players with PER. It was one of the first advanced analytics out there, so while it isn’t perfect by any means, I have to respect it as being a step in the right direction of trying to evaluate player past what’s in the box score. I think there are many better ways of measuring that today, but PER still has a place in history.

Please let me know if you have any thoughts or feedback on this that I can incorporate into the next post in the series. Thanks!

What is “Basketball, Stat”

I’ve always been obsessed in finding value that others don’t see. Perhaps it has to do with feeling that I played this role in teams growing up, but I’ve always believed that the “stars” in teams, whether or not it’s a sports team, receive much more credit than they should (win or lose), and then other “role players” are routinely undervalued.

For me, the example of this that I couldn’t get away from when thinking about the NBA was Shane Battier. I’ve spent so much of my NBA fandom thinking about Shane Battier, it’s quite frankly embarrassing. What was it about him that caused him to never make an All-Star team, yet still end his career in the top 60 of Volume Over Replacement Player (VORP)? He unquestionably made his team better when he was on the floor, but he never put up the counting stats to gain that recognition. To me, he epitomized the case of finding value in the role player that others may not recognize. Did he have a skill set that would allow him to lead a 20-win team into being a 40-win team? Probably not. But he had the more important value of allowing a 40-win team to break through 50-wins and above, by enabling stars around him to succeed.

And that leads us to one of the most interesting questions of basketball: How do you quantify value? The biggest misunderstanding of the analytics movement in sports is that “basketball people” feel like they’re being told what basketball is by “analytics people.” As someone who played the game (granted only at a high school level, but I’ve always played basketball) and still loves the analytics side, I think most people that are involved in analytics would tell you that they’re just trying to answer questions objectively. Are there elements of basketball, teams, and human interactions and behaviors that can’t be predicted and quantified? Absolutely!!! But anyone who is competitive is going to be looking for every edge they can find. For scouts, that might be finding a player with potential to develop skills that others don’t see. For the analytics community, that’s trying to use data to find information that isn’t visible to the naked eye.

Analytics are like a bikini, they show you a lot, but they don’t show you everything.

— Bob Myers, GM of the Golden State Warriors

So I absolutely agree with every skeptic that says that the game isn’t played on paper or that you can’t just shoot threes and layups and expect to win. Analytics isn’t about telling people what to do and how to do it. It’s about asking questions trying to learn more than your opponent to gain an edge.

And that’s what I hope to do here: ask questions. I hope that’s something that you can appreciate and enjoy. So with that, welcome to Basketball, Stat.

Hi, I’m Rob Antle

Welcome!

I wanted to do a quick post just to introduce myself and explain a little bit about why I’m doing this.

So to start with the most relevant thing about me, I love the NBA. Not only do I love the NBA, but I particularly love thinking about the NBA and all the different elements of it, from contracts and trades to x’s and o’s to stats and analytics. With that said, I’m particularly drawn to being able to clearly articulate and objectively answer questions that aren’t obvious to the average fan of the game.

Beyond that, my love of sports is not limited to the NBA. I’m also an avid football fan (and by “football” I’m referring to soccer, because why is American football not called “throwball” or something more sensible to how it’s played) and can honestly watch and enjoy any sport. Music is another passion of mine as I’ve played the drums regularly for over 20 years now.

I’m originally a Texas native and a Baylor grad (Sic ’em!), but I moved to the Bay Area a while back and have been here since. The Bay has become my home and my wife and I love it here. So much great food and drink, so much diversity, amazing weather, and beautiful surroundings have made it easy to call home. And that’s not to mention the amazing community of friends we’ve made since moving out here.

And now I’m at a crossroads, where I’m venturing out of my career I’ve known to pursue something that I’m passionate about. So if you happen to find yourself here somehow, I hope this gives you a little insight into who I am and why I’m writing. I hope you enjoy anything you find yourself reading here, and whether you do or not, I would love to hear from you! I love getting to know people, so please don’t ever hesitate to reach out.

Thanks for reading, and welcome to my small corner of the Internet I’m calling “Basketball, Stat.”

Design a site like this with WordPress.com
Get started