What's in a Combine?
Since 1985, the NFL Combine has taken place in late February, a few weeks after the big game. This year it began on February 17th. The Combine is a place where NFL executives conduct a multi-day interview to determine who they will draft in the upcoming year. During this time, collegiate football players partake in physical and psychological activities, including a short interview with NFL teams to make their case.
Many consider the most important information gained from this event to be the interview, psychological tests and, perhaps some show of physical talent. However, teams have already done their homework and studied game tape which becomes the predominant factor in their decision making. Specifically, they don't put too much stock on the physical tests, because results in the Combine are not strong predictors of success in the NFL (the counter factual is also true - having a bad Combine does not imply poor NFL performance). So...case closed?
Kind of. But since we have the data, let's see if we can predict whether or not a player will get drafted solely on their stats at the Combine. If we can get a reasonably working predictive model, we can use it to predict 2015 drafts!
First, some boxplots, because you know you wanna...
Distribution over time of the height of a players' Vertical - basically, how high they can jump straight up.
Looking at the Verticals for Quarterbacks, Running Backs, Corner Backs and Wide Receivers shows us that the data is pretty consistent over time, except for a few outlier players, which are to be expected. And 2008. I bet I know whose fault that was.
So given the consistency in athletic performance, controlling for position, can we build a model to predict if a player gets selected? I wouldn't be writing this thing if we couldn't.
I trained an ensemble of models* on data from 1999 - 2011, and attempted to predict whether or not a player would get drafted in 2012 and 2013**.
The first thing I looked for was significant variables. Turns out, that the only physical activity of any import is the Forty Yard Dash. One of my models, a Decision Tree, selects the variables which split the data into the most informative buckets:
The higher up the variable is in the tree, the more important it is in categorizing players.
This doesn't imply that having a good Forty will mean you get selected, but there's a definite correlation***.
For some positions like Wide Receiver and Corner Back (not shown below), the model categorized pretty well. For other positions, such as Quarterback and Running Back, the model did slightly better than guessing.
Models used in Confusion Matrices were, in order, Logistic Regression, Logistic Regression, Random Forest
As you can see, drafted and undrafted Running Backs and Wide Receivers were more correctly classified, compared to Quarterbacks.
Visually, we can see how well a model can separate draft picks correctly using our trusty old ROC curve (hover over lines to see the position in question):
Poor Quarterback classification could be because QBs play a very special role on the team where raw athletic talent is not nearly as important as all the intangibles they bring to the team (see link above for Brady, Tom). And good looks aren't tested in the Combine.
Finally, you might think, like I did, that if Combine metrics weren't very predictive, then perhaps you could just come up with some simple heuristic to separate picks. For example, one such heuristic could be that players with (a) Higher Verticals, or (b) Faster Forty's than average, get picked. Those that fall below average, do not. Here's how such a model fares:
Just better than guessing.
Some might say this kind of simple heuristic model isn't all that bad, given how unsophisticated and quickly we put it together. This kind of model might come in handy at a party when you're making bets and need a smart decision tool very quickly (an all too common scenario). The Data Man, however, has time for models. Especially when they outperform heuristics. And it's these models that I'll use to make predictions for the 2015 draft using this years Combine.
Stay tuned for the results!
*The ensemble included a decision tree, k-nearest neighbor, logistic classifier and a random forest classifier. See here for code!
**My dataset didn't have up to date draft picks for 2014
***In addition to decision trees, I looked at the p-value on a logistic regression and it was significant as well
