Over the first 5 weeks of the NFL season, the Kyle Orton-led Denver Broncos struggled to a 1-4 start. Then, yielding to fan pressure, they inserted Tim Tebow at quarterback and went 4-1 over the last 5 games in rather spectacular fashion, producing a flurry of ridiculous media coverage, spurring Denver fans to start wearing Jesus jerseys and creating a whole narrative around Tim Tebow’s Miracles. We thought we’d join the fray and quantify just how much of a miracle Tebow’s success has been.
The Kyle Orton era in 2011 was a rather unfortunate one for Denver. Given their points scored and points allowed, a Pythagorean expectation would predict them to win only 36% of their games, and with some close losses, they unsurprisingly fell to 1-4. After starting Tebow and winning 4 of 5, you might expect that the team’s performance had improved drastically, but that’s actually not the case. Tim Tebow’s Expected Points Added (EPA – AdvancedNFLStats.com’s measure of the points expected to come from the plays that a player executes) over his 5 games is more than 13 points below Kyle Orton’s. The Denver running backs’ and wide receivers’ EPA per game also dropped from the first 5 games to the last 5 games, from 0.5 to -3.8 and from 1.5 to 0.4 respectively. The only thing that has improved is the defense, and it hasn’t been by much – Football Outsiders’ Defense-adjusted Value Over Average (DVOA) for Denver’s defense, a measure of the effectiveness of a defense, has gone from an average of 9.5% to 8.7% (lower numbers, including larger negative numbers, are better in defensive DVOA). Given all of this, you would expect the Broncos’ probability of winning to drop even further – had they exhibited that performance over the first 5 games, their expected winning percentage would have dropped to about 20%. Tebow’s success, then, sounds more and more like a miracle.
And just how spectacular a miracle? Using Jeff Sagarin’s ratings for the last 5 weeks, which take into account strength of opponent and location of the game, the Broncos would have been expected to win 1.9 games over that 5 game stretch. So, it appears that Tim Tebow has (miraculously?) pulled an extra 2.1 wins out of thin air.
How can this be? Essentially, while Tebow and the rest of the Broncos have not been producing better numbers, they’ve been producing their best numbers at the right times. This can be seen in the difference between Expected Points Added and Win Probability Added (WPA) for Orton and Tebow. Win Probability Added is AdvancedNFLStats.com’s measure of how much a player’s plays have contributed to the team’s chance of winning the game. For instance, a 20 yard touchdown pass to take the lead as time expires will be worth much more WPA than a 20 yard touchdown pass to take the lead in the 1st quarter because, in the second scenario, there is still a significant probability that the opponent comes back to win. Tebow has a much worse EPA – meaning that he has objectively produced worse results over all of his plays – but a significantly better (though still negative) WPA – meaning that his plays have contributed more to the chances of his team winning. While Tebow’s plays have produced much fewer expected points than Orton’s, Tebow’s positive plays have come at crucial points in the game, when they have a much larger impact on the outcome: think his 20 yard TD run for the lead with 58 seconds left last week, or his 56 yard TD pass for the lead with 6:44 left the week before. Likewise, Tebow’s negative plays have come at points where their effect was less harmful. Similarly, the Denver running backs have also been exceeding their expected WPA over the past 5 games. Is this just coincidence or clutch play and strong leadership by a guy who “just wins football games”? The jury is still out for Tebow, but history does not bode well for Broncos fans.
The vast majority of players who started their careers with similar characteristics (a relatively high WPA given their EPA) eventually regressed towards the mean and saw their numbers come in line with expectations. A very clear relationship can be seen between a player’s WPA and EPA. The chart below shows the EPA and WPA of all seasons for all quarterbacks that have played in the league this year. Using a regression to quantify this relationship, we can then say what a quarterback’s expected Win Probability Added should be for the Expected Points Added during their play – or in other words, how much they should be expected to affect their team’s wins with their plays on the field, independent of timing during the game or score of the game when the plays occur. Tebow’s WPA so far this season is 0.39 higher than one would expect given the Expected Points Added by his plays. Other quarterbacks who started in a similar situation could not sustain this over multiple seasons. The graph below shows the average difference between actual WPA and expected WPA for all current quarterbacks who started their careers like Tim Tebow – with a significantly higher WPA than expected (we took all QBs who had a difference greater than or equal to Tebow’s in their first year attempting over 100 completions – 10 QBs fit the bill and had more than 1 season of play; interestingly this group included top QBs Aaron Rodgers, Tom Brady, Drew Brees and Eli Manning). Without fail, these quarterbacks saw their play converge towards the expected value over the long term (in case you were wondering, the spike in year 3 is due to exceptional seasons by both Tom Brady and Matt Ryan).
We can also look at another highly successful, mobile college quarterback with mechanical issues who succeeded early in the league, Vince Young. Young was dubbed a “winner” when he came into the league and got off to a solid 8-6 start as a starter in his first year. But after initially affecting his team’s wins much more than you would expect in his first year, his contribution to wins became much more in line with his play over time, as depicted by the red line in the above graph. After going 9-6 in his second year, he was replaced by Kerry Collins in his third year and came under fire from the media for reported motivation issues. Of course, Young had an outlier year in his 4th year, replacing Kerry Collins mid-season and ripping off a 7-3 record in his 10 games despite fairly average numbers. Accordingly he was dubbed a winner yet again, much like some members of the media are already doing this year. Regardless of the media representation of both players, the likelihood of Tim Tebow sustaining his current winning percentage looks rather dire.
Now before you Broncos fans get too angry, these numbers only argue that Tebow’s wins are more likely to become more in line with his play – if his play improves he’s likely to keep winning at a good rate. Or on the other hand, if you believe that Tebow is truly a clutch player that performs better in high impact situations, then he will sustain a higher winning rate with poorer play. But given that Tom Brady and Aaron Rodgers haven’t been able to do that over the long term, I wouldn’t bet the farm on it.
No comments:
Post a Comment