Sunday, November 15, 2009

Don't Lose Your Edge, Bill!


Predictably, the talking heads have already begun raking Bill Belichick over the coals for his decision to go for it on 4th-and-2 from their own 28 yard line with 2:08 on the clock. However, if you look at the math, the Pats' head coach appears to have made the correct decision.

The following paragraph has been edited for clarity, in addition to flipping the sign in the equations written to reflect what I actually wroete

On 4th down, with 2 yards or fewer to go, New England has gained a first down on approximately 66% of its attempts with Tom Brady as quarterback. The Colts had no timeouts one timeout. If the Patriots gain a first down, the game ends; they can simply kneel on the ball three times slowly walk to burn a few seconds, then take a knee on each down to end the game. If they don't gain a first down, the Colts would still need to score a touchdown to win the game. Let's give the Colts a probability P of getting the six if the ball starts at the 28 yard line. So if the Patriots try for the first, their chance of losing is
(Probability of 4th down failure) x
(Probability of Colts scoring a TD from the 28 Yard line) = 0.33P
The average New England punt nets about 40 yards. Let's give the Colts a probability Q of scoring a TD on a driving starting at the Indianapolis 32. Then, the chance of the Patriots losing is simply Q. For Belichick's decision to make sense, we just have to believe that he gave his team a higher chance of winning lower chance of losing. In math terms, that would mean 0.33P > Q 0.33P < Q. Doing some algebra leaves you with P > 3Q P < 3Q. In other words, for the Patriots to have made the right decision, we only have to believe the Colts odds of scoring a TD on a drive starting 28 yards from the end zone are less than three times the odds of the same outcome starting from 68 yards out. The win probability graph for the game suggests that, given 1st-and-10 from New England's 29, the Colts had roughly a 51% chance of winning in the actual situation. We have to believe that their chances under the punt scenario were below 17% above 17% for Belichick to have made a bad good decision. Considering the Colts' have scored touchdowns on 30% of their offensive possessions, my guess is that this was a good one.

Tom Brady is sitting in the postgame press conference doing yeoman's work trying to explain this to a roomful of stunned sports reporters in a language that they can understand. I don't envy him in this moment.

Update: Okay, the Patriots made one error
3rd and 2 at NE 28
(Shotgun) T.Brady pass incomplete short right to W.Welker (J.Powers).



Timeout #3 by NE at 02:08.


4th and 2 at NE 28
(Shotgun) T.Brady pass short right to K.Faulk to NE 29 for 1 yard (M.Bullitt).

The punt team should not have come on the field, forcing the Patriots to call a timeout. With the time out in hand, New England could have challenged the spot on the 4th down play, or allowed the Colts to score quickly so they could try for a comeback field goal (speaking of which, why didn't they let the Colts score a TD on the first play from scrimmage so they could try for a comeback field goal).

28 comments:

Neil Sinhababu said...

Wait, were the Colts out of time outs? This thing says otherwise.

Nick Beaudrot said...

Yeah, now I'm confused. I could have sworn the people on the hilight said they had used all their timeouts

Unknown said...

The Colts had 1 more timeout.

Neil Sinhababu said...

Now someone just needs to post that they had 2 timeouts and we'll have all the possibilities covered.

Unknown said...

Also, you have your greater than switched. For his decision to make sense, you want 0.33P < Q. (lower chance of losing by going for it vs punting).

Anonymous said...

The 4th down play brought it below 2:00, you can't challenge plays inside the 2 minute warning.

Anonymous said...

The logical error here is combining a probability averaging Pats fourth-down conversions over a (many?) season(s) with a probability averaging Colts offensive success over a single game. You need to use a consistent scheme.

Unknown said...

The 4th down play brought it below 2:00, you can't challenge plays inside the 2 minute warning.

Actually, because the 4th down play started before the 2-minute warning, it is not booth-reviewable, and must instead be challenged by the coaches. It would have been better for the Patriots had that not been the case: with under two minutes left, the decision to review is left to the officials, and they almost certainly would have. Then again, it's not clear that they would have overturned the call.

Anonymous said...

The fourth down play could have been challenged because it occurred prior to the two minute warning, despite the fact that there was 1:58 on the clock after the play. Had the Patriots a time out, they would have been able to challenge the spot. This is also the reason there was no talk of a booth review.

Justin Tiehen said...

The 66% chance of making 4th and 2 seems pretty disputable. Even if that's how they' performed historically, I'm sure that in most prior circumstances the opposing defense had some incentive not to give up 2 yards, but also plenty of incentive to prevent a bigger play, leading them to call defenses that gave the Pats a better chance of converting but a worse chance of going big. This would be true of nearly any 4th and 2 earlier in the game (including much of the 4th quarter) and most 4th and 2s where New England is losing and making a late drive. But those are the vast majority of your historical 4th and 2's.

If you're looking for an alternative probability of converting, why not take the probability of converting a 2 pt. conversion, which is 44%? Whether the Pats gain 2 yards in that scenairo or whether they gain 70 is all the same to the Colts D, in which case they should play Pats receivers exceptionally close, sell all out, etc., just as in a 2 point conversion scenario. In fact, I even suspect 44% is a little high, but I'll start there.

Nick Beaudrot said...

"The logical error here is combining a probability averaging Pats fourth-down conversions over a (many?) season(s) with a probability averaging Colts offensive success over a single game. You need to use a consistent scheme"

The 30% number is from this season. If I use 2008-2009 I still get about 30%.

Dan Schwab said...

I agree with everything Justin Tiehen said. I was about to write basically all of that but he did it already.

Unknown said...

So... I'm pretty sure you set up the inequality the wrong way there and therefore this whole analysis is backwards. To see it clearly, imagine the probability that the Patriots will fail on 4th down is 0. The Patriots will definitely win in that scenario, and 0P (in place of .33P) would then also be 0, which is necessarily less than the finite probability that the Colts would score if punted to (Q). That is, (chance of not making 1st down)*P must be LESS than Q in order for the decision to go for it to make sense. Where you went wrong was when, despite calling .33P "[the Patriots'] chance of losing [if going for it]" you said having a "higher chance of winning [if going for it]" meant maximizing P.

Unknown said...

sorry, should have said "meant 0.33P had to be greater" instead of "meant maximizing P"

Nick Beaudrot said...

The 2-point conversion isn't quite a good comparison. The playbook in the red zone has to be different. In theory Belichick could have drawn up a play that puts Wes Welker on a go route; that option wouldn't be available in the end zone. The defense can adjust accordingly.

Even if use 44%, Belichick improved his chances of winning from 70% to 71%.

If we use the league-wide 4th-and-2 average outside the red zone, which is 56%, he improved the team's odds from 70% to 79%.

In other words, at worst, it was a wash, and at best, he increased his teams chance of winning.

Unknown said...

I do, however, agree (along with Nate Silver) that the choice was the right one. You just made the right conclusion using the wrong analysis.

Charles Kuffner said...

"speaking of which, why didn't they let the Colts score a TD on the first play from scrimmage so they could try for a comeback field goal"

Boy, just try imagining Tom Brady explaining that to a bunch of stunned sportswriters in language they could understand.

Nick Beaudrot said...

"speaking of which, why didn't they let the Colts score a TD on the first play from scrimmage so they could try for a comeback field goal"

Boy, just try imagining Tom Brady explaining that to a bunch of stunned sportswriters in language they could understand.


This is on a day that MJD was praised for taking a knee at the 1! while his team was still losing!.

Though I cede your point: "yeah, coach said we should go for it on 4th, and if we don't get it, the D should let the Colts score, and then we drive the field". It's entirely the correct thing to do. But no one would have believed him. And, his defense might have just decided to stop playing the entire season.

Andrew said...

You can actually build a useful decision tree to structure the problem. One thing that many analysts have pointed to is how easily the Colts scored with 1:57 on the clock, but the Colts were also concerned with not leaving the Patriots time to mount their own scoring drive. They might also have benefited from a good run back.

Punting was all but certain to give the Colts the ball: by going for it, the Pats gave themselves a chance of denying their opponent an offensive snap.

Still, this wasn't a sure thing even had Faulk not bobbled the catch. Had NE converted, there would have been an immediate timeout for the two minute warning; then there would have been a Colts timeout after a short run. There could well have been a punt with 1:10 to go, reproducing the same situation but with an extra 6-10 yards and 50 fewer seconds on the clock.

My own take on the math is that the "go for it" decision was the right one. The main variable which causes the decision to flip is the 4th down conversion percentage, and Belichick was right to make the call if the anticipated success rate was above 50%.

However, Keynes was on to something when he asserted that it's better to fail conventionally than succeed unconventionally.

Nick Beaudrot said...

Andrew, I think your end game is a bit off. The play clock is 40 seconds. The Colts had 1 timeout.

1st and 10, Pats line up in the victory formation with 1:57 remaining; Brady runs backwards for five yards and takes a knee. Colts Timeout at 1:50.
2nd and 15 (1:50), Brady kneels (Colts realize they need to force him to kneel sooner; Brady knows the Colts know this and does it anyway).
3rd and 16 (1:10)
4th and 21 (0:30) punt. The punt takes about 6 seconds between the snap and the landing. That puts the Colts at their own 35 with about 24 seconds left. They have time for two plays to the sideline, then a hail mary; or one play to the middle of the field, then a hail mary. They are in real desperation territory at this point.

JRoth said...

OK, Nicholas, but "real desperation territory" is not the value you gave to a Colts win in the "Pats make the first down" scenario; you gave it a value of 0. So, in fact, you need to take that .33P and ADD the likelihood of the Colts scoring from their own 35 with half a minute to go (and I think that has to be more than a few percent, much as you handwave it away - I'm pretty sure that HoF QBs are able to do that sort of thing against on-their-heels defenses).

Combine that with the dubiousness of your 66% number, and I think it's much closer than you're allowing. I'm not saying Bill was wrong, but your smug certainty is awfully off-putting.

(Am I wrong, or does your analysis rest on the idea that the Colts have a 51% chance of scoring from the Pats' 28 and a ~30% chance of scoring from their own 32? That doesn't pass the smell test, but maybe you can convince me)

JRoth said...

And I just reviewed the Advanced NFL Stats post, which hinges directly on the relative probabilities between 28 and 68 yards out. First of all, they, too, treat as 0 the probability that the Colts can score in the last half-minute. More importantly, they argue that, if Manning can score better than league average from the 28 (ya think?), then he must be just as equally likely to score from the 68 (if you will). Hand-waving in place of analysis, and why? Because it's in support of an action they like (and, to be clear: I like going for it on 4th down as well; that doesn't actually mean it's always the right move).

Basically it hinges on 2 things: the 3X multiplier and the value you attach to Manning with the ball and half a minute left. The other stuff is important, but as long as that 3X multiplier holds (and you don't have to add on Peyton's hail mary likelihood), it's small potatoes.

Nick Beaudrot said...

First off, there are other things the Pats can do to consume more time. They could design more creative kneeldown plays that consumed a few seconds. They could fake an injury which forces runs 10 seconds off the clock since they have no timeouts. The could intentionally false start right before the play clock ends, which runs time off the clock.

Second, I'm pretty sure the chance of the Colts winning in the event of a go-for-it scenario are ... I dunno, 1 percent? I re-ran the analysis and it doesn't make much difference until you get up to numbers like 5%, which I find dubious (when was the last successful end-of-game hail mary?)

Third, the analysis doesn't hinge on the exact numbers. It lowers the multiplier from 3x to something like 2.5x. Unless you believe the gap between scoring odds on short field versus long field it's almost certainly is just wildly different, to the point where we have to believe the Colts chances on the short field are nonsensically high (i.e. above their red zone TD percentage mark), or their chances of scoring on the long field are nonsensically low (to the point where they would need a short field to score at all), or both.

JRoth said...

First off, there are other things the Pats can do to consume more time

Show me that they ever have, or don't mention it. You rarely see teams even do something so simple as retreat 5 yards to gain a few seconds, much less "design more creative kneeldown plays."

(when was the last successful end-of-game hail mary?)

But we're not talking about a simple "hail mary" - we're talking half a minute, which is ample time for 4-6 passing plays. If he manages a couple ~20 yard sideline passes, he gets basically 4 shots at the end zone from the ~30 yard line. I wouldn't say that adds up to 5%, but it's not 1% either.

You're saying that the numbers can't move enough, but I'm not sure why you get to declare that the Pats' chances on 4th down are, at worst, 60% (that's 2.5x). Drop it to 50-50, and now you're at 2X. Bump Peyton Manning's short-field rate against a tired defense that has played badly for the past 20 minutes up to just 56%, and now his rate from 68 yards needs to be just 28% to make it a wash. Give him a 2% chance of scoring in a first down situation, and he only needs to be 26% at 68 yards. Actually, if you give PM 60% chance of scoring from the 28, and a 2.5X rate for TB, then PM only needs to be 24% from 68 yards for it to wash (note: numbers done on the fly and in my head; sorry in advance if I've screwed up).

Point being not that Bill was clearly wrong; point being that it was close, not a slam dunk that only stupid reporters could fail to grasp. And it does, in fact, hinge on Belichick's arrogance about his ability to get a 1st down there (I'm sure he felt that it was 66% or better - doesn't actually mean he was right).

Nick Beaudrot said...

It's 24 seconds, not 30, since the punt takes 6 seconds to execute. Also at this point the timeoutless Colts *have* to throw to the sidelines, which means the Patriots can plan the D accordingly. I think you can safely budget for two completions.

I've had this argument with several people, and the main difference is that the pro-punt people seem to highball Manning's chances of scoring a TD on a short field. The Colts' 1st down conversion rate is just under 80%. To get to the 60% figure you need just under 90% chance of getting a first down (assuming some small but nontrivial chance of a TD without a first down). But, any situation where their offense is that good would mean it's that good in the long field, which pushes back in Belichick's direction.

Likewise, the "tired defense that has played badly for the past 20 minutes" argument would also increase the Colts' chances of scoring in the long field. Which also pushes back in Belichick's direction.

Nick Beaudrot said...

To clarify ... two completions before you have chance for only one more play. Maybe three if you're extremely lucky.

Anonymous said...

bunch of donk computer programmers/math majors explaining why their coach was right in calling a play that effectively ended their team's chance of winning. i am now dumber for reading this drivel. The coach is paid to make play calls. Sometimes you win, other times you lose. The End.

Anonymous said...

Indianapolis Colts > Pats and Who DAT > Pats. (the symbols here are CORRECT!!!)