Here’s Why All the Polling Models Are Probably Right

Let our journalists help you make sense of the noise: Subscribe to the Mother Jones Daily newsletter and get a recap of news that matters.

Paul Waldman describes the Bizarro-world war on polling guru Nate Silver that’s gained steam over the past week:

In the last few days, we’ve seen a couple of different Silver narratives emerge as attention to him has increased. First, you have stories about how liberals are obsessing over Silver, “clinging” to him like a raft in a roiling sea of ambiguous poll data. Then you have the backlash, with conservatives criticizing him not because they have a specific critique of the techniques he uses, but basically because they disagree with his conclusions….Then you’ve got the reporter backlash. At Politico, Dylan Byers raised the possibility that Silver would be completely discredited if Mitt Romney won, because “it’s difficult to see how people can continue to put faith in the predictions of someone who has never given that candidate anything higher than a 41 percent chance of winning.”

This whole thing is deeply weird. Could Nate be wrong? Sure. Ditto for Sam Wang and Drew Linzer and all the rest of the poll modelers out there. But if they’re wrong, it probably won’t be because of their models. After all, with minor differences they all do the same thing: average the publicly available state polls, figure out who’s ahead in each state, and then add up the electoral votes they get for each candidate. Sure, they all toss in a little bit of mathematical secret sauce, but not really all that much. You could do the same thing if you felt like it. Want to know who’s ahead in Ohio? Go add up the five latest polls and then divide by five. Voila. You are now your own Nate Silver.

Needless to say, though, the poll modelers are only as good as the polls they use. If the pollsters are systematically wrong, then the models will be wrong. And while there are a few small sources of potentially systematic bias (not calling cell phones, demographic weighting, etc.), by far the biggest is the pollsters’ likely voter screens. But even here, with one or two exceptions, this is pretty simple stuff. Most pollsters just ask a question or two that go something like this:

  • Are you planning to vote?
  • How sure are you that you’ll vote?
  • Really? Honest and truly?

That’s about it. If you tell them you’re highly likely to vote, they mark you down as a likely voter. If not, they don’t. There’s no rocket science here.

So if the modelers are wrong, it will probably be because the pollsters were systematically wrong. And if the pollsters are systematically wrong, it will probably be because this year, for some reason, people started lying about their likelihood of voting. And while anything’s possible, I sure can’t think of any reason why this year there would be a sudden change in how truthful people are about their intention to vote.

That’s what this whole controversy comes down to. Conservatives seem to be convinced that Democrats simply won’t turn out in high enough numbers to reelect Obama. A fair number of liberals fear the same thing. But there’s no analytic reason to believe this. The Obama campaign’s ground game seems to be as good as any in the business, and Obama voters are telling pollsters that they’re likely to vote in big enough numbers to give him the key swing states he needs to win. That’s the current state of our knowledge. It might be wrong, but if it is, the question isn’t going to be why Nate Silver went astray. The question is going to be, why was 2012 the year when people suddenly started lying to telephone pollsters?

UPDATE: Asawin Suebsaeng has a roundup of all the prognosticators here. It’s a nice, Cliff Notes version of who’s who and what they’re saying.


Headshot of Editor in Chief of Mother Jones, Clara Jeffery

It sure feels that way to me, and here at Mother Jones, we’ve been thinking a lot about what journalism needs to do differently, and how we can have the biggest impact.

We kept coming back to one word: corruption. Democracy and the rule of law being undermined by those with wealth and power for their own gain. So we're launching an ambitious Mother Jones Corruption Project to do deep, time-intensive reporting on systemic corruption, and asking the MoJo community to help crowdfund it.

We aim to hire, build a team, and give them the time and space needed to understand how we got here and how we might get out. We want to dig into the forces and decisions that have allowed massive conflicts of interest, influence peddling, and win-at-all-costs politics to flourish.

It's unlike anything we've done, and we have seed funding to get started, but we're looking to raise $500,000 from readers by July when we'll be making key budgeting decisions—and the more resources we have by then, the deeper we can dig. If our plan sounds good to you, please help kickstart it with a tax-deductible donation today.

Thanks for reading—whether or not you can pitch in today, or ever, I'm glad you're with us.

Signed by Clara Jeffery

Clara Jeffery, Editor-in-Chief

payment methods

We Recommend