- Kevin DeYoung - http://www.thegospelcoalition.org/blogs/kevindeyoung -

The Polls Tell Us Less than We Think

Posted By Kevin DeYoung On November 5, 2012 @ 6:59 am In Uncategorized | 10 Comments

I have a bold prediction for Tuesday: almost all the presidential polls out there (and there are LOTS of them) will prove to be accurate. And yet at this moment, no one really knows who will win.

The factor in these polls that most people forget about, or fail to understand, is the statistical concept “margin of error.” Most Americans give too much credence to surveys (64% of likely voters know that). It’s not that the polls are rigged (at least not most of them). The polls are often found to be “inaccurate” because we expect them to be more “accurate” than they claim to be. Every presidential poll has a margin of error (MoE), which is the pollster’s way of saying “here’s the statistical range we feel confident about.” But if we ignore the MoE, or misapply it, we will not know what the poll is really saying.

For example, let’s look at the latest presidential polls [1] taken in Michigan:

Baydoun/Foster – 1913 LV, 2.2 MoE, Obama 46, Romney 47
PPP – 700 LV, 3.7 MoE, Obama 52, Romney 46
Rasmussen – 750 LV, 4.0 MoE, Obama 52, Romney 47
Detroit News – 600 LV, 4.0 MoE, Obama 48, Romney 45
EPIC-MRA – 600 LV, 4.0 MoE, Obama 48, Romney 42

The margin of error indicates the range (plus or minus the given number) that will be true 95% of the time. The 95% figure comes into play, though it rarely gets mentioned, because it is the standard confidence interval. It means, in the Detroit News poll for example, that if they ran the same survey 100 times, in 95 of those times Obama would get 48% of the voter preference, plus or minus 4 points. This indicates the range of percentages about which the pollster feels confident. The Detroit News is confident (or at least was when the poll was taken) that somewhere between 44% and 52% of the likely voters prefer Obama. Or to put it another way, they think 19 out of 20 polls (i.e., the 95% confidence interval) will have Obama’s percentage between 44-52 and Romney’s between 41-49.

Make sense? I hope so because I want to make a few other important observations about MoE.

1. The margin of error does not refer to the gap between two candidates in the polls but to the polling numbers for each candidate. The +/- range is for the percentages themselves, not for the difference between the two percentages. So if Romney leads Obama by 6 percentage points in Florida (51-45 in a recent poll [2]) and the margin of error is plus or minus 3.5, the poll does not give Romney a decisive edge. True, it’s better to have the polls say you are ahead than behind, but what the poll indicates is that 95% of the time this survey should show Romney with 47.5-54.5 percent of the vote and Obama with 41.5-48.5 percent of the vote. The MoE refers to the percentages, not to the gap. So according to this Florida poll Obama might squeak out a 1 point win in Florida or Romney could cruise to a double digit victory.

2. Remember that the MoE has a confidence interval of 95 percent. This means 1 out 20 times you are going to get a strange outlier. If you’ve seen a wild poll or two in this election season that’s way out there compared to almost every other poll, that’s what pollsters expect. Five percent of the time the poll will be really different, not necessarily wrong, just really different.

3. Because of the MoE, polls tell us less than we think, even though candidates want to make more of them than they should. Look again at the Michigan polls above. The top one is the most recent and the bottom one less recent (in this case, only a week earlier). The Romney camp could say, “Look, we have the momentum in Michigan!” On the other hand, the Obama camp could say, “Look, we’ve been ahead in this state for months and every other poll is in our favor!” In reality, the polls say that the state is too close to call. In none of these polls is Obama’s range without overlap with Romney’s range. Obama fans should not think they have Michigan in the bag. Everyone of these polls says that Romney could actually be ahead of Obama. And yet, Romney fans should simmer down too. The latest poll with Romney up by 1 point is exactly what every other poll said was possible 95 times out of 100. It may not indicate momentum at all.

All this is to suggest that in this season’s plethora of polls pundits have been too quick to make big conclusions and rewrite campaigns narratives based on a recent poll which, given the margin of error, may say pretty much the same thing as the last poll.

The national polling indicates a dead heat, and in 10 or 12 battleground states the polls are too close to call. You can make a case that Obama will win handily, that Romney will make a good showing in several blue states but hasn’t broken through in the polls and will fall short in almost all of them. You can also make the case for a big Romney win, that last minute voters will break for the the challenger and Obama will lose almost every state where his numbers have been below 50 percent.

And it could be as close as Bush-Gore in 2000. Tuesday will tell.

Unless it really is Bush-Gore all over again.


Article printed from Kevin DeYoung: http://www.thegospelcoalition.org/blogs/kevindeyoung

URL to article: http://www.thegospelcoalition.org/blogs/kevindeyoung/2012/11/05/the-polls-tell-us-less-than-we-think/

URLs in this post:

[1] latest presidential polls: http://www.realclearpolitics.com/epolls/2012/president/mi/michigan_romney_vs_obama-1811.html

[2] recent poll: http://www.realclearpolitics.com/epolls/2012/president/fl/florida_romney_vs_obama-1883.html

Copyright © 2009 Kevin DeYoung. All rights reserved.