It is one of those remarkable examples of American innumeracy and general intellectual apathy that the term "margin of error" is tossed about so consistently improperly as it is on political shows discussing polls. A typical conversation will go like this:
"Obama leads McCain 47% to 45% with a 3% margin of error, so this race is a statistical dead heat"
Gar-bage. The margin of error describes the range about the mean within which we can be certain, with some arbitrarily chosen level of confidence (usually 95%), that includes the actual figure being estimated. Using the example above, a 3% margin of error means that there is a 95% chance that Obama's actual figure will be between 50% (47+3)and 44% (47-3), and a 95% chance that McCain's actual figure will be between 48% and 42%.
Knowing that, what are we to make of this term "statistically tied"? For that matter, what about those prognosticators that act like being "within the margin of error" has some special meaning? In both cases, the speaker is simply ignorant. Having a difference between the estimated means that is smaller than the margin of error means nothing in and of itself. McCain's chances of winning are not dramatically higher being 2.9% behind vs 3.1%.
The relevance of the margin of error comes in calculating exactly what the probability of victory is, which is beyond the interest of most poll followers. About the only value it has for them is in comparing polls. A candidate with a 2% lead in a poll with a 3% margin of victory is more likely to win than one with a 2% lead with a 5% margin of victory. That's it.
So pay no mind to the pundits who obsess over whether a poll is "within the margin of error". It's meaningless.
Subscribe to:
Post Comments (Atom)
3 comments:
Margin of error does have more meaning when it comes to seeing whether or not support for a candidate has changed.
Example: if BHO was at 48 yesterday and 49 today and the margin of error is 3 percent, it is baseless to say that he has gained support; one can conclude almost nothing.
Of course, 3-day trackers help smooth things out a bit.
Not true. In your scenario, we'd have gone from being 95% confident that his true figure was between 45% and 51% to being equally sure it is between 46% and 52%. Now granted that isn't exactly earth shattering, but it isn't baseless or nothing either.
"practically baseless". I know, I know; one would have to calculate a conditional probability, but the p-value of
Ho: Obama's support hasn't changed vs.
Ha: Obama's support went up
would be rather inconclusive, to say the least. :-)
I understand the point that you are making; my guess is that you would really enjoy this blog post Mano Singham's blog
It gives a cool table that gives you the conditional probability that candidate X is ahead given that X's lead is n points and the margin of error is m points.
Post a Comment