« China Responds To Green People Power. | WILLisms.com | Happy Easter. »

Manufactured Polling Outcomes.

WILLisms.com has commented quite a bit on polling over the past couple of months.

One common thread in much of our analysis is the existence of push-polling and other manipulative forms of opinion polls that guarantee outcomes. There are many ways, some more under-the-radar than others, to rig a poll.

Push polls are simply polls with extremely leading and biased questions, usually with innuendoes. In the Social Security game, offering a litany of (mostly misleading) negative information about how hard reform would be an example of push polling. This technique has obvious potential to engineer certain opinion outcomes, and because few people understand how polling really works, few people are willing to stand up and challenge the "will of the people," these "oracles of truth."

Generally the term "push poll" is reserved for the most egregious kinds of rumormongering, even fraud, but it also seems perfectly applicable to some of the polling commissioned by the elite media in recent months. The media may take offense to that, but that's tough for them. They deserve to be called out for some of this recent garbage they have been passing off as "public opinion polling."

But push-polls are just one form of polling that ought to be discarded out of hand. Polls that sample too many or too few women, union members, Democrats, minorities, rich people, urbanites, young people, or other demographic groups can lead to wild divergence from the real collective opinion of the public. For example, if union members make up 15% of the population in a Congressional district, but the poll sample is 40% union, the results are obviously going to be incorrect. This will not necessarily show up in the poll's headline, either.

Polling is crack cocaine to the media; they are willing to pay just about anything for it, because consumers of news simply cannot get enough of them. Polling, however, is notoriously bad at measuring overall levels of public opinion; it is even worse at predicting close election outcomes. Polls are far better at measuring differences and similarities between and among demographic groups. When the media are exposed for their bad, biased polling, they almost always respond with some form of:

"...it's just a poll. Are you calling the opinions of the American people biased?"

And they normally get away with it.

Organizations, even "non-partisan, non-profit" ones, engage in shameful polling practices, as well.

We've noted AARP's manipulative polling (also here).

AARP taught allied organization Rock the Vote a thing or two about manipulative push-polling, as well.

Recently, we've tried to unlock the mysteries of Social Security polling.

Polls are used all the time by the elite media to frame their stories. They often manufacture polls, quite overtly, to fit their existing narratives.

John Hawkins explains how the latest round of manufactured polls, used by the elite media to prove its narrative that Americans are angry that Bush intervened in the Terri Schiavo situation, are bunk (via PoliPundit):

...if you keep reading you find out the real cause of the dip:

"The poll also found an increased number of Democrats. In this survey, 37% said they were Democrats and 32% said they were Republicans. Last week, 32% said they were Democrats and 35% said they were Republicans."

So Bush's numbers dropped 7% in a week. However, there were 5% more Democrats polled and 3% less Republicans. Had they used the same percentages as the week before, it's entirely possible Bush's polling numbers would have gone up just a bit.

This happened quite a bit during the months prior to the 2004 election. The party ID samples would fluctuate wildly, week-by-week. They almost always overestimated the numbers of Democrats in the sample. At one point, moveon.org ran a full-page ad (it's a .pdf) in the New York Times complaining that Gallup was oversampling Republicans; Gallup promptly adjusted its poll samples to include more Democrats. Polling pundits like Ruy Teixeira, of Emerging Democratic Majority infamy, called moveon.org's ad "rock-solid."

Turns out, Gallup, and every other polling institution that sampled more Democrats than Republicans, were wrong; the only poll that matters, the one in November, proved that point. Pollsters in 2004 were so surprised by the election outcome that it seems like they must have missed the memo on that whole GOP realignment thing.

When it comes right down to it, we live in a poll-driven era, but even many professional politicians do not understand how polls work. Many Congressional Republicans, for example, buying into the poor polling on Social Security, fear for their political lives.

One other important aspect of polling is that it tends to have an echo effect, meaning the public wants to be part of the mainstream, part of the winning team, part of the bandwagon. Thus, if a candidate "surges" in the polls in the final days before an election, simply because pollsters adjusted party sampling ratios, the (small number of) remaining undecided voters may actually hop on board the cool-kid bandwagon. The same thing goes for individual issues in non-election years, which is what makes some of the Social Security polling so frustrating.

What to take from all of this:

1. Polling in the media today is scandalous. Yes, scandalous. But very few are willing to call it such.

2. Never accept a poll's headline at face-value. If a poll does not list how many Republicans and Democrats it sampled, throw it out.

3. Throw out polls with loaded questions. Throw out polls with a single night of surveying. Throw out polls with samples under 600 people.

4. Understand that the media will continue to run biased polls, but it is the duty of the blogosphere to call out poor polls until the media collectively shapes up its act.

5. Included in any poll really needs to be a full disclosure of question wording, question order, demographic samples, and other pertinent information. Without all that, a poll is meaningless.

Linked with the Traffic Jam.

Posted by Will Franklin · 27 March 2005 05:36 AM

Comments

Posted by: Zsa Zsa at March 27, 2005 11:16 AM

Posted by: monika at March 27, 2005 06:45 PM