Pseudo-Polls

The term pseudo-poll refers to a number of practices that may appear to be legitimate polls but are not. A legitimate poll uses scientific sampling to learn about the opinions and behaviors of a population. Pseudo-polls include unscientific (and thus, unreliable) attempts to measure opinions and behaviors as well as other practices that look like polls but are designed for purposes other than legitimate research.

A variety of techniques are used to conduct unscientific assessments of opinion, all of which are considered pseudo-polls. These can be used by media and other organizations as an inexpensive way to measure public opinion and get the audience involved. However, they are very problematic from a data quality standpoint and should not be referred to as polls.

One common approach is the call-in poll. This is a method used when audience members or newspaper readers are invited to call a phone number and register their opinions. This is also to referred as a SLOP, a term coined by survey researcher Norman Bradburn, which stands for “self-selected listener opinion poll.” A similar technique is used by newspapers and magazines that publish questionnaires and ask their readers to fill them out and mail them in. A newer approach, called a log-in poll, is where Web sites post questions for people to click on to register their opinions, with the aggregate results being displayed instantly.

A major problem with all of these types of pseudo-polls is that the participants are entirely self-selected. Only those people who tuned in to that particular broadcast at that time or read that newspaper or went to that Web site can be included. Further, those who make the effort to participate are often very different from those in the audience who do not. Those who participate are often more interested in the topic or feel very strongly about it.

Another big problem with pseudo-polls is that they are open to manipulation by those with a vested interest in the topic. With call-in polls there is no limit on the number of calls that can be placed, so people can (and do) call multiple times and groups can set up elaborate operations to flood the phone line in support of their point of view. Manipulation is also a problem with Internet log-in polls. For example, multiple clicks can be possible, and it is common for political campaigns to direct their supporters to Web sites that have log-in polls to boost their position's and/or candidate's standing.

For these reasons, these types of pseudo-polls often produce biased results and should be ignored. Legitimate pollsters who are concerned with poll accuracy avoid these types of bias by selecting respondents using probability sampling techniques. Although survey researchers know that these pseudo-polls should not be taken seriously, many members of the public do not make a distinction between these and the real thing. In fact, pseudo-polls may be incorrectly seen as even more credible than real polls because they often have much larger sample sizes.

There are other types of pseudo-polls for which the purpose is not opinion collection. One of these is called a “push poll.” This is not a poll at all, but rather a sneaky and unethical telemarketing trick used by some candidates' political campaigns. Large numbers of people are called and are asked to participate in what appears to them to be a legitimate poll. In the process people are told negative, often false, things about the opposing candidates. The purpose is to shift or “push” people's opinions about the candidates rather than to collect data.

...