Call-in Polls

A call-in poll is an unscientific attempt to measure public preferences by having radio or television audience members or newspaper readers call a telephone number and register their opinions. Usually a single question is posed, and people are asked to call one phone number in support of a viewpoint and another number in opposition. Call-in polls are used by some media organizations as a way to measure public opinion and get the audience involved. But they are very problematic from a data quality standpoint and should not be referred to as “polls.”

A major problem with call-in polls is that the participants are entirely self-selected. Only those people who tuned in to that particular broadcast at that time, or read that newspaper, can be included. Further, those who make the effort to participate are often very different from those who do not. That is because participants are usually more interested in the topic or feel very strongly about it. For these reasons, survey researcher Norman Bradburn of the University of Chicago coined the term SLOP, which stands for “self-selected listener opinion poll,” to refer to call-in polls.

Another big problem is that call-in polls are open to manipulation by any individual or group with a vested interest in the topic. With no limit on the number of calls that can be placed, people can call multiple times and groups can set up more elaborate operations to flood the phone lines with calls in support of their point of view. As a result, call-in polls often produce biased results, and their “findings” should be ignored. Legitimate survey researchers avoid the types of bias inherent in call-in polls by selecting respondents using probability sampling techniques.

There are many examples of call-in polls producing distorted results. In one famous example, USA Today conducted a call-in poll in 1990 asking its readers whether Donald Trump symbolizes what is wrong with the United States or symbolizes what makes the United States great. USA Today reported overwhelming support for Trump, with 81% of calls saying he symbolizes what makes the United States great. Later, USA Today investigated the results and found that 72% of the 7,802 calls came from a company owned by a Trump admirer.

Another example comes from a 1992 CBS television program called America on the Line, where viewers were asked to call in and register their opinions after President George H. W. Bush's State of the Union address. The views of the approximately 317,000 calls that were tallied were much more pessimistic about the economy than what was measured in a traditional scientific poll conducted by CBS News at the same time. For example, 53% of those who called in to the program said their personal financial situation was worse than 4 years ago, compared with 32% in the scientific poll. The views of those who called in were quite different than those of the general public on a number of measures.

Although those with survey research training know that call-in polls should not be taken seriously, many members of the public do not make a distinction between these pseudo-polls and the real thing. In fact pseudo-polls may be incorrectly seen as even more credible than real polls because they often have much larger sample sizes.

...