Margin of Error (MOE)

The margin of error (MOE) is a statistical concept that is most often associated with polls and pollsters. It serves to quantify the uncertainty associated with sampling in a poll or other survey. In survey research, it is almost never practical to measure the entire population. As a result, pollsters rely on random samples that are intended to be representative of the population. Because polls randomly sample from within a population, there will always be some amount of uncertainty, or variable error (variance), associated with their results. Simply put, if a U.S. pollster were to randomly sample 1,500 adults in a national survey, it is unlikely that these 1,500 people would perfectly reflect the opinions of the 200-plus million adults in the country.

The MOE can account only for random sampling error. It is unable to capture variance or bias that may be due to other aspects of total survey error, such as miscounts, incorrect coding, question bias, nonre-sponse caused by not gathering data from sampled [Page 451]respondents' when they could not be contacted or they refused to cooperate, and/or respondents lying or not answering all of the questions.

A real-life example illustrates the MOE's meaning and its use by pollsters and journalists. A Pew Research Center poll conducted October 27–30, 2004, asked respondents to identify for whom they were going to vote in the 2004 U.S. presidential election. The results found that 51% of respondents identified George W. Bush, 48% John Kerry, and 1% Ralph Nader. Pew reported that the sample size was 1,925 likely voters, with an MOE of approximately ±2.5 percentage points.

The MOE is typically calculated based on one of three levels of confidence: 99%, 95%, or 90%. Pollsters most commonly rely on the 95% level of confidence. Roughly speaking, MOEs at the 95% confidence level are 24% smaller than at the 99% level if the sample sizes are the same (an MOE of approximately ±1.9 at the 99% level of confidence would result in the example). When using a 95% confidence level, it is expected that the “true” percentage for the population will be within the MOE of the poll's reported percentage (i.e. the confidence interval) 95% of the time (19 times out of 20). Using the Pew poll example, this means that the true population's vote for Bush would have been expected to be between 53.5% and 48.5% (i.e. 51 ± 2.5), 95% of the time, had the same Pew survey been conducted many different times using different (but similarly designed) random samples of similar size.

In surveys that use a simple random sample, the MOE is easily calculated. At the 95% level, it is calculated by the following equation, ±1.96(SQRT(PQ/(«))(100), where P represents the percentage of interest (e.g. 51% support for Bush in the 2004 Pew poll) and Q represents 1 − P. The size of the sample on which the percentage is based is represented by n. The 1.96 is a constant associated with the 95% level of confidence. As the equation indicates, the MOE is very much affected by the survey's sample size. Thus, in the Pew example, had a simple random sample been used, the MOE would be calculated by ±1.96(SQRT((.51) (1 − .51))/(1925))(100) or ± 2.2, which is slightly less than what Pew reported.

...