Noncoverage

Every scientific survey has a target population that is operationalized by a sampling frame. Ideally, all units in the sampling frame should match those in the target population on a one-to-one basis. In reality, misalignment between the two occurs and is termed coverage error. Noncoverage is one of the elements of coverage error arising from the imperfectness of a sampling frame that fails to include some portion of the population. Because these frames cover less than what they should, noncoverage is also termed under cover age. Noncoverage is the most frequently occurring coverage problem, and it may have serious effects because this problem cannot be recognized easily in the given frame. Because the target population is defined with extent and time, the magnitude of noncoverage depends on the maintenance of the frame. Depending on whether the households or people covered by the frame differ from those not covered, noncoverage may introduce biases (coverage error) in survey estimates.

The classic example of noncoverage is the Literary Digest poll predicting Alf Landon as the overwhelming winner over the incumbent president, Franklin D. Roosevelt, in the 1936 election. Although it had surveyed 10 million people, their frame was comprised of the Literary Digest readers, a list of those with telephone service, and a list of registered automobile owners. Although the general voter population was the target population of the poll, the sampling frame excluded a large proportion of the target population and, more important, an unevenly higher proportion of the middle- and low-income Democratic voters. The general voter population was more likely to differ in their preference of presidential candidate than those who were covered in the frames. Because the sampling frame failed to represent the target population, the poll results favoring Landon's victory were in error.

There are two main sources of noncoverage error in general population surveys. The first is the problem of covering housing units and the second of covering people within housing units. The effect of these sources in noncoverage differs by survey mode. Telephone surveys are discussed more frequently than other surveys with respect to noncoverage error. Between the two noncoverage sources, coverage of people within housing units in telephone surveys has not been found to be as problematic as coverage of housing units. This is because the landline (wired) telephone survey frames are constructed using a directory listing, random-digit dialing, or a combination of the two. No matter which frame is used, telephone surveys cover households that own a telephone and subscribe to a telephone service. Because ownership of a landline telephone is found to be associated with socioeconomic status, it is acknowledged that the telephone is not the ideal mode for surveys in which the socioeconomically disadvantaged population is an important component of the sample. Since 2002, households in the United States with only cellular (mobile) phones have emerged as another non-coverage problem in traditional telephone surveys whose frames are based on landline telephone numbers. Because the cellular phone-only population in the United States (estimated to exceed 20% of adults in 2008) has distinctive characteristics and its proportion in the general population is continuing to grow, the survey research field is making a concerted effort to alleviate this problem.

...