Nonresponse Bias

Nonresponse bias occurs when sampled elements from which data are gathered are different on the measured variables in nonnegligible ways from those that are sampled but from which data are not gathered. Essentially, all surveys are likely to have some degree of nonresponse bias, but in many cases it occurs at a very small and thus a negligible (i.e. ignorable) level. The size of the bias is a function of (a) the magnitude of the difference between respondents and nonrespondents and (b) the proportion of all sampled elements that are nonrespondents. Thus, even if only one of these factors is large, the nonresponse bias may well be nonnegligible.

There are three basic types of survey nonresponse. The first is refusals, which occur when sampled individuals or households decline to participate. The second is noncontacts, when sampled individuals are never reached. The third type of nonresponse consists of situations in which the interviewer cannot communicate with the sampled person because of a language barrier or some mental or physical disability. Most nonresponse is the result of refusals and noncontacts.

It has long been thought that response rates are a good indicator of survey quality and nonresponse bias; however, recent research has challenged this notion. This is encouraging news for researchers because it means that surveys with lower response rates are not necessarily more biased than those with higher response rates. But this does not mean that nonignorable nonresponse bias cannot or will not occur. Nonresponse bias will be present when the likelihood of responding is correlated with the variable(s) being measured, and this correlation can vary across variables even within the same survey.

Nonresponse bias has been a growing concern to survey researchers as response rates have declined over the years. There are a variety of reasons for this, including an increase in refusals with the rise of telemarketing and an increase in technologies to screen calls such as answering machines, voicemail, and caller ID. It is important to consider nonresponse due to refusals and noncontacts separately because the characteristics of refusers can be different from those who are difficult to reach.

Researchers can use several methods to maximize response rates; much recent research has focused on the correlation between response rates and nonresponse bias; and strategies for reducing nonresponse bias are being designed.