Nonignorable Nonresponse

When patterns of nonresponse (either unit or item nonresponse) are significantly correlated with variables of interest in a survey, then the nonresponse contributes to biased estimates of those variables and is considered nonignorable. Recent trends of increasing survey nonresponse rates make the question whether nonresponse is ignorable or not more salient to more researchers.

Since data are only observed for responders, researchers often use participating sample members or members for whom there are complete responses to make inferences about a more general population. For example, a researcher estimating the average income of single parents might use income data observed for single-parent responders to make generalizations about average income for all single parents, including those who did not participate or who refused to answer the relevant questions. The underlying assumption is that single-parent sample members who do not respond or respond with incomplete data are similar to single-parent sample members who participate fully. This implies that the units with missing data or incomplete data are a random subsample of the original sample and do not differ from the population at large.

If this assumption is spurious (i.e. it is not true)—that is, units with missing or incomplete data are different in meaningful (nonignorable) ways from the rest of the sample on key variables of interest—then inferences with missing data can lead to biased estimates. For example, if lower-earning single parents have high unit nonresponse rates because they are more difficult to locate and contact, then the estimate of income, the key variable, will be upwardly biased. Thus, when survey participation rates are correlated with key variables, unit nonresponse is likely to be nonignorable.

Essentially every survey has some nonresponse either because of an inability to locate or contact a sample member, or because of a sample member's refusal to participate or to answer certain questions. When researchers make inferences from their sample to the population, then survey response rates are considered an indicator of the representativeness of the data, making the response rate an important criterion of data quality. Because of this, declining response rates make the question of whether or to what extent the nonresponse is ignorable especially important.

The growing problem of nonresponse has led researchers to increase efforts to reduce nonresponse and measure possible nonresponse error. Nonresponse due to noncontact is usually dealt with by improving tracking and locating efforts and by increasing the number of contact attempts at different times of day and days of week to maximize the probability of contact. Survey organizations may provide interviewer training in avoiding or converting refusals. Incentives are used to increase contact rates and decrease refusal rates. Efforts to maintain contact with sample members are used between waves in longitudinal studies to minimize sample attrition. Where nonresponse is due to a physical or mental limitation of the sample member, proxy interviews (e.g. by a family member) may provide key data. In some rare instances, researchers are able to compare survey responses to administrative data in order to measure the impact of nonresponse. Finally, researchers will also make statistical adjustments using external benchmarks such as census data to estimate the impact of nonresponse on their estimates.

...