Mode-Related Error

Face-to-face (in-person) surveys, telephone, mail, and Web surveys are common types of data collection modes in current survey research. These modes can be classified into two categories: (1) self-administered versus interviewer-administered, depending on whether interviewers are involved in interviewing, (2) and paper-and-pencil versus computer-assisted, depending [Page 484]on whether computerized instruments are employed in surveys. Currently, the two types of classification for survey modes are widely used for comparing response rates or survey errors. Total survey errors—sampling, coverage, nonresponse, measurement, and processing error—do not equally emerge from each mode. That is, the type and magnitude of error varies with the mode.

Sampling error is not directly influenced by modes. All sample surveys inevitably encounter this error, due to the fact that the whole target population is not selected as a sample. However, its magnitude varies, depending mostly on sample size. Generally, sampling error decreases as sample size increases, regardless of mode.

Coverage error arises from the mismatch between a sample frame and a target population frame. It is not dependent on the mode itself, but on the quality of the sample frame that the mode employs. For instance, in 2000 in the United States, the penetration rate of households with a landline telephone was more than 95% (including cell phone only households), whereas household Internet access was approximately 50%. This means that a Web mode is much more susceptible to coverage error than a telephone method in household surveys. Among data collection modes, the Web method seriously suffers from this error because of a poor frame, a disparity in Internet penetration between the poor and rich, and the existence of users with multiple email accounts.

For nonresponse error, nonresponse rates can affect the amount of error. However, it is important to note that reduction in the nonresponse rate does not always affect nonresponse error. There are two types of nonresponse in surveys: item and unit nonresponse. The former occurs when respondents avoid reporting one or more specific questions, whereas the latter arises primarily from noncontact, refusal, and inability to answer (e.g. a language barrier). Item nonresponse diminishes in interviewer-administered modes, whereas the occurrence of unit nonresponse varies across modes. In general, unit nonresponse rates for face-to-face surveys are the lowest, followed by higher nonresponse rates for telephone and mail surveys, with Web surveys having the highest unit nonresponse. Generally, interviewer-administered modes have higher response rates than self-administered.

There are a number of sources of measurement error, including social desirability effects, interviewers, respondents, questionnaires, and so forth. Social desirability can be a large threat to survey validity. Respondents tend to provide socially desirable and avoid socially undesirable responses in surveys. Indeed, interviewer-administered modes are more susceptible to social desirability bias than self-administered modes because respondents are reluctant to disclose socially stigmatized behaviors in the presence of interviewers, especially when queried about sensitive topics. Also, social desirability biases are reduced in computer-assisted self-interviewing (CASI) and audio computer-assisted self-interviewing (ACASI). CASI and ACASI lead to less item non-response for sensitive questions—the number of sex partners, abortion, drug or substance use, and so on—than computer-assisted personal interviewing (CAPI), computer-assisted telephone interviewing (CATI), and self-administered paper-and-pencil interviewing (PAPI). Directive probing or interviewers' characteristics (i.e. gender and race) may also lead to interviewer effects that vary responses across interviewers in interviewer-administered modes.

...