Standardized Survey Interviewing

In a standardized survey interview, the interview proceeds according to a script (the introduction and the questionnaire) that is intended to minimize any potential impact of individual interviewers' behavior on respondents' answers and the resulting data. Standardized interviewing procedures for sample surveys were developed over several decades of the 20th century as evidence accrued that even seemingly minor differences in how interviewers behaved in interviews sometimes affected answers and data quality. Interviewers' biases or assumptions about particular types of respondents could creep into the interview through subtle changes in wording or tone that could lead respondents to interpret questions or the situation differently than they would have with a different interviewer. Even without intending to influence answers, interviewers who attempted to increase rapport by rephrasing a question the second time they asked it, or politely did not present all the response alternatives to a question because they judged that some alternatives would not fit a particular respondent's circumstances, could harm the quality of the data. It also became clear that interviewers could, in all innocence and with good intentions, introduce bias through how they reacted when a respondent expressed reservations or uncertainty about an answer; interviewers could subtly encourage respondents to give answers that fit the interviewers' preconceptions rather than the respondent's actual circumstances or opinions. Standardized survey interviewing procedures are designed to circumvent these problems and to ensure that the data from all respondents are fully comparable because all respondents have answered the same questions under the same procedures. Standardizing the interviewing procedures is intended to address the measurement error due to interviewers, which is assumed to be independent of measurement error due to question wording (which can be addressed through better question pretesting) and measurement error due to respondents (which cannot easily be addressed by survey researchers).

Ideally, interviewers adhering to standardized procedures read (either from paper or from a computer screen) survey questions and all response alternatives precisely as worded by the designers of the survey, and they repeat the full question and all response alternatives when asked to repeat the question. In the strictest forms of standardized survey interviewing, interviewers also leave the interpretation of questions entirely up to respondents and only respond to any requests for clarification with neutral probes like “whatever it means to you” and “let me repeat the question.” The logic is that if only some respondents receive clarification or help with answering, then the stimulus (the question wording and response alternatives) is different for a different respondent, and thus there is no guarantee that the data are comparable.

The broad consensus is that standardized interviewing procedures are the most desirable for sample surveys and that more idiosyncratic or ethnographic forms of interviewing that are useful for other more qualitative research purposes are risky or undesirable in surveys. But this consensus can manifest itself somewhat differently in different survey organizations, where the precise procedures that count as “standardized” in one center can differ from those in other organizations. For example, organizations differ on whether providing clarification to a respondent counts as nonstandardized or standardized and on whether repeating only the most appropriate response alternatives is better than repeating them all. Survey organizations can also vary in how extensively they train and monitor their interviewers for adherence to the standardized procedures they advocate, which means that in practice some standardized surveys turn out to be less standardized than others.

...