Quality Control

Training Interviewers

In surveys that use interviewer-administered questionnaires, proper training of interviewers is often seen as the heart of quality control. To become successful and effective, interviewers first must be introduced to the survey research process through a general training, in which fundamental concepts such as basic interviewing techniques, obtaining cooperation, and maintaining respondent confidentiality are covered. Following general training, interviewers should be trained on the specifics of an individual project. Training sessions should strive to maximize active participation. Topics covered should be reinforced with group discussion and interaction, trainer demonstrations, and classroom practice and discussion. Role playing and practice are also key elements of effective basic project training. Assessment of the knowledge gained and retained by interviewers is also a part of survey quality control.

Monitoring Interviews

In surveys that use interviewer-administered questionnaires, monitoring interviews has become a standard quality-control practice. Telephone interviewer monitoring is accomplished by using silent monitoring equipment so that neither the interviewer nor the respondent is aware of it. In addition, facilities may use equipment that allows the monitors to listen to an interview and simultaneously visually observe the progress of an interview on a computer screen. Monitoring serves four objectives, all aimed at maintaining a high level of data quality. These objectives include (1) obtaining information about the interview process (such as procedures or interview items) that can be used to improve the survey, (2) providing information about the overall data quality for the survey in order to keep the data collection process in statistical control, (3) improving interviewer performance by reinforcing good interviewing behavior, and (4) detecting and preventing data falsification.

Generally speaking, there are two different types of telephone monitoring conducted: quality assurance monitoring and interviewer performance monitoring. Quality assurance monitoring involves the use of a coding scheme that allows monitors to score interviewers on a variety of aspects such as proper question delivery, use of nondirective probing, and correct entry of responses. Interviewer performance monitoring takes more of a qualitative approach. Under this [Page 650]process, monitors record any observations that might be useful in subsequent evaluation of the interviewing staff. Monitors will make observations that focus on appropriate administration of items, remaining neutral, tone of voice, pace, and so on, and will provide feedback, both in the form of positive reinforcement and constructive criticism with potential retraining as warranted.

While monitoring telephone interviewers is an efficient and nonintrusive activity, conducting monitoring of face-to-face interviews in the field is more challenging. Traditionally monitoring face-to-face interviews in the field is done in one of two ways. First, direct observations can be made by field supervisors who accompany field interviews during data collection. During these periods, supervisors will observe and evaluate field interviewers on adherence to survey protocols while noting areas requiring improvement. Second, tape recordings are used to record portions of interviews and are reviewed after the fact to evaluate interviewers on quality, proper administration of the survey, and so forth. While effective in capturing quality information, these two methods have traditionally come with a cost, as they are both intrusive and cumbersome. Recent advancements in digital recording have made monitoring possible without the need to carry recording equipment such as microphones and tapes. Computer audio-recorded interviewing allows for the recording of portions of interviews for quality control on computer-assisted personal interviewing (CAPI) laptops, eliminating the need for external equipment.

...