Record Check

A record check study is one in which the validity of survey self-reports is assessed by comparing them to evidence in organizational records. Record check studies have been employed to evaluate the quality of data from existing surveys and also to test the relative quality of data from different prospective survey designs. Such investigations are often viewed as the gold standard for judging the validity of self-report information on behavior and experiences.

Record checks are infrequently conducted because of the limited availability of record information, as well as the cost and effort of obtaining and matching record and survey data for the same individuals. Although there are many potential sources of records, in practice it can be difficult to gain access to information that gives insight into the quality of survey data. The cooperation of record-holding organizations is required and, unfortunately, often hard to obtain. Records must be collected and preserved in a manner that permits investigators to use the information to make valid inferences. Organizations assemble records for their own aims, not for the purpose of validating survey data. Thus, information that survey investigators deem important may be missing or poorly maintained in records. Records may be organized in such a way that they require substantial transformation before they can be employed in a record check study. Finally, records are not kept on many things about which survey investigators are interested; record check studies are therefore confined to those matters in which human behavior comes in contact with bureaucratic systems. Other important behaviors, as well as attitudes, perceptions, and other mental constructs cannot be examined through the record check technique.

These points notwithstanding, it is clear that evidence from record check studies has been influential in understanding reporting errors and in designing important surveys. Record checks have been conducted in a number of contexts—for example, health, labor, voting, and victimization. In research done in connection with the National Health Interview Survey, medical [Page 698]records were employed to check the quality of reporting of doctor visits, hospitalizations, and morbidity. Records were obtained from physicians and hospitals detailing the date of medical encounters, along with the reasons for the visits. Respondents were then interviewed about their health experiences, and the survey reports were matched against the records. Comparisons revealed, for example, whether respondents reported a recorded medical contact or not. A result of one of these studies was that the reference period for the question on doctor visits was set at 2 weeks before the interview because comparison of survey reports with visit records showed that accuracy of recall was sharply lower for longer reference periods.

Another important series of record checks involved the American National Election Study. For several post-election waves of this biennial survey, respondent reports of voting were checked against public voting records. In this case, respondents were asked about their voting first and then interviewers attempted to verify the reports by examining voting records in county clerk offices. These comparisons suggested that there is an overreporting bias in self-reported voting. More people said that they voted than actually did. Since these studies were conducted, there have been many papers written on the possible causes and consequences of vote overreporting. In addition, the American National Election Study question used to measure voting was modified in an attempt to reduce the hypothesized “social desirability” bias attached to self-report of voting. Finally, there have been investigations of whether statistical models predicting voting differed in their predictive power if the analysis was confined to only “valid” votes—those reported that matched the officiai records—or also included the “invalid” votes. The findings suggest that the models performed equiv-alently in these cases.

...