Internal Validity

Example of a Survey Lacking Internal Validity

The results of a nonexperimental pre-election study on the effects of political advertising were posted on the American Association Public Opinion Research listserve (AAPORnet) in 2000. This survey-based research was conducted via the Internet and found that a certain type of advertising was more persuasive to potential voters than another type. By using the Internet as the data collection mode, this survey was able to display the ads—which were presented as digitized video segments—in real time to respondents/subjects as part of the data collection process and, thereby, simulate the televised messages to which voters routinely are exposed in an election campaign. Respondents were shown all of the ads and then asked to provide answers to various questions concerning their reactions to each type of ad and its influence on their voting intentions. This was done in the individual respondent's own home in a room where the respondent normally would be watching television. Here, the Internet was used very effectively to provide mundane realism to the research study by having survey respondents react to the ads in a context quite similar to one in which they would be exposed to real political ads while they were enjoying a typical evening at home viewing television. Unlike the majority of social science research studies that are conducted under conditions far removed from real life, this study went a long way toward eliminating the potential artificiality of the research environment as a serious threat to its overall validity.

Another laudable design feature of this study was that the Internet sample of respondents was chosen with a rigorous scientific sampling scheme so that it could reasonably be said to represent the population of potential American voters. The sample came from a large, randomly selected panel of households that had received Internet technology (WebTV) from the survey organization in their homes. Unlike most social science research studies that have studied the effects of political advertising by showing the ads in a research [Page 346]laboratory setting (e.g. a centralized research facility on a university campus), the overall validity of this study was not threatened by the typical convenience sample (e.g. undergraduates “volunteering” to earn course credit) that researchers often rely upon to gather data. Thus, the results of this Internet research were based on a probability sample of U.S. households and, thereby, could reasonably be generalized to the potential U.S. electorate.

...