Internal Validity

Threats to Internal Validity

If an experiment extends over a long period of time, uncontrolled changes may occur in the experimental groups—this confound is known as the history effect. If a difference is observed between the experimental and control groups, it may be due to history effects or the interaction of history with treatment. For example, suppose an educational psychologist were to investigate the effectiveness of a reading program across the school year delivered to two groups (experimental and control) by specially trained teachers. Suppose further that for 2 months, a substitute teacher replaced the original teacher in one group—this may pose an uncontrolled effect, as the substitute teacher may have influenced students' reading performance.

As with history, maturation is also a concern when a study extends over a long period of time. As experiments go on, biological or psychological changes may occur in participants. For example, participants become older, and they may change emotionally or physically. For example, suppose an educational psychologist were to examine the effectiveness of a reading program over one school year. If factors such as physical, social, and intellectual development are not controlled for, these processes, rather than the reading program, may contribute partly or entirely to reading proficiency changes. Therefore, the investigator could not claim unequivocally that the reading program is beneficial; this study would lack internal validity.

Another threat to internal validity is testing. Many educational experiments use pretest and posttest designs; and often, both testing situations are similar. If so, then the participants may display a practice effect from repeated testing. If participants show an improvement on a particular test, it may be due to the repeat test, rather than the independent variable (such as the reading program). Additionally, subjects might be sensitized to certain aspects of the pretest on which they focus more, and, as a result, do better on the posttest. On the other hand, if different tests are used in pretest-posttest situations, this may cause differences in measurement that may bias the results. For example, if a pretest consists of a more difficult reading test and the posttest is easier, the improvement may not be due to the independent variable, the reading program, but to the differences in measurement. This is called an instrumentation effect.

Attrition occurs when there is loss of participants during the course of an experimental treatment. Participant dropout can bias the results because there may be important differences between the participants who discontinue in the study and the participants who remain in the study. That is, if certain types of participants are more likely to drop out of the study than others, then the group that concludes in the study consists of different types of individuals than the group that started. This can be detrimental to the results of experiments. If a study is measuring improvement to reading after the implementation of a reading program, and the students who drop out of the study are low-achieving students, then the results would be biased; the concluding sample would consist of high-achieving students and low-achieving students would not be represented.

...