Validity

Common Misconceptions

It is common to hear people refer to the “validity of the test,” which might give the impression that validity is a property of the measurement device. However, this is incorrect. Validity is not a property of any assessment device; rather, it is a property of the inferences that you—the test user—make. For example, consider once again the tape measure. We might be tempted to say that “the tape measure has validity.” However, if we made inferences about differences in intelligence based on that same set of measurements rather than differences in height, those inferences would likely be highly incorrect. Nothing has changed about the tape measure or the set of measurements generated from its application. What has changed is the inference about what is being measured.

Although this might seem an absurd example (presumably no one would use a tape measure to measure intelligence), it demonstrates that validity is not a property of the measurement instrument but of the inference being made. The phrase “the test has validity,” though technically inappropriate, is often used because there is a general assumption about which inferences are (and are not) to be made from the use of a well-known measurement device. For example, testing experts may say, “The Wonderlic has good validity.” On the surface, this may seem profoundly inaccurate; however, it should be understood that this statement actually means (or at least, should mean), “Inferences regarding individual differences in general mental ability, and inferences regarding the probability of future outcomes such as job performance, are generally appropriate by relying on observed scores generated from the appropriate use of the Wonderlic.” That we sometimes use shorthand to abbreviate such a long statement should not be taken to imply that validity is a property of the test. Rather, it should be interpreted as suggesting there is reliable and verifiable evidence to support the intended set of inferences from the use of a given measurement device.

The second common misconception is that there are different types of validity. Instead, validity is best thought of as a unitary concept addressing how completely and accurately a measure measures what it is intended to measure. However, no single method or strategy can provide all the evidence needed to make accurate or confident inferences. Thus, multiple strategies exist for generating such evidence; often, these strategies—or more aptly, the evidence generated [Page 865]from these strategies—are referred to as types of validity. This is an unfortunate choice of words because it often leads to the misconception that validity is many different things and that some types of validity are more or less useful than other types. Validity is a single, unitary idea: It concerns the degree to which the differences we observe in measurements can be used to make accurate and confident inferences about some unobservable phenomenon.

...