Argument-Based Approach to Validity

Assessments (measures and tests) are used to assign values to attributes of people (or groups of people or objects). The assessment typically involves a limited sample of the person’s behavior collected under standardized conditions, while the attribute of interest is defined much more broadly. The need for validation arises from this gap between the observed assessment results (i.e., a score) and the more general claims associated with the attribute being assessed (e.g., level of reading achievement). Validation examines how well the claims based on the assessment scores are justified.

Rather than defining validity as an abstract property of an assessment, the argument-based approach treats validation as an evaluation of the plausibility of the interpretation and uses proposed for the assessment scores. It employs two kinds of arguments in doing so. The interpretation/use argument (IUA) specifies a chain or network of inferences and supporting assumptions leading from the assessment results to the proposed interpretation and use. The validity argument evaluates the plausibility of the IUA in terms of its completeness (how well it represents the proposed interpretation and use), its coherence (how well it hangs together), and the plausibility of its inferences and assumptions. This entry further describes the argument-based approach to validity and discusses its historical antecedents, the roles of the IUA and the validity argument, and the use of inferences, arguments, and supporting evidence. It then looks at the development of the IUA, assessment, and validity argument and determination of the necessary and sufficient conditions for validity.

The argument-based approach is very general in that it is applicable to any interpretation or use of assessment scores, but it is contingent in that the evidence required for validation depends on the claims being made. Once the interpretation and use are specified as an IUA, the requirements for validation can be specified. That is, the IUA provides a framework for validating the proposed score interpretations and uses.

The validity argument subjects the IUA to challenges by questioning its assumptions and, where appropriate, by subjecting specific assumptions to empirical checks. Validation is never final or absolute because any of the claims being made can be overturned by new evidence, but an IUA that survives all reasonable challenges can be accepted as valid and used to support score-based claims about individuals or groups in the population. An exception to a standard interpretation or score use may be claimed on the basis of unusual circumstances (e.g., for a person with a disability), but in general, if the IUA represents the proposed interpretation and uses accurately, and its inferences and assumptions are supported by appropriate evidence, it can be considered valid.