Validity

Argument-Based Approach to Validation

Educational and psychological tests are pervasive in society, offering measurements of many distinct attributes, such as academic aptitude, personality characteristics, and career interests. The responsible use of tests can lead to increased information about individuals so as to help them lead more productive, happier lives. However, when tests are used to make unwarranted claims about individuals, it hurts not only the persons writing the test but also the general public, who may question the development and use of tests for the good of society.

Establishing the validity of inferences made about individuals (henceforth referred to as examinees) based on their test scores is fundamental to the appropriate development and use of tests.

The first step toward establishing the validity of test-score interpretations for particular uses is a written statement or series of written statements about how the test scores are going to be interpreted. The argument-based approach to validation requires that [Page 996]these statements be clear and explicit and also include a rationale for the proposed interpretation and use. For example, imagine a developer of a science achievement test who intends to use the test to select high school students for a special university program in physics. The developer would begin the validation process by writing an explicit statement about how the test scores on the achievement test are going to be interpreted; for example, high scores will be interpreted to indicate competence in the following knowledge and skills: designing experiments, testing hypotheses, and deductive reasoning. The statement would also indicate the proposed uses of test-scores; for example, examinees scoring above a certain cut-off will be selected for inclusion in the university physics program. A rationale for viewing the test items, and hence test scores, as appropriate measures of the construct of interest (science achievement) and why the scores should lead to a particular use must also be outlined. For example, the rationale must provide reasons for accepting the items as appropriate measures of the knowledge and skills in science achievement and why these knowledge and skills (and not others) are the appropriate ones to use in the selection of examinees.

...