Validity: Content

Introduction

Content validity is the degree to which an assessment represents the content domain it is designed to measure. When an assessment is judged to have high content validity, the content of the assessment is considered to be congruent with the testing purpose and with prevailing notions of the subject matter tested.

Evaluating content validity is an important part of validating the use of a test for a particular purpose, especially when the test is educational in nature, such as when it is desired to measure a person's knowledge of specific subject matter (e.g. achievement and licensure tests). Although the process of test validation is dynamic and a test is never ‘validated’ per se, in many cases, evidence of content validity is a fundamental component for defending the use of a test for a particular purpose.

Content validity is a notion with which lay people can easily identify and understand. For example, parents and teachers expect items on an elementary mathematics test to be consistent with the elementary mathematics curriculum taught in the schools. Similarly, the general public expects items on a test used to license public accountants to be congruent with the tasks public accountants confront when performing their duties. The evaluation of test content vis-a-vis testing purpose is a natural first step in judging the utility and quality of an assessment. Thus, it is not surprising that the term content validity was included in the earliest efforts to develop standards for test development and evaluation (i.e. American Psychological Association [APA], 1952).

Aspects of Content Validity

There are at least two aspects of content validity: domain definition and domain representation. Domain definition refers to the operational definition of the content domain tested.1 This operational definition is often accomplished by providing descriptions of the content areas and cognitive abilities the test is designed to measure. To adequately define the content domain, several different sources may be used, depending on the purpose of the assessment. These sources include basal textbooks and curriculum objectives for educational tests, job analysis results for employment or licensure tests, and theories of mental abilities and functioning for aptitude tests.

Content domains are often formally defined using test specifications. These specifications, [Page 1076]which are typically in the form of a two-by-two grid listing the content areas along one dimension and the cognitive levels along another dimension, specify the relative weights that are assigned to each of these facets of the content domain. Most technical manuals for tests include these descriptions and specifications. By reviewing the descriptions of the content areas and cognitive levels measured on an assessment, along with the percentages of items that are designed to measure each area/level, one can get a good sense of how the test developers conceptualized the content domain. An understanding of this conceptualization is critical for evaluating the appropriateness of a test for a particular purpose.

Domain representation refers to the degree to which an assessment represents and adequately measures all facets of the content domain tested. The degree to which the content of an assessment spans the entire domain (domain coverage) is one important aspect of domain representation. Another important aspect is domain relevance, which addresses the relevance of each test item to the domain tested. By evaluating domain coverage it can be ascertained whether the entire content domain is being measured, as well as whether critical facets of the content domain are under-emphasized. By evaluating domain relevance, the adequacy and importance of each test item for measuring the content domain can be scrutinized. In addition, problems of content-irrelevance can be identified. The current version of the Standards for Educational and Psychological Testing (AERA, APA & NCME, 1999) underscores the need for providing evidence of domain

...