Norm-Referenced Testing: Methods and Procedures

Introduction

This entry provides a definition of norm-referenced testing emphasizing its specific characteristics as it pertains to educational and psychological testing. The different uses and applications of norm-referenced tests as highlighted by the most popular types of comparisons made are also noted. Third, the advantages and disadvantages of norm-referenced testing are discussed while future applications and uses of norm-referenced testing are described in the concluding section.

Definition and Characteristics of Norm-Referenced Testing

Norm-referenced testing refers to the process of developing and applying tests to enable the interpretation of an examinee's score relative to that of other examinees. Norm-referenced tests provide answers to the question: ‘How much of the measured construct does a person possess in relation to the comparable norm group?’ Norm-referenced testing differs from criterion-referenced testing (CRT) as the primary purpose is to determine how examinees compare to each other and not to what examinees know or can do. While the use of information from these tests might differ, for both norm- and criterion-referenced tests, many of the same stringent test development procedures have to be followed.

The distinguishing characteristics of norm-referenced tests noted by Popham (1990), Thorndike (1997), and many others include:

• individuals are placed on a continuum defined by a range of particular behaviours expressed in a specific population;
• a quantitative answer to the question ‘Where does this individual rank relative to that of a specific-referenced group?’ is provided;
• items typically span the entire range of difficulty values;
• standards of performance are not inherent in the instrument and are only defined after the distributions of scores have been determined;
• no set minimum passing score is defined a priori;
[Page 626]
• test scores are expected to show considerable variability.

Uses and Applications

The primary reason for using norm-referenced testing is to discriminate among examinees; that is, to highlight differences among examinees by producing a dependable rank order of examinees on the construct measured by the test. Typically, most aptitude, interest, and personality inventories are norm-referenced since there are no universally acceptable or relevant criteria to compare the scores against. Norm-referenced testing provides useful information for making decisions in the worlds of learning, work, and play; for example, determining appropriate development levels of children with specific difficulties or selecting the most suitable applicant for a job.

In practice, comparisons in performance among examinees are made using norm tables. Norm tables provide a frame of reference for interpreting the scores of particular examinees in comparison to the particular normative group that defines the set of standards (Murphy & Davidshofer, 1998). Depending on the purpose of testing, norm tables can be developed at different levels (national, local) and for different samples of the population (e.g. males and females or second language speakers). It is important to note that norm tables can only be developed for instruments in which a total score can be calculated, for example a test or a scaled questionnaire. However, if norm tables are to be useful, they must be based on recent data, be representative of the population, and be relevant to the group of interest (Murphy & Davidshofer, 1998).

The process of constructing norm tables, called norming, can be a complex, costly, and time-consuming process that usually entails two stages: (1) the test is administered to a representative sample of examinees for whom the test was developed; and (2) the raw scores obtained on the test are converted to scale scores that define the type of norms developed. Three additional points need to be noted. First, norms can only be developed for standardized tests. If tests are not standardized, by definition any comparison would not make sense as examinees take the test under different conditions. Second, norm tables must be completed before any norm-referenced test scores are released. Third, all norm tables have to be regularly updated since norm groups (especially national samples) generally change over time. Depending on the purpose and use of the test, norms are typically updated every 5 to 8 years.

...