Reporting Test Results in Education

Introduction

In this entry we shall discuss both how test reports are currently reported as well as enumerating steps that might be taken to improve reporting still further. To the extent that space allows I will try to show rather than tell, although a combination will be used when that seems helpful. I will span three situations for which test results are reported. These are:

• (i)
  results that are reported to an individual examinee,
• (ii)
  results that are reported to an institution,
• (iii)
  results that are reported for a state or nation.

While all of these situations share a number of common aspects, there is also enough that is unique to justify separate treatment. I shall begin with a statement of purpose, then examine the extent to which these purposes are fulfilled in some representative reports, and then finally will try to extend practice by suggesting modifications that could aid in achieving these goals.

There are essentially four questions that a score report should answer, the first three of which are:

• 1
  What is my score? For individuals this might be a single number or a set of numbers, for institutions or nations a summary statistic or a distribution.
• 2
  How do I compare to others? A fact without context is of little value. A single number tells us nothing without the ancillary knowledge about how everyone else did. Even so-called ‘criterion-referenced tests’ have latent in them the performance of a reference population. Thus, a 4-minute miler is applauded even if he ran alone on the track, but the ‘objective criterion’ of [Page 818]4-minutes gets its meaning from the knowledge of how many have tried to do it and how few have succeeded.
• 3
  How stable is my score? If you stand on a bathroom scale and it reads 100 kg, how much will it change if you get off and then get on again?

While all of these questions are important to answer, this ordering represents the typical priority. Hambleton and Slater (1996) in a survey of educational policy makers support this prioritization. This result was confirmed in subsequent experiments on this same class of test users (Wainer, Hambleton & Meara, 1999). Thus the visual emphasis given to each question should reflect this prioritization. In addition, there is a fourth question, strongly related to the first three, whose answer is too often left implicit,

• 4
  What does my score mean? Obviously, this is a validity question, and its answer depends on the score level, how that score compares with others, and how stable the score is. The precise form of this question varies with who asks it. But the answer is almost always a probability statement. For the individual, the question might specialize to ‘Can I get into Princeton?’; for an institution it might become ‘How well can this student handle the coursework here?’; and for a national report it often reflects the policy issues that drive the assessment with causal questions like, ‘Has the intervention aided minority achievement?’

Individual Score Reports

Let us consider a score report for a college admissions test that is typical of those provided annually to well more than a million high school students in the United States.

...