Test Directions and Scoring

Test Administration Scoring and Reporting

Standardized directions to examinees have the goal of ensuring that the procedure of test taking is properly understood. When tests are administered by computer or require special equipment, some practice time must be provided. Test takers will generally be informed of time constraints, [Page 976]answering procedure, and, if pertinent, about when to omit item responses. Implicit is the goal of not providing unfair advantage to anyone.

With respect to the scoring process, standards 5.1, 5.2, 5.8 and 5.9 are summarized below: interpretability of test scores requires that tests be administered and scored following the developer's instructions, and only special situations or an examinee's disability would justify an exception. Scoring processes must be monitored in order to assure accuracy, correcting any systematic source of errors. When test scoring requires human judgement, adherence to scoring criteria should be checked regularly.

Test Development and Revision

Scoring rules are associated with test framework and item format, and must be consistent with the purpose of the test and facilitate meaningful score interpretation. Standards 3.6 and 3.14 are explicitly relevant to scoring.

Only after the nature of the item and response format has been specified, can scoring rules be selected. Scoring rules for multiple-choice items are obviously different from rules for short-answer items, extended-response formats, or performance assessments. In any case, all formats need some indication about how to score the responses. For short-answer formats, a list of acceptable answers is usually enough. For extended-response formats, instructions concerning the answers that will be scored correct have to be more detailed; in this modality, the scorers are usually provided with scoring rubrics specifying the evaluation criteria. For performance assessments, as well as for extended-response format, analytic or holistic scoring can be used – analytic scoring procedures give not only a total score on explicit criteria that reflect the test framework, but also separated scores for critical dimensions; holistic scoring procedures simply provide an overall score. The rationale of the various rules for multiple-choice testing programmes as well as the empirical data generated in the context of this format will be the object of the next section.

...