Test Designs: Developments

Introduction

Test design refers to the process by which a test developer builds psychological tests – cognitive, affective, achievement, aptitude, certification, licensure, interest, motivation, personality, and others. The process addresses the issues of why a test is being used, what the test will cover, and [Page 970]how it will test for those. A well-designed test does not happen by accident, but results from a systematic and informed series of choices on a number of issues. This entry reviews those issues, and emphasizes advances in the test development process. There are several more lengthy treatments of the topic, variously emphasizing item writing (Haladyna, 1999; Osterlind, 1998), analysis of test scores (e.g. Embretson, 1985; McDonald, 1999), cognitive psychology (Frederiksen, Mislevy & Bejar, 1993), or new developments (Hakel, 1998).

AN OVERVIEW OF THE TEST DESIGN PROCESS

Test design may be considered in stages. One first determines a use (purpose) for the test – that is, a reason for giving the test – and how test scores will be used. Tests are given to produce scores, from which some kind of decision about the test taker will be made. Less frequently, tests may secondarily be given to convey something about the ones giving the test. For example, a selection test for a company might serve as a job preview, or might convey a message, such as ‘this is a high-tech operation’, or ‘we demand high levels of integrity’. The purpose may affect both the development of test specifications and the selection of item types and delivery means. A selection test to be administered worldwide might be delivered on the web; a self-assessment might be delivered in a magazine.

Next one develops specifications for the test, and decides how the test will be delivered. Following this are the item writing, test assembly, review, and standard-setting (cut-score) phases, with empirical item trials throughout, or whenever possible. Each of these stages are reviewed.

Test Use

Tests are used in many ways. Some of the most important are: selection, classification, certification, licensure, promotion, diagnosis, student modelling, and self-assessment. Selection refers to using test scores to admit applicants into jobs, the military, or educational or training programmes. Classification refers to using scores to place applicants into jobs or programmes for which they may best be suited in light of the pool of selected individuals. For example, an employer might assign an applicant with strong social skills (e.g. based on a personality test) to a job involving working with people. Or a college may place an applicant into a remedial reading programme based on a reading score. Classification is similar to selection, but differs in several ways – e.g. in how the efficacy of the process can be evaluated (Scholarios, Zeidner & Johnson, 1997), and in the legal issues surrounding use. Certification, or licensure, refers to using test scores to formally ascertain one's level of proficiency in a technical or professional speciality. Examples are driver's licence examinations, architectural, auto mechanics, and real-estate licensure examinations, and nursing, business, teaching, law, and medical board examinations. Promotion testing is similar to certification and licensure testing, except that it pertains to movement through the ranks of a particular job, and does not typically imply any legal status associated with proficiency level, as a licensure test does.

...