Test User Competence/Responsible Test Use

Introduction

The purpose of this entry is to highlight some efforts to promote responsible use of tests throughout the world. Psychological and educational tests serve as key professional tools for psychologists and non-psychologists, and test publishers serve as gatekeepers to determine who has access to different types of tests. A persistent problem, acknowledged by professional associations and publishers, has been the misuse of test data and its subsequent potential harm to test takers.

Three-Level Publisher Classification System

The American Psychological Association (APA), five decades ago, addressed ethical use regarding the sale and distribution of tests (APA Committee on Ethical Standards for Psychology, 1950). Their [Page 979]1954 technical recommendations (APA AERA & NCMUE, 1954) introduced a 3-level system for classifying tests, which emphasized the importance of professional credentials to justify access to tests at the highest level: level C (Moreland, Eyde, Robertson, Primoff & Most, 1995). Though the 3-level classification reappeared in the 1996 test standards, it no longer exists in APA standards or ethics codes (APA, 1992; American Educational Research Association et al., 1999; DeMers & Turner, 2000).

The current status of the 3-level classification and qualification requirements was examined using material from 8 USA test publisher catalogues. Seven of the publishers used a 3-level system, each with their own definitions and interpretation of the requirements, based on questions in their qualification forms. User questionnaires included level of training, professional association membership, licensure, course work, and workshops. Robertson, who in 1986 reported on a survey of test qualification forms of 13 major publishers, concluded that ‘conditions of sale often were stated so generally that it was not possible to ascertain the exact criteria used to establish professional credibility’ (p. 9). His conclusion still holds. Credentials and degrees serve as a crude screen to place test purchasers into broad categories that cover a wide range of criteria. The publishers require test purchasers to adhere to the APA test standards or to the standards of other professional associations.

Empirical Test User Research and Training

The Joint Committee of Testing Practices (JCTP), which now includes APA and six other professional associations, was established in 1985 to provide ‘a means by which professional organizations and test publishers can work together to improve the use of tests in assessment and appraisal’. Two of JCTP's products dealt with empirical research on test user qualifications and training. JCTP formed the Test User Qualifications Working Group (TUQWoG) with the purpose of developing behavioural competencies for test users, based on two job analysis methods: the critical incident method and Primoff's job element method (Moreland, Eyde, Robertson, Primoff & Most, 1995).

The empirically based results included 12 minimum competencies for proper test use, such as avoiding errors in scoring and recording … keeping scoring keys and test materials secure … and establishing rapport with examinees to obtain accurate scores (Moreland et al., 1995: 16). A set of 86 specific test user competencies were derived from critical incidents gathered from 62 experts on 48 tests. Seven factors of test misuse were identified that dealt with: (a) comprehensive assessment, (b) proper test use, (c) psychometric knowledge, (d) maintaining integrity of test results, (e) accuracy of scoring, (f) appropriate use of norms, and (g) interpretive feedback. Furthermore, a system for empirically clustering tests for possible 3-factor or 8-factor solutions, based on likelihood of test misuse, was developed. A sample qualification form was developed and adopted by three test publishers and was adapted by others. Two of the eight test publishers currently ask test purchasers to accept and comply with Principles of Effective Test Use and another publisher lists Elements of Sound Testing.

...