Assessor's Bias

Introduction

Psychological assessment is subject to various errors of measurement. While some are random, as assumed in classical test theory, others are systematic and lead to consistent distortions of the true value of a characteristic. These latter errors may be partially due to assessor's biases. This term does not refer to elementary professional mistakes such as implementing test instructions incorrectly, but to systematic tendencies in case-related information processing that reduce the validity of data. Although these biases normally impair objectivity and reliability, they remain undetected when they are consistent across individuals and time. In addition, a low interrater agreement is not necessarily a sign of assessor's bias but may be due to valid differences between settings and informants (Lösel, 2002).

Not all types of assessment information are equally susceptible to assessor's biases. Whereas standardized tests or biographical inventories are less affected, their impact may be strong in unstructured interviews, behaviour observations, or trait ratings. For example, some studies on the judgement of job performance have shown that more than half of the variance is due to differences in the assessors (Scullen et al., 2000). In a meta-analysis, approximately 37% of the variance in ratings was attributed to them (Hyot & Kerns, 1999).

This entry concentrates on biases in assessments by other persons (e.g. psychologists, psychiatrists, teachers, or lay informants). Although these biases are similar to the numerous response sets and distortions in self-reports, some seem less important (e.g. lying, simulation, dissimulation, social desirability, or positive self-presentation) and others more relevant (e.g. halo, leniency, stringency, or contrast effects). In the following, we will first describe several of these errors and afterwards address factors that differentiate and moderate these distortions. Finally, we will take a brief look at approaches for detecting and reducing assessor's biases.

Examples of Assessor's Biases