Criterion Theory

The Ultimate Criterion

Eventually, we are interested in predicting the ultimate criterion, that is, the full domain of employees' performance, including everything that ultimately defines success on the job. Given the totality of this definition, the ultimate criterion remains a strictly conceptual construct that cannot be measured or observed. To approach it, however, and to describe the connection between the outcomes valued by the organization and the employee behaviors that lead to these outcomes, J. F. Binning and G. V. Barrett introduced the concept of behavior-outcome links. In practice, these links should be based on a thorough job analysis, in the form of either a job description that analyzes the actual job demands or a job specification that reveals the constructs required for good performance.

The Operational Criterion

The conceptual nature of the ultimate criterion requires practitioners to deduct and develop the criterion measure or operational criterion, an empirical measure that reflects the conceptual criterion as well as possible. Using this operational criterion as a proxy for the conceptual criterion of interest, the usual approach in personnel psychology is to establish a link between performance on a predictor and performance on the operational criterion as an indication of the predictor's criterion-related validity. Operational criteria might include the following:

• Objective output measures (e.g., number of items sold)
• Quality measures (e.g., number of complaints, number of errors)
• Employees' lost time (e.g., occasions absent or late)
• Trainability and promotability (e.g., time to reach a performance standard or promotion)
• Subjective ratings of performance (e.g., ratings of knowledge, skills, abilities, personal traits or characteristics, performance in work samples, or behavioral expectations)
• Indications of counterproductive behaviors (e.g., disciplinary transgressions, personal aggression, substance abuse, or voluntary property damage)

In practice, operational criteria should satisfy at least three independent requirements.

• 1.
  Operational criteria must be relevant to the organization's prime objectives. Although this may sound obvious in theory, in practice, criterion choices are often based on convenience (e.g., using data from performance records that are “lying around anyway”), habit, or copying what others have used. Though recorded output data such as sales volume might be easily accessible, they may represent a more suitable criterion measure for some organizations (e.g., car dealers striving for high momentary sales figures) than for others (e.g., car dealers living on good word-of-mouth propaganda resulting from superior customer service).
• 2.
  Operational criteria must be sensitive in discriminating between effective and ineffective employees. This requires (a) a linkage between performance on the operational criterion and the employee's actual performance on the job (i.e., the link between the operational and the conceptual criterion) and (b) variance among employees. Speed of production, for example, may be an unsuitable criterion in cases in which speed is restrained by the tempo of an assembly line; likewise, the number of radioactive accidents is likely to be low across all nuclear power plant engineers, making this theoretically relevant criterion practically useless.
• 3.
  Operational criteria need to be practicable, as the best-designed evaluation system will fail if management is confronted with extensive data recording and reporting without seeing an evenly large return for their extra efforts.

The Criterion Problem

The criterion problem describes the difficulties involved in conceptualizing and measuring how the conceptual criterion of interest, a construct that is multidimensional and dynamic, can best be captured by an operational criterion. This problem, according to Binning and Barrett, is even bigger if the job analysis on which the criterion is based is of poor quality and if the link between the operational and the conceptual criteria has been weakly rationalized.

...