Criterion Problem

Criterion Problem in Academic Settings

In academic settings, there are two typical and readily available criterion variables: GPA and student retention (whether a student has remained with the university). Although each of these criteria is certainly important, most researchers agree that there is more to being a good student than simply having good grades and graduating from the university. This is the essence of the criterion problem in academic settings: the specific behaviors that persons on the admissions staff would like to predict are not captured well in the measurement of the readily available GPA and retention variables. The first step in solving the criterion problem, then, is identifying which types of behaviors are important.

Thomas Taber and Judith Hackman provided one of the earliest attempts to model the performance behaviors relating to effective performance in undergraduate college students. After surveying many university students, staff, and faculty members, they identified 17 areas of student performance and two broad categories: academic performance and nonacademic performance. The academic performance factors, such as cognitive proficiency, academic effort, and intellectual growth, are reasonably measured in the easily obtained overall GPA; these factors are also predicted fairly well by the traditional variables that predict GPA (e.g., standardized tests, prior GPA). The nonacademic performance factors, on the other hand, are not captured well in the measurement of GPA. These factors include ethical behavior, discrimination issues, and personal growth, and none of these are well predicted by traditional predictor variables.

In more recent research, Frederick Oswald and colleagues modeled undergraduate student performance for the purposes of scale construction. By examining university mission statements across a range of colleges to determine the student behaviors in which stakeholders are ultimately interested, they identified 12 performance factors that can be grouped into three broad categories: intellectual, interpersonal, and intrapersonal. Intellectual behaviors are best captured with GPA and include knowledge, interest in learning, and artistic appreciation. Interpersonal behaviors include leadership, interpersonal skills, social responsibility, and multicultural tolerance. Intrapersonal behaviors include health, ethics, perseverance, adaptability, and career orientation. However, GPA does not measure the interpersonal or intrapersonal behaviors particularly well. It is interesting to note that this research also showed that while traditional predictors (e.g., standardized tests, prior GPA) predict college GPA well, they do not predict the nonintellectual factors; noncognitive variables, such as personality and scales developed to assess these performance dimensions, are much better predictors of the nonintellectual factors.

...