Incremental Validity

Estimating Incremental Validity

The most widely used method of assessing incremental validity is hierarchical multiple regression, which allows researchers to assess the amount of variability explained by the predictors, after previous predictors have already explained their share of variance. Adding predictors to a model necessarily increases its predictive power, so an important question becomes, How do you determine when incremental validity ceases to be significant?

Arguably, the most important consideration is parsimony—that is, defining a model that is as simple as possible, without sacrificing substantial predictive power. One way to do this is by examining a model's adjusted R2, which indicates the proportion of variance explained in the outcome after the estimate has been slightly decreased for each predictor included in the model. This is done because adding predictors to a model always increases its predictive power. So, to increase the adjusted R2 value, the incremental validity of each added predictor must outweigh its subsequent decrement in the adjusted R2.

Applying Incremental Validity

Of all forms of validity (e.g., construct, criterion), incremental validity is the most applied, in that it is [Page 341]typically used to better predict valued outcomes in real-world settings and often influences (and is influenced by) other considerations, such as time, money, and effort. This is especially true in the area of personnel psychology, where incremental validity is given more attention than in “purer” forms of psychology. Some have even developed utility analyses, which are used to translate organizational incremental validity estimates into financial savings.

One particularly prevalent application of incremental validity has been predicting applicants' future job performance. Here, research has led to the nearly unequivocal conclusion that general mental ability (GMA, or g) is the best predictor of job performance across nearly all occupations. However, tests of GMA do not predict performance perfectly, so an important finding is that, under certain circumstances, predictors such as personality inventories, structured interviews, and work samples add incremental validity to the prediction of job performance.

...