Item and Test Bias

Item Bias and Differential Item Functioning

Although the terms item bias and DIF often are used interchangeably, DIF refers to differences in the statistical properties of an item between groups of examinees of equal ability. Two types of DIF can exist. Uniform DIF is a difference in item performance that is consistent across the ability distribution, whereas nonuniform DIF is a difference that is not consistent across the ability distribution (i.e., a group by ability-level interaction). Groups often are referred to as the reference (e.g., majority) and focal (e.g., minority or studied) groups. The concept of comparing groups of equal ability is a cardinal feature separating DIF from the traditional item bias detection methods. Traditional methods, because they do not control for ability differences, are affected by differences in the examinee group ability distributions. Overall ability differences may explain differential item performance, resulting in an item appearing to be, for example, more difficult when the examinees in the focal group are less able overall. Impact is a more appropriate term to refer to differences in item performance that can be explained by group ability differences. DIF detection methods “condition on” or control for ability, meaning that examinees are necessarily matched on ability; thus, only examinees of equal ability (e.g., total test score) in the reference and focal groups are compared.

Items that exhibit DIF threaten test score validity and may have serious consequences for groups as well as individuals, because correct responses are determined by the trait claimed to be measured and factors specific to group membership. The most obvious consequence is the potential impact of DIF on the observed score distributions of specific groups. The less obvious consequence of DIF, yet critically important to the construct validity of a test, is its impact on the meaning and interpretation of test scores, even in the absence of mean score differences between groups. DIF items may cancel and result in similar score distributions across groups. However, when scores are composed of different items systematically scored as correct, it is invalid to infer that “equal” scores are comparable or have the same meaning. In fact, the Standards for Educational and Psychological Testing (Section 7.10) states that mean score differences are insufficient evidence of bias.

...