Differential Item Functioning

Differential item functioning (DIF) is the preferred psychometric term for what is otherwise known as item bias. An item displays DIF when test takers possessing the same amount of an ability or trait, but belonging to different subgroups, do not share the same likelihood of correctly answering the item. Thus, differentially functioning items elicit different responses from test takers of the same ability level. Because subgroups of test takers are often defined in terms of demographic membership (e.g., sex, race, socioeconomic status), items displaying DIF are sometimes considered “biased” against a particular subgroup. Consider, for example, the items on a standardized test of verbal ability. If the content of one item is sports related, then boys of a particular ability level may have an unfair advantage over girls of that same general verbal ability level. Thus, the item favors boys because it measures a trait other than (or in addition to) verbal ability (in this case, sports knowledge).

It is important to note, however, that the mere presence of item score differences among subgroups does not necessarily indicate the presence of DIF. To return to the example of standardized tests, we would expect 12th-grade examinees to perform better on a verbal ability test than 9th-grade examinees taking the same test. Score differences between these groups would not result because the test is biased against ninth graders, but because of true overall differences in verbal ability. A true between-group difference in ability is referred to as impact, which is conceptually distinct from DIF. Complex statistical procedures exist for distinguishing when differences stem from inherent group differences or item bias.