Cross-Cultural Assessment

Introduction

Cross-cultural assessment refers to the use of assessment procedures with testees from different cultural backgrounds. Various instances can be distinguished: (i) an existing procedure is used in another country than the one in which it was originally designed, (ii) individuals within a single country differ from each other in ethnic or cultural background, or (iii) testees currently living in different countries take part in the same assessment procedure. Underlying all these forms of cross-cultural assessment are certain issues about the cross-cultural comparability or equivalence of test scores. These are briefly discussed in the first section. In the second section some traditions of cross-cultural test use are mentioned with a view to evaluate how serious the threats are to meaningful and valid cross-cultural assessment.

Equivalence Issues

Suppose a second generation migrant takes a test of word knowledge in a language that is not the home language of the parents. Then the question arises whether the obtained score is affected by the home language of this testee, and whether this is relevant for the interpretation of the score. Common sense tells us that the score can be a valid indicator of the current level of skill or achievement, but that it is likely to give a biased impression of the testee's language abilities, and about the testee's intellectual capacities in case the word knowledge test is part of an intelligence battery. This example shows that scores on one and the same instrument can be used to make inferences or generalizations about more than a single trait.1

If persons from different cultural backgrounds, who have the same test score, do not have the same standing on the trait to be assessed, the instrument concerned is called biased or inequivalent. Thus, it often depends on the generalization whether or not scores are biased for testees belonging to a certain cultural population. A definition in which this is taken into consideration is the following: cultural biasedness or inequivalence implies that an observed difference between two cultural groups on a score variable is not matched by a corresponding difference in respect of the trait in terms of which the scores are interpreted.

From the 1960s cultural bias or inequivalence began to be addressed as a psychometric issue. Initially the focus was very much on item bias; items that were unexpectedly difficult for a new cultural group, to which a test was applied, were identified as biased and removed from the instrument. Gradually awareness increased that inequivalence is a more comprehensive issue (e.g. Malpass & Poortinga, 1986; Poortinga & Malpass, 1986). A framework for the analysis of cultural equivalence (or absence of bias) has been described by Van de Vijver and Leung (1997; Van de Vijver & Poortinga, 1997). They distinguish three levels of equivalence that are hierarchically ordered:

• (i)
  structural or functional equivalence; viz., a test measures the same trait (or set of traits) cross-culturally.
• (ii)
  metric or measurement unit equivalence; viz., differences between scores have the same meaning across cultures; the metric of the score variable is the same.
• (iii)
  scale equivalence or full score equivalence; viz., scores have the same meaning cross-culturally and allow identical (quantitative) interpretations in terms of norms or criteria.

The general consequence of all forms of bias is that they make the scores of a test in some way incomparable across the cultural populations concerned. The test user or test author has to show that scores are not affected by bias [Page 285](Poortinga & Malpass, 1986). This is done by demonstrating that scores meet certain conditions of invariance of relationships cross-culturally. The remainder of this section gives examples of various levels of bias and how these can be identified.

...