Skip to main content icon/video/no-internet
icon backReturn to Entries

Assessment, Cultural Validity of

The validity of research with subcultural groups or individuals identifying themselves as members of different ethnic groups is often compromised by the use of measures developed on samples of European Americans. Less well recognized is that validity of individual assessments is also limited by the use of instruments standardized on samples that are not representative of the diversity of the American population. No cultural group is homogeneous. Each new cultural group contributes to the American experience in different ways. Thus, assessment suffers when instruments are standardized on samples with restricted cultural diversity. The dynamic contextualism that is at the heart of applied developmental science suggests several ways of exploring and improving the cultural validity of developmental assessments. This entry briefly identifies factors applied developmental scientists must consider when constructing or selecting culturally valid instruments for use.

Methodological Assumptions

In examining the validity of developmental assessments for subcultural groups, researchers may be tempted to turn to the classic research designs, especially that of the experiment. In the experiment, independent variables are manipulated by the researchers to observe the effects on the dependent variables. The true experiment requires random assignment to the control versus the experimental group or to levels of the independent variable. When applying the design of the experiment to subcultural groups, researchers would use culture as an independent variable in an attempt to understand behaviors, which are the dependent variables. However, culture is usually an assigned or status variable that cannot be manipulated easily; in particular, it cannot be randomly assigned to research participants. Thus, research using culture as an independent variable will be limited in its ability to establish causality and explain behavior (Busch-Rossnagel, 1992).

Population Generalizability

Because such research using culture as a dependent variable holds limited promise for establishing the “why” of behavior, much cross-cultural or subcultural work is conducted to examine the population generalizability of tests or results of previous research. Research on population generalizability can yield two results: significant differences between the two groups or no significant differences. The finding of no significant differences is essentially meaningless because it is an acceptance of the null hypothesis.

The finding of significant differences suggests the rejection of the null hypothesis of population generalizability, but this result may also be uninterpretable because culture or ethnicity is usually confounded with other variables, particularly socioeconomic status or educational attainment. Consider as an example the results of a study by Luis Laosa (1980), who found differences between Chicana and Anglo mothers when observing their teaching behaviors. When he controlled for the differences in the educational levels of the two groups of mothers, the ethnic group difference disappeared. Thus, when we use pseudoexperiments to look at cultural groups, we run the risk of assuming cultural differences, when they are likely the result of a confounded variable.

External Validity

These confounded relationships are seen as threats to external validity in the traditional approach to validity of research design, which seeks to ascertain general laws that hold regardless of others variables. However, within applied developmental science, confounded variables, or interactions, hold the key to understanding human behavior: Identification of the confounding variables and other dimensions of the context of research enhances the interpretation of the research findings. However, this endeavor is difficult because the key constructs are often not adequately defined. For example, what is culture? Culture includes psychological aspects of shared norms, values, and beliefs along with material entities, such as paintings, music, and dress, and social structures of organizations and institutions. To be of use to applied developmental science, culture must be defined as a developmental process, so that specification of the cultural context means that the developmental processes that are assumed to be the mediating variables between culture and outcome are explicitly defined in research hypotheses.

...

  • Loading...
locked icon

Sign in to access this content

Get a 30 day FREE TRIAL

  • Watch videos from a variety of sources bringing classroom topics to life
  • Read modern, diverse business cases
  • Explore hundreds of books and reference titles

Read next

More like this

Sage Recommends

We found other relevant content for you on other Sage platforms.

close

Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches

close
close
close Signed In to profile You are signed in as:
close
Logged in Institution Institution