Multilevel Modeling

Consequences of Ignoring Multilevel Influences

A number of theoretical and methodological fallacies may occur as a result of ignoring multilevel structures. This entry focuses on the following theoretical fallacies:

• Misspecification fallacies involve assigning the wrong level of theory to a construct. For example, a researcher may wish to study organizational flexibility, but using only measures of individual employee flexibility does not operationalize the construct at the appropriate (organizational) level.
• Cross-level fallacies involve inappropriately equating findings at one level to other levels. If a researcher adopts a theory of individual flexibility to understand organizational-level flexibility, he or she must show how the theory adequately explains phenomena at the organizational level or risk committing a cross-level fallacy.
• Contextual fallacies involve instances in which a researcher ignores important contextual (higher-level) influences on lower-level relationships and outcomes. This is a big one in psychology: For years, the field studied individuals as if situations did not matter. For example, if organizational flexibility constrains or enhances individual-level flexibility, then both levels should be studied to truly understand how individual flexibility relates to individual-level outcomes.

The bottom line is that ignoring multilevel relationships when they exist may lead us to inappropriate conclusions from our research. This is not a trivial issue because the validity of our science depends on adequately understanding these relationships.

Properties of Emergence

Measuring individual-level cognitions, attitudes, and behaviors is relatively straightforward. But how do such constructs manifest themselves in higher-level units such as teams and organizations? Sometimes, higher-level constructs are easy to measure (e.g., organizational size, location, or money spent on human resources). But more often then not, researchers want to understand psychological phenomena that exist at higher levels (e.g., climate, customer satisfaction, or turnover). Perhaps one of the most important theoretical advancements in recent years has been the clarification of the process of emergence—that is, how lower-level psychological constructs manifest as higher-level constructs. There are numerous ways this can occur, but for simplicity's sake, two extreme forms will be discussed.

First, composition represents similarity, consensus, or “sharedness” among within-unit observations. For example, if lower-level observations are hypothesized to be sufficiently similar to form some aggregate, higher-level construct, one is dealing with a composition model. In such a model, the higher-level construct is the unit average of within-unit members' scores. For example, organizational climate may be the average of all employees' climate perceptions within that organization. Notice what just happened: The mean scores within the organization were used to create an organization-level score based on that mean. Hence, there is no direct measure of organizational climate in this case but instead an indirect measure based on an average of employees' scores. Beyond having a strong theory to justify aggregation, one must statistically demonstrate it through indexes of agreement and consensus. A composition model, by focusing on similarity, frequently makes a claim that the lowerand higher-level constructs are reasonably isomorphic (i.e., similar in their nature).

...