Random-Effects Models

Random-effects models are statistical models in which some of the parameters (effects) that define systematic components of the model exhibit some form of random variation. Statistical models always describe variation in observed variables in terms of systematic and unsystematic components. In fixed-effects models, the systematic effects are considered fixed or nonrandom. In random-effects models, some of these systematic effects are considered random. Models that include both fixed and random effects may be called mixed-effects models or just mixed models.

Randomness in statistical models usually arises as a result of random sampling of units in data collection. When effects can have different values for each unit that is sampled, it is natural to think of them as random effects. For example, consider observations on a variable Y that arise from a simple random sample from a population. We might write the model

where μ is the population mean and ∊i is a residual term to decompose the random observation into a systematic part (a fixed effect) μ, which is the same for all samples, and an unsystematic part ∊i, which in principle is different for every observation that could have been sampled. Because the ith individual in the sample is a random draw from the population, the value Yi associated with that individual is random and so is ∊i. Thus in this simple model, μ is a fixed effect and ∊i is a random effect.

More complex sampling designs typically lead to more complex statistical models. For example, two-stage (or multistage) sampling designs are quite common. In such designs, the sample is obtained by first sampling intact aggregate units such as schools or communities (generically called clusters in the sampling literature), then sampling individuals within those aggregate units. If our observations on Y arise from a two-stage sample, then a natural model might decompose the variation in Y into systematic parameters associated with the overall population mean (μ), and the effect of the aggregate unit (ζ), in addition to the residual (∊). Because there are now two aspects of the data to keep track of (aggregate units and individuals within those units), we might use two subscripts, denoting the jth observation in the jth aggregate unit (the ith cluster) by Yij and write the model as

Here ζi, the effect of the ith aggregate unit, is a random effect because the aggregate units were chosen at random as the first stage of the sample. Similarly, the ∊ijS are also random effects because the individuals within the aggregate units were chosen at random in the second stage of the sample.

In most research designs, the values of the outcome variable associated with particular individuals (or residuals associated with particular individuals) are of little interest in themselves, because the individuals are sampled only as a means of estimating parameters describing the population. Similarly, in research designs that use multistage sampling, the specific values of the cluster-level random effects are of little interest (because inferences of interest are about the larger population). However, in both cases the variances associated with individual-level or cluster-level random effects are of interest. These variances, often called variance components, have an impact on significance tests and precision of estimates of fixed effects, and they play a key role in [Page 1198]determining the sensitivity and statistical power of research designs involving random effects. Consequently, random-effects models in some areas (such as analysis of variance) are often called variance component models.

...