Analysis of Variance (ANOVA)

Analysis of variance (ANOVA) is a statistical technique that is used to compare groups on possible differences in the average (mean) of a quantitative (interval or ratio, continuous) measure. Variables that allocate respondents to different groups are called factors; an ANOVA can involve one factor (a one-way design) or multiple factors (a multi-way or factorial design). The term analysis of variance refers to the partitioning of the total variation in the outcome variable into parts explained by the factor(s)—related to differences between groups, so-called explained or between variation—and a part that remains after taking the factor(s) into account, the so-called unexplained, residual, or within variation.

Consider a one-factor example in which the target population contains respondents from four different ethnic backgrounds (e.g. Chinese, Japanese, Korean, Vietnamese) and the research question is whether these ethnic groups have different average incomes. The null and alternative hypotheses for this example tested with the ANOVA are Ho: μ1 = μ2 = μ3 = μ4 and HA: not all μj equal, where μj (j = 1,…, 4) denote the population mean incomes for the ethnic groups. The test statistic, denoted by F and following an F-distribution, is based on the ratio of the between variation (the variation between the sample group means) and the residual (within groups) variation. A statistically significant result is obtained if the former is large compared to the latter. The conclusion that can be drawn from a significant result is that the mean incomes for the ethnic groups are not all four equal. Of note, no causal conclusions can be made, since this is a nonexperimental study.

In a factorial design, for instance, by the inclusion of gender as a second factor in the previous example hypotheses about main and interaction effects can be tested. A significant main effect of gender implies that the marginal mean incomes of men and women (irrespective of the four ethnic groups) differ. A significant interaction effect of gender and ethnicity on income implies that the differences in mean income between men and women are different among the four ethnic groups.

Some important assumptions underlying the ANOVA are independence of observations and approximately normally distributed residuals, as well as approximately equal residual variances in the subgroups.

Note that the practical conclusions that can be drawn from an ANOVA are somewhat limited. The null hypothesis “all means are equal” is evaluated against the rather uninformative alternative hypothesis stating nothing more than “not all means are equal.” Rejecting the null hypothesis in an ANOVA does not inform the researcher about which pairs of means differ from each other. Therefore, an ANOVA is often followed by pair-wise comparisons to further investigate where group differences are found. Since several tests are performed in such a case, the alpha level used per comparison is usually corrected to protect for an increased Type I error probability (post-hoc corrections). Several correction methods are developed, but unfortunately it is not always clear which method should be preferred. Another approach for further investigation of differences between specific means or investigation of a specific structure in the group means is contrast testing.

...