Main Effect

A main effect is a statistical term associated with experimental designs and their analysis. In the analysis of variance statistical test, which often is used to analyze data gathered via an experimental design, a main effect is the statistically significant difference between levels of an independent variable (e.g. mode of data collection) on a dependent variable (e.g. respondents' mean amount of missing data), ignoring the influence of other factors. To better understand the statistical concept of a main effect, it is helpful to understand a few key terms and experimental conditions under which a main effect may be found.

When conducting research, it is not uncommon to use a factorial analysis of variance to determine how two or more categorical independent variables (called factors in analysis of variance) affect a continuous dependent variable. Each factor in a factorial analysis of variance contains two or more categories or levels of that factor that are manipulated to determine how the factor influences the dependent variable.

For example, a survey researcher investigating item nonresponse may want to know how the two factors survey type (containing the levels “paper-and-pencil survey” vs. “computer-assisted survey”) and mode of administration (containing the levels “interviewer-administered” vs. “self-administered”) separately and together influence the dependent variable percentage of item nonresponse. A sample of respondents is randomly assigned to one of the four conditions in the experiment: (1) paper-and-pencil interviewer-administered, (2) paper-and-pencil self-administered, (3) computer-assisted interviewer-administered, and (4) computer-assisted self-administered. A factorial analysis of variance can be used to investigate the main effects of the two factors (survey type and mode of administration) on the amount of item nonresponse.

In such a factorial analysis of variance, a main effect is a statistically significant difference between the levels of one factor on the dependent variable regardless of the influence of any other factor. In this survey research example, a main effect for the factor “mode of administration” would occur if self-administration resulted in a statistically significant difference in the average amount of item nonresponse when compared to interviewer administration, regardless of any influence that the factor “survey type” (paper-and-pencil vs. computer-assisted) might have on item nonresponse. Ignoring the influence of all other factors on the dependent variable when determining a main effect is referred to as collapsing across levels of the other factor. The illustrations in Figures 1 and 2 help visualize this process.

Figure 1 illustrates a main effect for mode of administration. The two parallel lines on the graph show a difference in the amount of item nonresponse between self-administered surveys and interviewer-administered surveys. Mentally collapsing across the factor survey type, one can see that self-administration resulted in more item nonresponse (s) than interviewer [Page 448]administration (i). There is no main effect for survey type because each level of that factor contains identical amounts of item nonresponse (p and c).

Figure 1 A main effect for mode of administration

Figure 2 Main effects for survey type and mode of administration

Figure 2 shows a slightly more complex relationship, with main effects for both survey type and mode of administration. The main effect for the factor survey type shows that paper-and-pencil surveys have a greater amount of item nonresponse (p) when collapsed across mode of administration than the amount of item nonresponse in computer-assisted surveys (c). The main effect for the factor mode of administration shows that self-administration results in more item nonresponse (s) than interviewer administration (i) when collapsed across levels of survey type.

...