Factor, Random

When to Consider a Factor Random

It is not always clear when a factor should be considered random. The independent variable in an experiment is often considered fixed by default, because the experimenter has presumably selected all of the levels of interest for the experiment. However, there are cases when the researcher tests a sample of all potential levels of interest, even in an experiment, particularly when the factor is more abstract. As an example, suppose researchers conduct a study on the effects of a website’s interactivity on a user’s engagement in the information presented on the site. Interactivity is an abstract concept that ranges vaguely from less to more interactive with no inherent discrete levels. In an experiment in which a researcher wants to test the effects of interactivity, he or she must create distinct conditions so as to assess differences between lower and higher interactivity. Therefore, he or she may choose to create three conditions: a low interactivity condition, a medium interactivity condition, and a high interactivity condition, each with particular features that have been selected to make the website seem more or less interactive. Although these three levels are fixed in the experiment, they are intended to represent interactivity more broadly, and thus, this factor is a random factor.

Part of what defines a random factor are the assumptions made about the results. In a fixed-effects model (one in which the factor is considered to be a fixed factor), the researcher can realistically only make inferences about the effects of levels of interest measured in the study. In the example of the interactivity study, if the researcher found that the high interactivity condition leads to significantly higher engagement in the information than the low interactivity condition, the researcher can reasonably only conclude that use of the particular website presented in the high interactivity condition results in more engagement than use of the website presented in the low interactivity condition, not that interactivity in the abstract has these effects. In practice, experiments like this are commonly tested as fixed-effects models, and results are generalized to levels beyond what has been tested in the study. This type of experiment would be more appropriately tested as a random-effects model.

...