Unit of Analysis

When engaging in research, it is essential to have a clear understanding from the outset what the study is trying to accomplish. That is, what the specific research question(s) is(are), and what data are necessary to collect to answer the research question(s). Otherwise, confusion typically ensues. One of the most fundamental considerations in conducting research is to determine what the primary unit that will be the subject of statistical analysis is, or should be. This is called the unit of analysis. Often, it is dictated by the data that are collected, rather than by a sound theoretical justification. Therefore, it is necessary to be clear as to what the unit of analysis should be before data collection. For example, in educational research, the unit of analysis could be people or schools. The unit of analysis may be the same or different from the unit of generalization, the unit of sampling, or the unit of measurement.

To understand the unit of analysis, it is essential to understand these related terms. An example may help clarify the distinction between the terms. Consider the case where a researcher seeks to investigate the effect of a curriculum on student achievement. Data can be collected at a variety of levels: the student level, the classroom level, the school level, the district level, the state level, and so on. Here, the example is restricted to the first three levels: the student level, the classroom level, and the school level. Similarly, the researcher can sample schools, classrooms, or students to collect the data. The choice of which unit to choose for sampling and data collection depends, in part, on the unit of generalization. To understand the unit of generalization, it is useful to understand what it would mean to generalize at the three different levels. If the study seeks to generalize at the school level, that would mean that the researcher is [Page 1585]interested in how implementing the new curriculum effects the achievement of students at the school level. The interest is not on individual achievement but on the average achievement for the school. Similarly, the researcher may wish to generalize to the classroom level, at which the results would be reported for each classroom. Last, the goal could be to determine the effect of the instruction at the individual level, and hence, the results of the individual would be paramount. Therefore, if the goal were to generalize at the school level, for instance, it would be wise to do the sampling at the school level, to be sure that there are enough schools on which valid inferences can be made, and that the sample of schools is representative. If the goal is at the individual student level, in contrast, then it may be possible to gather enough students on which to base valid inferences, and also that is representative, without having enough schools for a valid inference to be made. Therefore, prior to sampling and data collection, the unit of generalization should be determined.

It is typical that the unit of sampling and the unit of generalization are the same, but they may be different from the unit of measurement. Suppose in the example that the unit of generalization is to be schools. Therefore, the results will be school-level achievement results, and the sampling will likely occur at the school level. However, the unit of measurement will likely take place at the student level. To determine the school-level achievement, it is necessary to collect data on student achievement and possibly then to aggregate that student-level data into school-level data, perhaps as the mean of the student-level results. It may be that other data are collected as well, not only achievement data on students, and as such data collection could take place at any and all of the three levels: at the student level, classroom level, and school level. At the student level, data may be collected on not only the achievement of the student but also on other variables such as the gender of the student, the ethnicity of the student, and so on. At the classroom level, data regarding the number of students in a classroom, the number of years of experience of the teacher, and the gender of the teacher may be collected. And at the school level, data regarding the total enrollment of the school, the demographic makeup of the school, the student-to-teacher ratio, and so on, can be collected. All these data can be used to assess the school-level performance. Therefore, although the unit of sampling was the school, the unit of measurement may be any one of the three levels, or some combination thereof. In this case, the unit of measurement is different from the unit of generalization and from the unit of sampling.

...