Measurement Error

Understanding the Meaning of Error

Errors can be made at any or all of these steps. Education may be denned incorrectly, either at the conceptual or the operational level. An incorrect conceptual definition of education is a theoretical mistake—a misunderstanding of what education means in the context of theory construction. For example, education might be conceived by a researcher as a person's “mastery of correct social appearances.” An incorrect operational definition is an error in the way education is conceived for the purpose of gathering information. An operational definition corresponding to the “social appearance” conceptual definition might be “the extent to which a person exhibits behaviors that are seen as refined in a given culture.” This operational definition would lead to seeking information about people that would allow them to be placed in categories of social sophistication.

Errors in conceptual definitions are identified and debated in theoretical discussions of the concepts (constructs) that are employed in social research. A researcher who argues that education means “a mastery of social appearances” may be confronted by another who claims that education really concerns “the accumulation of knowledge.” The research community debates the most useful conceptual definitions of the construct. Such debates take place as research, employing alternative conceptual definitions of key concepts, is carried out. Theoretical arguments and empirical data lead the research community to adopt some conceptual definitions as correct and to treat others as erroneous. Measurement begins with reasoned decisions about the essence of the concept to be gauged.

This theoretical beginning is essential to the measurement process, but discussions of measurement and measurement error typically start at the operational definition phase. The conceptual definition is taken as established. Attention is focused on how well the theoretical idea is translated into more concrete language that allows information about people to be gathered and used to assign them to categories. The issue of measurement error becomes, “How well do the adopted operational definition and its attendant procedures capture the concept of interest?” There may be a mismatch between the concept and its operational definition. Education, conceptually denned as “the accumulation of knowledge,” might be operationally denned as “performance on an examination of arithmetic and reading skill.” The education scores resulting from the examination may be criticized because the test does not address other significant areas of knowledge and learning. To the degree that the operational definition fails to capture the theoretical concept, the resulting measure is termed invalid.

...