Reliability, Unitizing

When Not to Measure Unitizing Reliability

There are two circumstances in which unitizing reliability need not be measured. One circumstance is when the act of unitizing itself is trivial because unit identification is obvious. This is the case for most circumstances involving mediated messages; the beginning and ending of the television program, the commercial, the music video, the newspaper article, and analogous units are usually clear. A second circumstance is when measurement is performed on a continuous scale. Nonverbal behavior often falls within this category, as examples such as the distance separating two conversationalists, their angle of body orientation, direction of eye gaze, and speech rate, pitch, and volume are all continuous measures. As a result, this entry is limited to the study of verbal behavior, in which unitizing reliability is often problematic and its measurement often a requirement.

In these circumstances, the unitizing stage must be treated as seriously, and descriptions of this stage in research reports described as carefully, as the categorizing stage. Researchers must carefully consider the correct unit size for the study’s particular needs. Definitions of the unit need to be specified as clearly as possible. In instances to be described later in this entry, unitizing reliability needs to be computed, and unitizers need to be trained until their reliability is sufficiently high. Research reports should describe these issues in detail. The rest of this entry describes how the analyst should proceed in order to ensure that the reliability of unit assignment is adequate. The assumption throughout is that unitizers are working with written transcripts of talk; content or interaction analysis should never be performed directly from recordings.

Unitizing Verbal Messages

The history of published content and interactional analytic studies relying on verbal messages reveals many instances in which the importance of unitizing reliability was ignored. Sometimes, reliability was not reported, or reported using a poor index. Sometimes, the coding unit was not described, or was described too vaguely; one often sees the ambiguous term “thought unit” used without definition. Although the situation has improved, these problems still occur.

The analyst’s first decision is determining the correct unit size, which is totally dependent on the needs of the given study. The following are four possibilities, each with an example of a circumstance in which it is the correct size for a specific research question.

...