Intercoder Reliability Techniques: Percent Agreement

A number of techniques are available to communication scientists to assess agreement among independent judges or observers, and many of these techniques highlight a diverse—and sometimes different—set of conceptual and operational properties. Percent agreement (also known as simple agreement) is one intercoder reliability technique that relies on the proportion of agreement of coded units between two independent judges. The simplest form of intercoder reliability, percent agreement is best suited for a nominal measurement level. The percent agreement between two judges depends on a number of properties, but perhaps the most influential properties are the number of coding categories and the frequency of categories used by independent judges. A useful heuristic for understanding agreement for individual variables, percent agreement should be used in tandem with more ...