Intercoder Reliability Techniques: Scott’s Pi

Scott’s pi is a measure of intercoder reliability for nominal level data with two coders. Scott’s pi was developed by William A. Scott in 1955. The formula for Scott’s pi is:

π=Pr(a)Pr(e)1Pr(e).

Pr(a) represents the amount of agreement that was observed between the two coders. Pr(e) represents the amount of agreement that is expected between the two coders. Scott’s pi works by comparing the amount of agreement observed between the two coders with how much agreement is expected if both coders chose randomly. If both coders are consistent, the amount of observed agreement will be higher than the amount of agreement that is expected due to chance. In essence, Scott’s pi compares coding results to what would happen if two people coded the data at random. ...

  • Loading...
locked icon

Sign in to access this content

Get a 30 day FREE TRIAL

  • Watch videos from a variety of sources bringing classroom topics to life
  • Read modern, diverse business cases
  • Explore hundreds of books and reference titles