Verbal Protocol Analysis

Conducting Verbal Protocol Analysis

Like any data-collection or analysis technique, there is no best way to conduct VPA; however, the following general principles likely apply to most uses.

• Collect concurrent data. Research strongly suggests that data collected in real time are superior to those collected after the fact; therefore, whenever possible, have participants vocalize their thoughts as they engage in the task.
• Record the data. Early forms of VPA relied on the experimenter's notes because recording equipment was not readily available. However, the ubiquity of analog and digital recording equipment makes recording protocols simple and cost-efficient.
• Transcribe the data. Although qualitative data are never easy to manage, computer programs are available to provide assistance, but only when data are fully transcribed.
• Plan your work. Have an idea of what you are looking for before you immerse yourself in the data. As with any research technique, this means drawing from relevant theory, making specific predictions about what you expect to find, and specifying defensible ways to determine whether your predictions were met.

Uses in the Organizational Sciences

Although VPA is certainly considered an unorthodox research method among most organizational scientists, it is not completely unheard of. A handful of studies have used VPA in an organizational context, and in most cases, it has helped to answer questions that are firmly rooted in traditional organizational science domains but require information that traditional methods simply cannot provide.

A good example is the use of VPA to investigate the processes that individuals use when deciding whether to apply for a given job. The researcher could use more traditional techniques, such as self-report, by asking participants what information they think they typically pay attention to while reading job ads, or a type of policy-capturing methodology to elicit similar information. However, VPA is likely a better choice because it allows the researcher to directly assess the real-time reports of participants' strategies, a benefit that few methodologies offer. Other examples of the use of VPA in the organizational sciences include examining the thought processes that job seekers use to evaluate potential employers' reputations and assessing the construct validity of the organizational culture profile through VPA alone, which brings up an interesting point regarding the potential use of VPA as a psychometric aid.

Uses in the Organizational Sciences

Traditional psychometric assessment devices are necessary for the further development of quality measurement systems, but they may not be entirely sufficient. This is especially true when assumptions are made regarding the cognitive processes that individuals use when responding to a given instrument. The importance of this sentiment—often referred to as cognitive process validity—was recently recognized [Page 867]by the Society of Industrial and Organizational Psychology, which argued that such assumptions should be empirically tested before instruments are used in applied contexts. Verbal protocol analysis provides one possible means of doing this.

...