John Henry Effect

The term John Henry effect was coined to explain the unexpected outcome of an experiment caused by the control group's knowledge of its role within the experiment. The control group's perceived role as a baseline or a comparison group to the experimental condition, specifically one testing an innovative technology, can cause the control group to behave in an unnatural way to outperform the new technology. The group's knowledge of its position within the experiment as a baseline comparison causes the group to perform differently and, often more specifically, better than usual, eliminating the effect of the experimental manipulation. Deriving its name from the folktale the “Ballad of John Henry,” the John Henry effect is similar to the Hawthorne effect in which participant behavior changes as a result of the participants’ knowledge that they are being observed or studied. This change in participant behavior can confound the experiment rendering the results inaccurate or misleading.

The effect was studied and explained by education researcher Robert Heinich after his review of several studies that compared the effects of television instruction with those of standard classroom [Page 661]teaching. Heinich noted that many of these studies demonstrated insignificant differences between control and experimental groups and often included results in which the control group outperformed the experimental condition. He was one of the first researchers to acknowledge that the validity of these experiments was compromised by the control groups’ knowledge of their role as a control or baseline comparison group. Comparing the control groups with the title character from the “Ballad of John Henry,” Heinich described how a control group might exert extra effort to compete with or even outperform its comparison group.

In the “Ballad,” title character John Henry works as a rail driver whose occupation involves hammering spikes and drill bits into railroad ties to lay new tracks. John Henry's occupation is threatened by the invention of the steam drill, a machine designed to do the same job in less time. The “Ballad of John Henry” describes an evening competition in which Henry competes with the steam drill one on one and defeats it by laying more track. Henry's effort to outperform the steam drill causes a misleading result, however, because although he did in fact win the competition, his overexertion causes his death the next day.

Heinich's use of the folktale compares the performance by a control group with the performance of Henry in which an unexpected result develops from the group's overexertion or out-of-the-ordinary performance. With both Henry and the control group, the fear of being replaced incites the spirit of competition and leads to an inaccurate depiction of the differences in performance by the control and experimental groups.

The effect was later studied extensively by Gary Saretsky, who expanded on the term's definition by pointing out the roles that both competition and fear play in producing the effect. In most cases, the John Henry effect is perceived as a control group's resultant behavior to the fear of being outperformed or replaced by the new strategy or novel technology.

...