Multiple Comparison Procedures

The issue of multiple comparisons has created considerable controversy within epidemiology. The fundamental questions are which procedure to use and whether probabilities associated with multiple tests should be adjusted to control Type I errors. The latter topic appears the most contentious.

It is helpful to make a distinction between multiple comparisons, which usually involve comparison of multiple groups or treatment arms on one dependent variable, and multiple testing, which usually involves the comparison of two (or more) groups on multiple dependent variables. Although both procedures raise many of the same questions, they differ in some important ways. Multiple comparison procedures are more formalized than those for multiple testing.

A Type I error, also referred to as alpha or by the Greek letter a, refers to the probability ...