Type II Error

Hypothesis Testing

Developed by Jerzy Neyman and Egon S. Pearson, hypothesis testing has become one of the most widely used quantitative methodologies in almost all areas dealing with experiments and data. In this decision process, a simple statement or null hypothesis is formulated on the true status of an unobserved phenomenon. At the same time, an alternative hypothesis is defined to reflect the opposite situation. The outcome of the hypothesis testing can be one of the two following results:

• The null hypothesis can be rejected in favor of the alternative because the observed pattern in the sampled data cannot be explained merely by random variation.
• There is not enough evidence to reject the null in favor of the alternative because the observed pattern in the sampled data is most likely a result of chance.

Note that failing to reject the null is not the same as claiming that the null is true. In principle, one is testing for “guilt” rather than “innocence”: Like in any judiciary system, a defendant (hypothesis) is considered innocent (true) until proven guilty (false) beyond any reasonable doubt. The alternative hypothesis will, therefore, always reflect the property that one would like to prove. Consider, for instance, a clinical trial comparing an experimental drug with a standard treatment. The purpose of the trial is to detect a difference in the effect of both drugs. The null hypothesis will state that the experimental drug is, on average, not better than the conventional drug. Any differences that are observed can be explained by random variation in the measurement process. The alternative hypothesis can be formulated in several ways. For instance, a two-sided alternative could state that the experimental drug has, on average, a different effect. If one expects to find an improvement, then a one-sided alternative like “the new drug is better, on average, than the conventional drug” can be considered.

Once the appropriate hypotheses have been formulated, one would like to test whether the null can be rejected in favor of the alternative. At this point, an experiment can be designed, a relevant test statistic and its distribution under the null can be derived, and data can be collected. When the test favors a decision that is in agreement with reality, a correct decision has been made. However, this decision process can also result in several errors (see Table 1).

On the one hand, a Type I error occurs when the null hypothesis is wrongly rejected in favor of the alternative. On the other hand, a Type II error is made when a false null hypothesis is not rejected. In the clinical trial example, a Type II [Page 1578]error occurs if there is not enough evidence against the null, when in reality the drug has an effect different from that of the conventional treatment.

...