Hypothesis Testing

A scientific hypothesis is tested by evaluating the logical consistency of its implications and/or the accuracy of its predictions. Other grounds for assessing hypotheses include breadth of prediction, scientific fertility, simplicity, and aesthetic appeal; however, the term hypothesis testing refers only to accuracy. Statistical hypothesis testing, a form of inductive inference, is used extensively in medical research and described here as a form of proof by contradiction.

A hypothesis is rejected by a test if the hypothesis logically implies something false or strongly predicts something contradicted by data. The 2,500-year-old proof by Hippasus of Metapontum that √2 is not a ratio of whole numbers exemplifies the former. Hypothesizing the opposite, that √2 = a/b for whole numbers a and b, Hippasus deduced the impossible: that both numerator and denominator must remain divisible by 2, even after all possible cancellations of 2 from both a and b. Unable to deny the logic of this contradiction, a rational mind instead rejects the hypothesis, concluding that √2 cannot be such a ratio. This is proof by contradiction or, from the Latin, reductio ad absurdum.

Data may also contradict hypotheses. In deterministic settings, that is, when predictions are made with certainty because all relevant influences are presumed known, one valid incompatible datum overturns a hypothesis. The hypothesis “Elixir A cures all cancer” is overturned by a single treatment failure, demonstrating conclusively that other treatment is sometimes required. This is an empirical analog of proof by contradiction.

In medical sciences, though, knowledge is incomplete and biological variability the rule. Hence, determinism is rare. Medical hypotheses describe tendencies that are exhibited variably, in [Page 608]complex systems governed by probabilities rather than individually predictable fates. The hypothesis “Elixir A increases the fraction of cases alive two months postdiagnosis” does not imply that a particular individual will live for 2 months. Unless 2-month survival is already extremely high, this hypothesis cannot be overturned by one or even several early deaths.

But suppose, in a trial of Elixir A, that all 10 clinically similar but otherwise unrelated patients who receive it die before 2 months postdiagnosis. If extensive data show that only half of similar untreated cases die this quickly, most would reconsider further use of Elixir A. Although 10 patients on any new treatment may all be unlucky, the chance of this happening in a specified study is below 2-10 = .098% if Elixir A is beneficial. Logically, either luck has been extraordinarily poor, or Elixir A doesn't work as hypothesized. A longer consecutive run of deaths would be even less likely, for example, 2-15 = .003% for 15 deaths, and hence more difficult to attribute to bad luck. With “enough” accumulated evidence, most persons bow to its weight and reject the initial hypothesis, because the hypothesis made a strong prediction that failed. Specifically, the hypothesis had predicted strongly, that is, with very high probability, that data would show a 2-month case fatality more similar to what the hypothesis describes (< 50%) than to what was actually seen (100%).

Such probabilistic proof by contradiction exemplifies statistical hypothesis testing, the focus of this entry. Statistical hypothesis tests influence most research on which medical decisions are based. Their general use is to select, from among statistical associations in data, those hardest to explain by play of chance in a particular data sample. The selected associations, unless explicable by study design problems, receive preferential evaluation for causal involvement in disease initiation, promotion, progression to disability, and therapeutic benefit.

...