“Sequential Tests of Statistical Hypotheses”

Most statistical work builds on a well-established paradigm. Abraham Wald fully developed and established a new paradigm based on sequential tests of statistical hypotheses. Although formulated in 1943, the new paradigm was first published as “Sequential Tests of [Page 1501]Statistical Hypotheses” in The Annals of Mathematical Statistics in 1945. In traditional statistical hypothesis testing, an experiment is performed using a predetermined sample size chosen to ensure sufficient statistical power so the null hypothesis is likely to be rejected when in truth it should be rejected. In sequential hypothesis testing, experiments are designed so that after each observation, a decision could be made to accept or reject the null hypothesis or to gather another observation. Thus, a final decision would be based on a sequence of statistical tests.

Wald’s early work in mathematics focused on geometry, but he became a leading econometrician in Vienna. After the German occupation of Austria, Wald escaped to the United States in 1938. He was appointed a fellow by the Carnegie Corporation, studied statistics at Columbia University with Harold Hotelling, and then joined the faculty.

In 1943, while working as part of the Statistical Research Group at Columbia University, the U.S. Navy Department of Ordinance asked for help with a statistical quality control problem. Parts, including munitions, needed to be machined to a certain tolerance or the resulting equipment might not work or might wear out prematurely. It was too expensive and time-consuming to measure precisely every component, so a random sample of components was usually selected and measured to determine whether the manufacturing process was likely to lead to acceptable quality. If tolerances were not met, the manufacturing equipment or processes needed to be modified. The Columbia Statistical Research Group was asked to think about improving the efficiency of the statistical quality control analyses.

There are simple, straightforward ways to improve the efficiency of sampling. For example, you are testing 1,000 units and you are willing to reject the null hypothesis if 50 units are outside of the acceptable tolerance band: If you test 900 units and 50 units are already outside of tolerance, there is no need to test the remaining 100 units. You know you will reject the null hypothesis. Previous researchers, who were cited in Wald’s article, identified special circumstances where somewhat more sophisticated versions of sequential hypothesis testing showed greater efficiency than classic hypothesis testing approaches.

Wald realized that costs are associated with rejecting the null hypothesis when it is true, not rejecting the null hypothesis when it is false, and collecting and analyzing incremental data. These costs could be translated into weights and applied to the sequential collection of data and testing of hypotheses. Applying this approach to quality control of war material saved money but perhaps more importantly reduced the time required to get equipment to troops.

Wald realized there were many different sequential test procedures that could be implemented—but how to choose among them was the question. Wald developed a detailed framework and the sequential probability ratio test, which he proved had optimal statistical characteristics including great efficiency. He showed that compared with the most statistically powerful test from a fixed sample size experimental approach, the sequential probability test achieved comparable results with about half the expected sample size. Moreover, he showed the approach could be carried out without knowledge of the probability distribution of the test statistic.

...