Sample Size Calculations and Statistical Power

Sample size determination and prospective power analysis are important factors in planning a statistical study, because a study executed with an inappropriate sample size may result in wasted resources. If the sample is too small, the inference goal may not be achieved and true effects may not be detected, and if it is too large, money and resources have been unnecessarily expended, and subjects may have been exposed unnecessarily to a drug or the treatment. Conversely, with an optimal sample, the investigator has increased chances of detecting true effects without wasting resources or exposing subjects to unnecessary risk. For this reason, many funding agencies required sample size determinations to be supported by statistical power analysis. To illustrate, the National Institutes of Health (NIH) Policy Manual (NIH, 1988) states that requests for approval from the Office of Management and Budget to conduct epidemiological studies should include a discussion of sample size and statistical power analysis, among other things. In general, a power of at least 80% is considered satisfactory.

The process of determining the optimum sample size that will give adequate power to detect effects is a task that involves the entire research team. Several factors influence the statistical power of the study and consequently the required sample size. The factors include the hypothesis to be tested, the probability model to test the hypothesis, the significance level a, and a guess on the variance and effect size; that is, in the simplest terms, the expected difference between groups based on scientific considerations. Because some of the factors that influence the statistical power of the study have to be guessed, the statistical power should be tested under different scenarios of sample size, assumptions, and study conditions. For example, if it is important to detect a relative risk of 3, then power analysis should also include a range of neighboring values such as 2 and 4.

There are many formulas available for sample size calculations, although some of the commonly used formulas are based on approximations that assume that large sample sizes will be used in the study. In 1989, in an article titled ‘How Appropriate Are Popular Sample Size Formulas?,’ Kupper and Hafner (1989) discuss the use of some of the formulas that have large-sample approximation and recommend the use of formulas that consider statistical power. In epidemiological studies that plan to test hypotheses, it is very important to select a formula that will consider power. Note that not all the formulas consider power estimates. Generally speaking, power analysis is important in obtaining a balance between Type I and Type II errors.