Hypothesis Testing

Hypothesis testing constitutes a quantitative method for refuting or supporting theories based on observations from a sample. This entry provides an overview of a commonly used statistical procedure that combines Ronald Fisher’s statistical theory with Jerzy Neyman and Karl Pearson’s approach to drawing inferences from a sample to the population. Historically, these two competing statistical theories produced heated debates between their originators. Later, however, ideas from the two approaches were merged in the social sciences and are often treated as complementary ways for making inferences about the probability of the existence of the hypothesized relationship in the population of interest.

Hypothesis testing in social sciences typically involves testing a hypothesis of relationship between variables (H1) against a hypothesis of no relationship (H0). For example, H1 will suggest that watching educational television programming is associated with better school readiness skills in 4-year-old children, whereas H0 will maintain that there is no association between watching educational television programming and school readiness. Statistical tests enable researchers to reject the null hypothesis and conclude that the relationship between the variables observed in the sample was unlikely to occur by chance. This can be interpreted as evidence in support of the theory about the relationship between variables on the population level. However, hypothesis testing cannot definitively confirm or refute the theory. Inferences from test statistics are probabilistic; therefore, they are open to a certain (known) probability of error. The study might fail to detect a relationship between variables that exists in the population (e.g., in the example above—researchers conclude that educational television viewing and school readiness skills are not associated, when in fact they are). Alternatively, a relationship that does not exist in the population can be found in the sample, leading to the erroneous conclusion that the variables are related in the population (e.g., researchers conclude that educational television viewing and school readiness skills are positively associated, when, in fact, they are not).

Despite the widespread use of null hypothesis testing, there is an ongoing controversy concerning the validity of this approach. A number of alternatives to null hypothesis testing, such as Bayesian statistics, have been advanced. Within the frequentist statistical approach, reporting confidence intervals and effect sizes to supplement or replace hypothesis testing became a norm in many research fields.