Statistical Significance

The Significance Test and Hypotheses

A hypothesis is a statement about an underlying population that may be true or false. Relationships in political science are generally probabilistic rather than deterministic, and this fact has major implications in the design of attempts to prove the existence of hypothesized relationships between variables. In political analysis, researchers are often interested in establishing a causal relationship between variables. Establishing this causal relationship involves meeting three conditions: (1) establishing association between variables, (2) establishing an appropriate time order between variables, and (3) eliminating alternative explanations. Statistical significance is most related to establishing association between variables and is only tangentially related to the other two conditions.

Establishing statistical significance involves testing a hypothesized statistical relationship. Ronald A. Fisher is credited most with the development of the statistical significance concept. Fisher proposed a procedure that produces significance levels from the data about a single hypothesis with a known distribution and a specified test statistic (such as z, t, F, and χ2). Meanwhile, Jerzy Neyman and Egon Pearson proposed a procedure that tests the probability of a null or restricted hypothesis against a research (alternative) hypothesis. As political science has matured as a discipline, political scientists rely more and more on quantitative data and statistical methods to test hypothesized relationships. Over time, the fusion of Fisher's significance test procedure and Neyman-Pearson's hypothesis test procedure produced the null hypothesis statistical test, which is now the dominant procedure for determining statistical significance in political science and in social science in general.

An important aspect of significance tests is determining whether to use a one-tailed or two-tailed test of significance, and this decision is related directly to the form of the paired hypotheses. The null hypothesis (H0) is the hypothesis to be tested and is usually a statement of a parameter value, say θ, that corresponds to a parameter value of no effect, say θ ∗. In turn, the null hypothesis takes the form H0: θ = θ∗. In many political and social research applications θ∗ is zero. Meanwhile, the form of the alternative hypothesis determines whether the significance test is one-tailed or two-tailed. The two-tailed test takes the form Ha: θ ≠ θ∗. This means the alternative hypothesis includes values falling below and above the value of θ∗ listed in H0. Conversely, one-tailed alternative hypotheses have the directional forms of Ha: θ < θ∗ or Ha: θ > θ∗. The alternative hypothesis Ha: θ < θ ∗ refers to detecting whether θ is smaller than the particular number θ ∗. The alternative hypothesis Ha: θ > θ∗ refers to detecting whether θ is larger than θ ∗. In short, the researcher predicts deviation in H0 in a particular direction. It is generally safest to use a two-tailed test, but there are some situations in which the one-tailed test is more appropriate.

...