Statistical Significance

Chance Effects on Empirical Data

The substantive population of a research effort consists of all individuals to whom the research conclusions apply. It may consist of all hyperactive boys in a study of hyperactivity. Any of its characteristics (e.g., the mean attention span of all hyperactive boys, u) is a parameter. Suppose that a psychologist collects data from a randomly selected sample of 100 hyperactive boys. A characteristic of the sample is a statistic (e.g., the sample's mean attention span, X¯).

The sample mean is unlikely to be identical to the population mean because of chance influences. For example, chance factors during data collection (a) determine who are included in the sample and (b) render some boys more attentive than usual while other boys are being distracted more than usual. Consequently, different samples of 100 hyperactive boys selected and tested in exactly the same way produce different mean attention spans.

Suppose that the psychologist selects randomly 100 hyperactive boys and assigns randomly 50 to Group 1 and 50 to Group 2. The random selection and random assignment procedures warrant the suggestion that Groups 1 and 2 are the respective samples of two substantive populations with the same mean (i.e., uspanI= uspanII). Be that as it may, the means of Groups 1 and 2 (X¯spanI and X¯spanII, respectively) are not expected to be literally the same by virtue of happenstance.

Statistical Populations and Research Manipulation

To test whether Drug D affects the attention span of hyperactive boys, the psychologist gives Group 1 Drug D and Group 2 a placebo. The two substantive populations now become two methodologically defined statistical populations, namely, (a) hyperactive boys given Drug D and (b) hyperactive boys given a placebo. Their means are uDrugD and uPlacebo, respectively. The means of Groups 1 and 2 are X¯DrugD and X¯Placebo, respectively.

If Drug D is not efficacious, uDrugD = uPlacebo because the distinction between the two methodologically defined populations becomes mute. This equality implies that X¯DrugD = X¯Placebo. However, because of the aforementioned chance effects, (X¯DrugD X¯Placebo) is not expected to be exactly zero.

An efficacious Drug D would change the attention span, thereby leading to uDrugD ≠ uPlacebo. It follows that (X¯DrugD X¯Placebo) is not zero. Thus arises the following conundrum: A nonzero (X¯DrugD X¯Placebo) is expected [Page 964]regardless of the efficacy of Drug D. Psychologists use statistical significance to handle the dilemma.

Inferential Statistics

Psychologists use NHSTP to learn something about population parameters (e.g., uDrugD − uPlacebo) on the basis of the statistical significance of their corresponding sample statistics (X¯DrugD X¯Placebo). This is achieved by using the standardized form of an appropriate theoretical distribution to make a binary decision regarding chance effects on the data in probabilistic terms.

...