Null Hypothesis Significance Testing

Full-Suite Analysis

Suppose extensive testing has revealed that average self-esteem scores are μ= 68 and 72 for women and men, respectively, and that the standard deviation within each gender is σ= 20. A sample of 200 scores with a mean of 71 is drawn from one of the two populations. The null hypothesis H0 is that women were sampled, and the alternative hypothesis H1 is that men were sampled. Analysis begins with the calculation of the probability of obtaining a mean of 71 or higher if H0 is true. The z score for the sample mean is

and the probability of a score at least this extreme is .017.

Evaluation of the data under the alternative hypothesis H1 yields z = .71, p = .24. That is, the data are not improbable under the assumption that men were sampled. The likelihood ratio (LR) of the two p values, p(D|H1)/p(D|H0), is 14.12, meaning that it is more than 14 times more likely that a sample of men rather than women would yield data of the kind found in the empirical sample. But how likely is it that the sample consisted of men? It is necessary to be explicit about the prior probability of sampling men. A simple intuition is that women and men were equally likely to be sampled, that is, p(H0) = p(H1) = .5. The summed products of these prior probabilities and their respective p values is the overall probability of the observed data. Here, p(D) = p(H0)p(D|H0)+p(H1)p(D|H1) = .13. This probability is critical for the calculation of the probability of the null hypothesis given the observed data. Bayes' theorem gives p(H0|D) as p(H0)p(D|H0)/p(D) = .07. Because the prior probabilities of the two hypotheses are the same, the ratio of the two posterior probabilities is the same as the LR. It can now be said that the sample is more than 14 times more likely to comprise men than women. The assumption of equal priors was just that, an assumption. Suppose the researcher knew that self-esteem scores were collected at four different sites, only one of which comprised men. Now p(H0|D) = .18, meaning that it is only 4.7 times more likely for the sample data to come from men than from women. Although the prior probability that men were sampled was low, the evidence is still strong enough to reject the null hypothesis that women were sampled and to accept the alternative.

...