p-Value

The probability value (abbreviated “p-value”), which can range from 0.0 to 1.0, refers to a numeric quantity computed for sample statistics within the context of hypothesis testing. More specifically, the p-value is the probability of observing a test statistic that is at least as extreme as the quantity computed using the sample data. The “probability” is computed using a reference distribution that is generally derived using the null hypothesis. The phrase extreme is usually interpreted with respect to the direction of the alternative hypothesis—if two tailed, then the p-value represents the probability of observing a test statistic that is at least as large in absolute value as the number computed from sample data. Put another way, the p-value can be interpreted as the likelihood that the statistical result was obtained by chance alone.

P-values often are reported in the literature for analysis of variance (ANOVA), regression and correlation coefficients, among other statistical techniques. The smaller the p-value, the more the evidence provided against the null hypothesis. Generally, if the p-value is less than the level of significance for the test (i.e. α), then the null hypothesis is rejected in favor of the alternative and the result is said to be “statistically significant.”

In the context of survey research, interest may be given in comparing the results of a battery of questionnaire items across demographic or other groups. For example, a sequence of p-tests may be conducted to compare differences in the mean ages, education level, number of children in the home, annual income, and so on, for people who are surveyed from rural areas versus those from urban areas. Each test will generate a test statistic and a corresponding two-sided p-value. Comparing each of these separate t-values to .05 or .10, or some other fixed level of alpha, can give some indication for the significance of the hypotheses tests that the mean levels of age, education, and so on, are equal for citizens living in urban and rural areas versus the alternative hypothesis that these means are different. However, using the separate p-values alone inflates the Type I error rate of the entire sequence of tests. To avoid this inflation, adjusted p-values are often used to provide an overall error rate that is equal to the nominal alpha level specified by the researcher. The most straightforward of these adjustments is the Bonferroni adjustment, which compares either the p-value to alpha divided by the number of tests or compares the p-value multiplied by the number of tests to the nominal alpha level. If the adjusted p-values are less than alpha, then the specifie null hypothesis is rejected.

The computation of p-value can be illustrated using data obtained from a single survey item that asks a random sample of 20 households the following question: How many children live full time within your household? The null hypothesis is that an average of 2 children live full time per household; the alternative hypothesis posits the average number exceeds 2. The reference distribution for the sample mean values for samples of size 20 is provided in Table 1.

...