True Positive

Definition

Null hypothesis significance testing examines the probability of two outcomes (the null and alternative hypothesis). Because these two hypotheses are mutually exclusive and exhaustive, there are four potential results. Two of these results are correct decisions. Two potential results are incorrect decisions. A true positive is one of the potential correct decisions. It is finding a difference between groups (e.g., males and females) or a relationship (e.g., the association between height and weight) that truly exists and does not represent a chance fluctuation. The table below outlines the four possible conditions and states the research question in terms of group differences. The same table can be made for predictive or correlational relationships.

It is desirable to find a difference between the groups (or an association between the variables) when it truly exists. In other words, the goal of a research project is to detect the true state of affairs. If men and women are different with respect to height, then the researcher wants to be able to reproduce this difference. The concept of a true positive (and the concepts of Type I error, Type II error, and true negative) is a theoretical abstraction. The true state of affairs can never really be known (because of random error, measurement error, etc.). Thus, it is not possible to determine whether an error has been made or whether a true positive or true negative exists. The use of statistical significance testing attempts to limit the likelihood of making an error and to increase the chance that the correct decision is made.

The word positive in this case does not reference good. The positive with respect to the true positive is more akin to the presence (versus the absence) of a condition or relationship (e.g., having chicken pox). Therefore, a true positive indicates that condition, association, or difference exists.

In Relation to Power Analysis

Another concept that is related to the idea of true positive is the concept of statistical power. Statistical power analysis is the sensitivity of a research design to detect a true positive. In other words, it is the ability of the research study to find a difference or reject the null hypothesis when it is in fact false. Statistical power and alpha level (the likelihood of a false negative or a Type I error) are impacted by the design of the research study and influence each other. Statistical power is determined by the effect size, directional nature of the test, and the alpha level. The alpha level is set a priori (prior to running the research study) by the researcher.

...