Prior Distribution

The term prior distribution refers to an empirical or expected probability density function used in Bayesian statistical analysis, which represents an analyst's belief about the distribution of an unobserved parameter, θ, in the population. In Bayesian data analysis, inferences are made on the basis of estimated probability distributions for unobserved or “latent” variables based on observed data. Bayes' theorem is generally phrased in terms of distributions of observed and unobserved variables:

By using this formulation of Bayes' theorem, statisticians are able to make inferences about some parameter of interest, θ, given the observed data, x. This density function, f(θ |x), is referred to as the posterior distribution of θ, and it represents a probability density function for the latent variable, θ, based on the observed data, x. In this formulation, f(x|∘) is the “likelihood function,” which represents a statistical model that has been fit to the distribution of observed data, x, given the underlying parameter, θ. f(x) is the empirical distribution of the observed data, x, and f(θ) is the prior distribution for the unobserved parameter, θ.

Whereas f(x|∘) and f(x) are quantities estimated using observed data, the prior distribution, f(θ), is typically unobserved and therefore must be selected by the analyst based on a belief about how the parameter θ is distributed in the population. Selection of an appropriate value for a prior distribution is therefore critical to the accuracy of results from a Bayesian statistical analysis.

Bayesian procedures incorporate information from the prior distribution in order to make inferences. Generally speaking, the prior distribution for θ is an unobserved probability density function that must be estimated somehow by the analyst. Often, this can be done by making some reasonable assumptions about the distribution of θ in the population, or by collecting data and empirically estimating this function. Because the statistician can never be sure that the particular choice of a prior distribution is accurate, one criticism of Bayesian statistics is that one cannot be sure how well the posterior distribution represents the distribution of θ given x. The impact of the prior distribution on the final results of the analysis (i.e., the posterior [Page 787]distribution) will vary depending on the statistician's choice for the distribution. Prior distributions that have significant influence on the posterior distribution are referred to as relatively “informative,” whereas prior distributions with relatively little influence are called “non-informative” priors. For this reason, analysts tend to choose relatively non-informative prior distributions in situations where they are less confident about the distribution of θ in the population.