Likelihood Ratio

The likelihood ratio is used to compare different data models, perform hypothesis tests, and construct confi-dence intervals. The likelihood of a model is a measure of how well the model fits the observed data. Model likelihood is most often used when fitting a parametric model (e.g., a normal distribution) to data to find the parameters that best describe the data. The general formula for the likelihood function is

where

θ is the model parameterization,

D is the observed data,

L(θ|D) is the likelihood of the model parameterization given the observed data D, and

P(D|θ) is the probability density function of the data given the model parameterization θ.

The basis for this formulation of the likelihood function is best understood by using Bayes's theorem to calculate the probability of the model given the observed data:

[Page 602]If we assume that all models are equally likely and note that the probability of the observed data is fixed, this formula is proportional to the likelihood formula given above. That is, for any two model parameterizations if the likelihood of one parameterization is greater than that of the other, then the probability of that parameterization is also greater by the above formula.

When comparing two models, we use their likelihood ratio. The formula for the likelihood ratio is

In hypothesis testing, the top model parameterization (θ0) represents the null hypothesis and the bottom parameterization (θ1) represents the alternative hypothesis. We will reject the null hypothesis in favor of the alternative hypothesis if λ(D) < c, where c is some preselected critical value. Confidence intervals, sometimes referred to as supported intervals when using this approach, can be calculated in a similar manner. In this approach, the value of θ that maximizes the likelihood (the maximum likelihood estimate) is used to determine the denominator (θ1) and alternative parameterizations are used in the numerator (θ0). The supported region consists of those values of (θ0) where λ(D) < c, where c is some critical value. Some statisticians and epidemiologists argue that these approaches are superior to traditional approaches because they incorporate the probability of the observed data given the alternative models into the calculation, not only the probability under the null hypothesis.

Another common use of the likelihood ratio occurs when performing a likelihood ratio test. A likelihood ratio test is used when comparing a model with fewer parameters with a more complex one (i.e., one with more parameters). In general, the more complex model will fit the observed data better. To determine if this increase in goodness of fit is enough to justify the increase in model complexity, we first calculate the likelihood ratio test statistic:

where

L0 is the likelihood of the simpler model using the maximum likelihood parameter estimate and

L1 is the likelihood of the more complex model using the maximum likelihood parameter estimate.

This statistic is then used as the test statistic in a chi-squared test with n degrees of freedom, where n is the number of additional parameters in the more complex model. If the chi-squared test is greater than the 1 − a region of a chi-squared distribution, the more complex model is accepted as valid; otherwise the simpler model is used.

...