Likelihood Ratio Test

The likelihood ratio test is a test of a statistical hypothesis that uses the likelihood ratio as a test statistic. It is available in a broad class of hypothesis-testing problems where the underlying statistical model involves a parametric family of distributions. Many well-known statistical tests are, in fact, likelihood ratio tests. In some cases, it has desirable optimality properties. For large samples, a convenient approximation is available for computing approximate p values, subject to some regularity conditions. In some less regular cases, approximate p values can be obtained via computer simulation.

Suppose that data y = (y1, y2,…, yn) are modeled as a realization of a random vector whose distribution depends on unknown parameters θ= (θ1, θ2,…, θk). If the distribution is discrete, let f(x;θ) denote the probability mass function; if continuous, let f(x;θ) denote the probability density function as a function of a real vector x and the unknown parameters.

The likelihood function is

that is, f(x;θ) is evaluated at the data and viewed then (primarily) as a function of the unknown parameters. When the distribution is discrete, this is precisely the probability of getting the observed data as a function of the parameters; in the continuous case, the interpretation is similar, so in a certain sense, larger values of the likelihood function indicate better agreement between parameters and data.

Let Θ denote the set of all possible parameter vectors under the model, and let Θ0 denote the set of those parameter vectors in Θ permitted under the hypothesis (this hypothesis is sometimes called the null hypothesis). Also, let Θ1 denote the alternative, that is, the set of those parameter vectors in Θ not permitted under the hypothesis. The likelihood ratio is defined in several different but essentially equivalent ways. One is

Because larger likelihood means better agreement between parameters and data, the denominator measures the best agreement possible under the hypothesis. If the best agreement over all Θ is attained under [Page 534]the hypothesis, Λ= 1. Otherwise, it is greater than 1, with better performance of the alternative over the hypothesis giving a bigger likelihood ratio. Thus, larger values of L provide more evidence against the hypothesis.

An alternate definition is

where the numerator measures only the best agreement under the alternative. Thus, Λ= max Λ1, 1), so when there is any evidence against the hypothesis, they coincide. The likelihood ratio is also sometimes defined as 1/Λ or 1/Λ1, in which case smaller values of the statistic provide more evidence against the hypothesis.

If two test statistics are increasing functions of one another, then because they always give the same p value, the tests based on them are equivalent. In many common problems, the likelihood ratio turns out to be an increasing function of a more familiar test statistic. Some examples are one-sample, two-sample, and regression t tests; analysis of variance F tests; and certain tests based on the mean of the data for binomial and Poisson models.