Confidence Level

In statistical inference, it is common practice to report the point estimate of a population parameter along with its standard error (square root of the variance). Often, the point estimator and its standard error are combined by adding and subtracting from the point estimate a multiple of the standard error to obtain an interval estimator. Suppose θ denotes an estimator of θ and v() denotes its variance, then is an interval estimator of the parameter 9. The constant c is chosen in such a way that if the sampling process is repeated for a large number of times and for each sample an interval estimator is obtained, then approximately a pre-defined percentage of the intervals will enclose θ. This pre-defined percentage is known as the “confidence level” (or “coverage probability”) of the interval estimator. Hence, interval estimators are also commonly known as “confidence intervals.” In most cases, for a two-sided confidence interval, the value c is obtained by solving the equation

where 100 (1 − α)% is the chosen confidence level of the desired confidence interval and the probability is calculated using the sampling distribution of . Generally, the sampling distribution of is not known and is assumed to be either a normal (Gaussian) or Student's t-distribution depending upon the sample size and distributional characteristics of the population. If v() is also not known, then its estimated value, (), is used in the calculation.

If is biased or () is not calculated according to the sampling design or an incorrect sampling distribution of is assumed, then the actual confidence level of the 100(1 − α)% confidence interval will be different from the nominal confidence level 100(1 − α)%. For example, Carl-Erik Sarndal, Benqt Swensson, and Jan Wretman examined the effect of bias on confidence level. Cherie Alf and Sharon Lohr showed that the true confidence level for a 95% confidence interval for the population mean may be less than 95% depending upon the intracluster correlation coefficient (i.e. if the sample design characteristics are ignored in the variance calculations, then the resulting confidence interval will not have the correct confidence level).