Unbiased estimator

One of the important objectives of scientific studies is to estimate quantities of interest from a population of subjects. These quantities are called parameters, and the actual nature of the parameter varies from population to population and from study to study. For example, in a study designed to estimate the proportion of U.S. citizens who approve the President's economic stimulus plan, the parameter of interest is a proportion. In other cases, parameters may be the population mean, variance, median, and so on. In cases where the population values are represented by a parametric model, the parameters represent the whole population instead of a particular characteristic of it.

Parameters are estimated based on the sample values. Estimators are functions of sample observations used to estimate the parameter. It would be expected that a good estimator should result in an estimate that is close to the true value of the parameter. Because in practice the parameters are unknown, it is not possible to compare the estimate with the true value. Unbiasedness serves as a measure of closeness between an estimator and the parameter.

Suppose the population parameter θ is to be estimated based on the sample observations X1,X2,…, Xn. An estimator

=h(X1,X2,…, Xn) is an unbiased estimator of θ if the mean or expectation of over all possible samples from the population results in θ. In statistical terms,=h(X1,X2,…, Xn) is an unbiased estimator of θ if and only if

for all θ, where E(

) represents the expected value of. That is, if in repeated sampling from the population, the average value of the estimates equals the true parameter value, then the estimator is unbiased.

As an example, suppose the population mean μ is to be estimated based on a random sample X1,X2, …, Xn from the population, then the sample mean

= (X1 + X2 + … + Xn)/n is an unbiased estimator of the population mean. Statistically,

for all μ. To see this, a pathological example will be used. Suppose the population consists of only five subjects and the measurements on these five subjects are listed as follows:

resulting in population mean

[Page 1584]

Table 1 All Possible Samples and Corresponding Sample Means From the Population (3, 5, 3, 2, and 2)
Sample	Sample Mean
(3; 5)	4.0
(3; 3)	3.0
(3; 2)	2.5
(3; 2)	2.5
(5; 3)	4.0
(5; 2)	3.5
(5; 2)	3.5
(3; 2)	2.5
(3; 2)	2.5
(2; 2)	2.0

Now consider estimating μ based on without-replacement-samples, that is, samples containing distinct members of the population, of size 2 from this population. Table 1 shows possible samples and the corresponding sample means. Note that the average of the sample means is (4.0 + 3.0 + 2.5 + 2.5 + 4.0 + 3.5 + 3.5 + 2.5 + 2.5 + 2.0)/10 = 30/10 = 3 = μ.

Similarly, a sample proportion is an unbiased estimator of the population proportion. In general, the sample median is not an unbiased estimator of the population median. Sample variance, defined by the formula

is an unbiased estimator of the population variance σ2.

When the population is infinite or large, it is not possible to investigate each and every member of the population to determine the population characteristics, referred to as parameters. An unbiased estimator of a population parameter closely approximates the parameter. Without evaluating the whole population, the population parameter can be computed with accuracy based on the unbiased estimator from a sample drawn from the population. This is because in repeated sampling, the unbiased estimator results in an average value that is equal to the parameter itself.

...