Weights

Weights in Stratified Sampling

Stratified sampling is a method of sampling within subpopulations. Each subpopulation is called a stratum. When strata vary considerably in size or distribution, stratified sampling often improves the representativeness of the sample. By assigning a proper weight to each stratum, a more efficient estimator for the mean of the whole population will be obtained.

Suppose the entire population is divided into h strata. Let μj be the population mean for the jth stratum and pj be the proportion of the jth stratum in the whole population, j = 1,…, h. Then the whole population mean is

When a sample of size n is obtained, let nj be the sample size of the jth stratum; then n = n1+n2 + … + nn. Let

j and wj be the sample mean and the attached weight of the jth stratum, respectively. Then the weighted sample meanw and its expected value E(w) are

and

When all the pj's are known, the weights are often set to be wj = pj. Denote

p as the weighted sample mean in Equation 1 with wj = pj, then it follows from Equation 2 thatp is unbiased.

Notice that the sampling proportion nj/n does not necessarily equal the population proportion pj in the above estimation. Stratified sampling allows a small stratum to have enough representation in the estimation.

In practice, either proportionate or disproportionate sampling is used to determine nj. In proportionate sampling, nj is set proportional to pj. In disproportionate sampling, nj is chosen to minimize the variance of

p. Notice that, unless all the μj's are equal, proportionate sampling yields a

p having a smaller variance than the arithmetic mean of the simple random sampling of size n. Disproportionate sampling almost always leads to a

p that has a smaller variance than the one corresponding to proportionate sampling.

Weights in Weighted Least-Squares Regression

Let yi be the response and xi = (1,xi1,…, xip) be the set of predictors, = 1, 2,…, n. The multiple regression model can be expressed as

where β = (β0, β1,…, βp)′ is the vector of regression coefficients, with ′ being the notation for transpose, and ei is the residual defined as the difference between the value of the dependent variable and the predicted value. Least squares (LS) that minimizes sum of squares of the residuals

is the most widely used method in estimating β. Its justification needs three assumptions: All the ei's have expectations of zero, have a common variance σ2, and are uncorrelated. When all the three are met, the LS estimator

is optimal among all estimators that are linear combinations of yi's in the sense of having a minimum variance. When any of the assumptions is violated,

does not enjoy such an optimal property. With the intercept β0 in Equation 3, E(ei) = 0 is automatically satisfied. When e's are correlated or have heterogeneous variances, a better estimator than that in Equation 4 can be obtained by using proper weights. Consider the case when ei’ are independent and σi2 = Var(ei) are unequal [Page 1619]but known. Then Equation 3 can be transformed

...