Multicollinearity

Consequences

Imagine an investigation into the causes of wellbeing of unemployed workers (measured as a scale from 0 to 100). A great number of explanatory variables will be taken into account, such as annual income (the lower the annual income, the lower the well-being), duration of unemployment (the higher the number of weeks of unemployment, the lower the well-being), and others. Now, some of these independent variables will be so highly correlated that conclusions from a multiple regression analysis will be affected. High multicollinearity is undesirable in a multicausal model because standard errors become much larger and CONFIDENCE INTERVALS much wider, thus affecting SIGNIFICANCE TESTING and STATISTICAL INFERENCE, even though the ORDINARY LEAST SQUARES (OLS) estimator still is unbiased. Like variances and covariances, multicollinearity is a sample property. This means analyses may be unreliable and sensitive toafew outliers or extreme cases.

The problem of correlated independent variables in a causal model is an old sore in social scientific research. In multiple regression analysis, it is referred to as multicollinearity (the predictors in a vector model almost overlap in a straight line), whereas in ANALYSIS OF VARIANCE and COVARIANCE, one tends to use the term nonorthogonality (the factors of the factor-response model are not perpendicular to each other). Multicollinearity and nonorthogonality are, in fact, synonyms.

Let us emphasize that strange things may happen in an analysis with high multicollinearity. Kendall and Stuart (1948) provided an extreme example of one dependent variable Y and two independent variables X1 and X2, in which the correlations Y −X1 and Y −X2 are weak (one of them even zero) and the correlation X1 − X2 is very strong (0.90, high multicollinearity), whereas the multiple correlation coefficient Ry·12 is approximately 1. This classic example shows a classic consequence of multicollinearity: insignificant results of testing individual parameters but high R-SQUARED values. The standardized coefficients (a direct function of sample variances), which normally cannot be greater than 1 because their squared values express a proportion of explained variance, exceed 2; consequently, both variables appear to explain each more than 400%, with only 100% to be explained!

...