Regression Artifacts

The term regression artifacts refers to pseudoeffects from a regression type of analysis. These incorrect causal estimates are due to biases from causes other than the cause of interest. Note that such artifacts are problems only when making causal inferences, not when merely predicting some future outcome. A famous regression artifact, for example, was that the first major evaluation of Head Start concluded that the summer program actually caused children to do worse in school. If this were merely a prediction, it would have been correct that having attended Head Start would predict poorer performance in school. These children did poorly in school because of their disadvantaged status, which Head Start could not completely compensate for. This correct prediction becomes an artifact only when someone makes a causal conclusion that Head Start is responsible for their below-average academic performance later.

The most general reason for regression artifacts is that the statistical analysis reflects an incomplete picture of reality, which is called a specification error. Specification errors include the omission of relevant variables and other mismatches between statistical assumptions and reality. A relevant variable is any variable that is associated with the cause of interest but that also causally influences the outcome variable directly. In the Head Start case, being from a disadvantaged background was associated with attendance at Head Start, but it also caused poor academic performance. Mismatches between reality and linear statistical assumptions include a curvilinear relationship and an interaction effect, in which the effect of one causal variable depends on another variable. Another common mismatch is that unbiased causal evidence usually requires that other relevant variables (potential confounds) be measured without error. Socioeconomic status (SES) was controlled for statistically in the first major evaluation of Head Start, but measurement error in SES reduced its ability to fully correct for disadvantage. Regression estimates of causal influences are unbiased only if statistical analyses include perfectly valid and reliable measures of all relevant variables and correctly reflect any complexities among their [Page 1229]interrelationships. To the extent a regression type of analysis falls short of that ideal, there is potential for regression artifacts, that is, pseudoeffects masquerading as causal evidence.