Regression

Regression analysis is, by far, the most commonly used statistical technique in political science. While regression analysis can be defined in many different ways, it is a tool for describing the relationships among variables. Regression is generally used for two complementary purposes. First, it measures the effects of one or more independent variables on a single dependent variable. Second, it can be used to predict the values of a dependent variable that can be expected to occur at specified levels of the independent variables. In the former capacity, regression is a useful tool for theory testing. In the latter capacity, regression is a forecasting tool that is useful for decision making. These two uses are closely connected. Theories are evaluated by their ability to predict as yet unobserved phenomena, and forecasting is carried out most effectively when predictions are based on a substantive theory. In the following, the major features and applications of regression analysis are discussed.

If there is one independent variable, the analysis is commonly called a simple or bivariate regression. If there is more than one independent variable, it is called multiple regression analysis. In traditional nomenclature, the dependent variable is generically represented as Y and the independent variable is designated as X. If there are multiple independent variables, the Xs are given subscripts, say X1, X2, …, Xj, …, Xk for k variables (note that the specific order of the Xjs generally does not matter). For the moment, we will assume that all variables are relatively continuous and measured at the interval level.

The immediate objective of a regression analysis is to show how the conditional distribution of Y varies across the values of X (or across the values of the Xjs in a multiple regression). Attention is usually focused on the conditional mean of Y rather than the entire conditional distribution. If the conditional Y distributions (and, specifically, the conditional Y means) do, in fact, differ systematically across the X values, then Y is said to be related to X. If the conditional Y distributions do not vary across the X values, then X and Y are unrelated to, or independent of, each other.