Nonlinear Models

In political science, as in many other disciplines, linear regression is the workhorse tool for statistical analysis. It is easy to interpret, and under many conditions the estimates it provides are unbiased and efficient. Unfortunately, many of the theories in the social sciences imply a nonlinear relationship between variables. In those cases, it is inappropriate to use a classic linear regression model because, at a minimum, one of the assumptions of the model would be violated.

This entry discusses two types of linear models: first, those that through a relatively simple process can be transformed in a way that allows running of a classical linear regression model; second, those that are essentially nonlinear, for which a transformation is not possible. In that situation, nonlinear least squares estimation is necessary.

The linear regression model specifies a linear relationship between a response—or dependent—variable and an explanatory—or independent—variable. It is assumed that a vector of res ponse variables, y, can be approximated by a linear function of the vector of explanatory variables Xi:

which can be expressed more generally as

While in the linear model F is a linear function and the error is additive, in nonlinear models F can take other functional forms such as exponential, logistic, or other, more complicated forms, though the setup still assumes that the error is additive.

It is perhaps easier to understand the difference between linear and nonlinear models if we think of a regression model, in scalar form, with only one explanatory variable and no disturbance:

The marginal effect of the explanatory variable, x, on the response variable, y, or, in other words, the effect of a one-unit increase of x in y, can be estimated by taking the partial derivative with respect to x:

In the classical linear model, when x increases by one unit, the effect on y is always a constant, β, regardless of the current level of x. In contrast, in a nonlinear model, the marginal impact of the explanatory variable, x, on the response variable, y, is dependent on the level of x. In other words, β is not necessarily a constant, but instead dy/dx is a function of x.

A more complicated model has y as a nonadditive function of the independent variables and the disturbances, that is,

While in general this can cause problems, there are some cases where this can be transformed away. For example, if we assume that the response function that describes y is,

we can linearize the model by first taking the logarithms obtaining a linear response function and rewrite the equation such that

The response variable is now ln (y/(1 – y), a quantity known as the logit, and the response [Page 1708]function is linear. Note here that y is continuous; if it is binary, the standard logit model is not the same. The procedure is similar for other transformations that fall in the category of generalized linear models.

However, there are some models for which slightly more complicated procedures are necessary to make the linear model appropriate. One of the most common is the Box-Cox transformation, where transformations are indexed by Λ, an unknown parameter, and all the parameters in the model are estimated by standard methods of inference. The transformation

...