Residuals

Use of Residuals

A key step in modeling is the examination of the residuals for the purpose of checking if the model is a good fit in light of the data. In particular, residuals are used to check the validity of the assumptions of the model and to identify observations that are considered outliers (unusually large or small observations) and influential in estimating the model. These may be achieved through graphical techniques or performing formal statistical tests applied to residuals. One may work with raw residuals or standardized residuals. Standardized residuals are obtained [Page 839]by dividing the raw residuals by their estimated standard error.

An Illustration

In ordinary regression modeling, a response variable (denoted by Y) is modeled using a linear relationship with a single variable, known as the predictor variable (denoted by X). Mathematically, this model is represented by the following equation:

where β0 and β1 are the intercept and slope of the line, respectively, and ∊ is the random error term. As an example, suppose it is of interest to model job satisfaction (Y) based on the number of years on the job (X). The data are given in Table 1. A scatterplot of the data is displayed in Figure 1, represented by dots. The estimated model is given by

which is represented by the solid line in Figure 1. Residual is graphically represented as the difference between a given observation and the fitted line, as illustrated by the vertical line in Figure 1. For this particular observation, Y = 5.2 and X = 7, so that the estimated response is given by

Figure 1 Scatterplot of Job Satisfaction Data With Best Fit Line

Consequently, the residual for this observation is

A plot of the residuals versus the predictor variable checks the assumption that the relationship between X and Y is linear. This residual plot applied to the current example is displayed in Figure 2. If the assumption of linearity is appropriate, there should not be any obvious systematic pattern in this residual plot, which is true in this case. Hence, there is no evidence to reject the linearity assumption of the model.

Other Types of Residuals

In more complex statistical models, residuals are defined differently, and, in some cases, there may be [Page 840]more than one type of residual. For instance, in logistic models, there are deviance residuals in addition to the commonly defined residuals. In survival models, where the response variable of interest is time until an event occurs, there are also deviance, Cox-Snell, martingale, and score residuals, to name a few. The definitions of these residuals are no longer as simple as observed minus expected. However, no matter how these residuals are defined, their main purpose is still the same, that is, checking the appropriateness of the model.

...