Effect Coding

Multiple Regression Framework

In linear multiple regression analysis, the goal is to predict, knowing the measurements collected on N subjects, a dependent variable Y from a set of J independent variables denoted

$$\{X_1,\dots ,X_j,\dots ,X_J\}.$$

We denote by X the $N\times (J+1)$ augmented matrix collecting the data for the independent variables (this matrix is called augmented because the first column is composed only of 1s), and by y the $N\times 1$ vector of observations for the dependent variable. These two matrices have the following structure.

$$X=\left[\begin{array}{l}1x_{1,1}\cdots x_{1,k}\cdots x_{1,K}\\ ⋮⋮\ddots ⋮\ddots ⋮\\ 1x_{n,1}\cdots x_{n,k}\cdots x_{n,k}\\ ⋮⋮\ddots ⋮\ddots ⋮\\ 1x_{N,1}\cdots x_{N,k}\cdots x_{N,k}\end{array}\right]$$

$$\text{and}y=\left[\begin{array}{l}{y}_1\\ ⋮\\ y_n\\ ⋮\\ y_N\end{array}\right]$$

The predicted values of the dependent variable $\widehat{Y}$ are collected in a vector denoted $\widehat{y}$ and are obtained as

$$\widehat{y}=Xb\text{with}b={\left(X^{\text{T}}X\right)}^{-1}{X}^{\text{T}}y.$$

[Page 478]where T denotes the transpose of a matrix and the vector b has $J$ components. Its first component is traditionally denoted b0, it is called the intercept of the regression, and it represents the regression component associated with the first column of the matrix X. The additional $J$ components are called slopes, and each of them provides the amount of change in Y consecutive to an increase in one unit of its corresponding column.

The regression sum of squares is obtained as

$$S{S}_{\text{regression}}=b^{\text{T}}{X}^{\text{T}}y-\frac{1}{N}{(1^{T}y)}^2$$

(with 1T being a row vector of 1s conformable with y). The total sum of squares is obtained as

$$S{S}_{\text{total}}=y^{T}y-\frac{1}{N}{(1^{T}y)}^2.$$

The residual (or error) sum of squares is obtained as

$$S{S}_{\text{error}}=y^{T}y-b^T{X}^{T}y.$$

The quality of the prediction is evaluated by computing the multiple coefficient of correlation denoted $R_{Y.1,\dots ,J}^2$. This coefficient is equal to the squared coefficient of correlation between the dependent variable (Y) and the predicted dependent variable ($\widehat{Y}$).

An alternative way of computing the multiple coefficient of correlation is to divide the regression sum of squares by the total sum of squares. This shows that $R_{Y.1,\dots ,J}^2$ can also be interpreted as the proportion of variance of the dependent variable explained by the independent variables. With this interpretation, the multiple coefficient of correlation is computed as

$$R_{Y.1,\dots ,J}^2=\frac{S{S}_{\text{regression}}}{S{S}_{\text{regression}}+S{S}_{\text{error}}}=\frac{S{S}_{\text{regression}}}{S{S}_{\text{total}}}.$$

Significance Test

In order to assess the significance of a given $R_{Y.1,\dots ,J}^2$, we can compute an F ratio as

$$F=\frac{R_{Y.1,\dots ,J}^2}{1-R_{Y.1,\dots ,J}^2}\times \frac{N-J-1}{J}.$$

Under the usual assumptions of normality of the error and of independence of the error and the scores, this F ratio is distributed under the null hypothesis as a Fisher distribution with ${\nu }_1=J$ and ${\nu }_2=N-J-1$ degrees of freedom.

Analysis of Variance Framework

For an ANOVA, the goal is to compare the means of several groups and to assess whether these means are statistically different. For the sake of simplicity, we assume that each experimental group comprises the same number of observations denoted I (i.e., we are analyzing a “balanced design”). So, if we have K experimental groups with a total of I observations per group, we have a total of $K\times I=N$ observations denoted $Y_{i,k}$. The first step is to compute the K experimental means denoted ${\text{M}}_{+\text{},k}$ and the grand mean denoted $M_{+,+}$. The ANOVA evaluates the difference between the mean by comparing the dispersion of the experimental means to the grand mean (i.e., the dispersion between means) with the dispersion of the experimental scores to the means (i.e., the dispersion within the groups). Specifically, the dispersion between the means is evaluated by computing the sum of squares between means, denoted SSbetween, and computed

...