Normalizing Data

The Norm of a Vector

In linear algebra, the norm of a vector measures its length, which is equal to the Euclidean distance of the endpoint of this vector to the origin of the vector space. This quantity is computed (from the Pythagorean theorem) as the square root of the sum of the squared elements of the vector. For example, consider the following data vector denoted $y$:

$$y=\left[\begin{array}{l}35\hfill \\ 36\hfill \\ 46\hfill \\ 68\hfill \\ 70\hfill \end{array}\right].$$

The norm of vector y is denoted $\left|\right|y\left|\right|$ and is computed as

$$\begin{array}{l}\left|\right|\text{}y\text{}\left|\right|=\sqrt{{35}^2+{36}^2+{46}^2+{68}^2+{70}^2}\\ =\sqrt{14,161}=119.\end{array}$$

Normalizing With the Norm

To normalize y, we divide each element by $\left|\right|y\left|\right|=119.$ The normalized vector, denoted $\text{}\text{}\text{}y$, is equal to

$$\tilde{y}=\left[\begin{array}{l}\frac{35}{119}\hfill \\ \frac{36}{119}\hfill \\ \frac{46}{119}\hfill \\ \frac{68}{119}\hfill \\ \frac{70}{119}\hfill \end{array}\right]=\left[\begin{array}{l}0.2941\hfill \\ 0.3025\hfill \\ 0.3866\hfill \\ 0.5714\hfill \\ 0.5882\hfill \end{array}\right].$$

The norm of vector $\tilde{y}$ is now equal to one:

$$\begin{array}{l}\left|\right|\tilde{y}\text{}\left|\right|=\\ \sqrt{{0.2941}^2\text{}+\text{}{0.3025}^2\text{}+\text{}{0.3866}^2\text{}+\text{}{0.5714}^2\text{}+\text{}{0.5882}^2}\\ =\sqrt{1}=1.\end{array}$$

The Standard Deviation of a Set of Scores

Recall that the standard deviation of a set of scores expresses the dispersion of the scores around their mean. A set of N scores, each denoted Yn, whose mean is equal to M, has a standard deviation denoted $\widehat{S}$, which is computed as

$$\widehat{S}=\frac{{\displaystyle \sum {(Y_N-M)}^2}}{N-1}.$$

For example, the scores from vector y (see Equation 4) have a mean of 51 and a standard deviation of

$$\begin{array}{l}\widehat{S}=\sqrt{\frac{{(35-51)}^2+{(36-51)}^2+{(46-51)}^2+{(68-51)}^2+{(70-51)}^2}{5-1}}\\ =\frac{1}{2}\sqrt{{(-16)}^2+-{15}^2+{(-5)}^2+{17}^2+{19}^2}\\ =17.\end{array}$$

z Scores: Normalizing With the Standard Deviation

To normalize a set of scores using the standard deviation, we divide each score by the standard deviation of this set of scores. In this context, we almost always subtract the mean of the scores from each score prior to dividing by the standard deviation. This normalization is known as z scores. Formally, a set of N scores each denoted Yn and whose mean is equal to M and whose standard deviation is equal to $\widehat{S}$ is transformed in z scores as

$$z_n=\frac{Y_n-M}{\widehat{S}}.$$

With elementary algebraic manipulations, it can be shown that a set of z scores has a mean equal of zero and a standard deviation of one. Therefore, z scores constitute a unit-free measure that can be used to compare observations measured with different units.

Example

For example, the scores from vector y (see Equation 1) have a mean of 51 and a standard deviation of 17. These scores can be transformed into the vector z of z scores as

$$z=\left[\begin{array}{l}\frac{35-51}{17}\hfill \\ \frac{36-51}{17}\hfill \\ \frac{46-51}{17}\hfill \\ \frac{68-51}{17}\hfill \\ \frac{70-51}{17}\hfill \end{array}\right]=\left[\begin{array}{l}-\frac{16}{17}\hfill \\ -\frac{15}{17}\hfill \\ -\frac{5}{17}\hfill \\ \frac{17}{17}\hfill \\ \frac{19}{17}\hfill \end{array}\right]=\left[\begin{array}{l}-0.9412\hfill \\ -0.8824\hfill \\ -0.2941\hfill \\ 1.0000\hfill \\ 1.1176\hfill \end{array}\right].$$

[Page 1096]The mean of vector z is now equal to zero, and its standard deviation is equal to one.

...