Factor Analysis

Factor Models

The main idea behind a factor model is that a large number N of variables can be explained by a small number q of factors. In a factor model, a vector of N observations at time t—for example, YN(t)—is decomposed into a common component XN(t) and an idiosyncratic component ZN(t):

$${\mathrm{Y}}_N(t)={\mathrm{X}}_N(t)+Z_N(t).$$

The common components are a linear combination of the latent common factors f(t), with weights given by the so-called factor loadings LN:

$${\mathrm{X}}_N(t)=L_N\cdot f(t)=l_1\cdot {\mathrm{f}}_1(t)+l_2\cdot f_2(t)+...+l_q\cdot {\mathrm{f}}_q(t).$$

In this notation, the matrix LN is N × q, whereas the vector of factors is q × 1. Both X and Z are unobservable. The covariance matrix of the observations can be decomposed accordingly into

$$C^{\mathrm{Y}}=C^{\mathrm{X}}+C^Z=L_N\cdot C^f\cdot L_N^T+C^Z.$$

In public debates, for example, the covariance matrix CX captures the correlations between the semantic concepts and the common factors, whereas the covariance matrix CZ explains the covariation between specific semantic meanings in the debates that cannot be explained by the factors.

Factor models are appealing for two reasons: they are a dimension-reduction tool and, at the [Page 503]same time, a meaningful representation of the principal components of the covariance matrix of the observations. While the first feature (dimension reduction) is common to all factor models, the second property is achieved only when the number of series, N, is large. The traditional or strict approach assumes that N is finite and that the covariance matrix of the errors CZ is diagonal (for identification, some other conditions need to be imposed on the loadings). Then, the parameters in the models can be estimated by maximum likelihood. The most recent literature of approximate factor models differs from the traditional one in that the covariance matrix of the idiosyncratic components CZ is allowed to be nondiagonal and the cross-section size N is large. Allowing for a nondiagonal CZ is important: It means that one allows for nonzero correlation among specific concepts in these debate analyses.

With sufficiently large N, the parameters in the model can be estimated by principal components. While FA is based on a statistical model, principal components analysis (PCA) is a tool for dimension reduction. More precisely, given a T × N data set Y= [Y1, . . . ,YN] with T observations and N variables, the principal components are the projections P = [P1, . . . ,Pq] of the T observations onto a subspace W = [w1, . . . ,wq] of the original

...