Exploratory Factor Analysis

Exploratory factor analysis (EFA) is a multivariate statistical technique to model the covariance structure of the observed variables by three sets of parameters: (a) factor loadings associated with latent (i.e., unobserved) variables called factors, (b) residual variances called [Page 539]unique variances, and (c) factor correlations. EFA aims at explaining the relationship of many observed variables by a relatively small number of factors. Thus, EFA is considered one of the data reduction techniques. Historically, EFA dates back to Charles Spearman’s work in 1904, and the theory behind EFA has been developed along with the psychological theories of intelligence, such as L. L. Thurstone’s multiple factor model. Today, EFA is among the most frequently used statistical techniques by researchers in the social sciences and education.

It is well-known that EFA often gives the solution similar to principal component analysis (PCA). However, there is a fundamental difference between EFA and PCA in that factors are predictors in EFA, whereas in PCA, principal components are outcome variables created as a linear combination of observed variables. Here, an important note is that PCA is a different method from principal factor analysis (also called the principal axis method). Statistical software such as IBM® SPSS® (PASW) 18.0 (an IBM company, formerly named PASW® Statistics) supports both PCA and principal factor analysis. Another similarity exists between EFA and confirmatory factor analysis (CFA). In fact, CFA was developed as a variant of EFA. The major difference between EFA and CFA is that EFA is typically employed without prior hypotheses regarding the covariance structure, whereas CFA is employed to test the prior hypotheses on the covariance structure. Often, researchers do EFA and then do CFA using a different sample. Note that CFA is a submodel of structural equation models. It is known that two-parameter item response theory (IRT) is mathematically equivalent to the one-factor EFA with ordered categorical variables. EFA with binary and ordered categorical variables can also be treated as a generalized latent variable model (i.e., a generalized linear model with latent predictors).

Mathematically, EFA expresses each observed variable (xi) as a linear combination of factors (f1, f2, ..., fm) plus an error term, that is, $x_i={\mu }_i+{\lambda }_i{}_1{f}_1+{\lambda }_i{}_2{f}_2+\cdots +{\lambda }_{im}{f}_m+e_i,$ where m is the number of factors, μ is the population mean of $x_i,{\lambda }_{ij}\text{s}$ are called the factor loadings or factor patterns, and ei contains measurement errors and uniqueness. It is almost like a multiple regression model; however, the major difference from multiple regression is that in EFA, the factors are latent variables and not observed. The model for EFA is often given in a matrix form:

$$x=\mu +\Lambda f+e,$$

where x, µ, and e are p-dimensional vectors, f is an m-dimensional vector of factors, and Λ is a $p\times m$ matrix of factor loadings. It is usually assumed that factors (f) and errors (e) are uncorrelated, and different error terms (ei and ej for i ≠ j) are uncorrelated. From the matrix form of the model in Equation 1, we can express the population variance-covariance matrix (covariance structure) Σ as

$$\Sigma =\Lambda \Phi \Lambda \prime +\psi$$

if factors are correlated, where Φ is an m × m correlation matrix among factors (factor correlation matrix), Λ′ is the transpose of matrix Λ in which rows and columns of Λ are interchanged (so that Λ′ is a m × p matrix), and Ψ is a p × p diagonal matrix (all off-diagonal elements are zero due to uncorrelated ei) of error or unique variances. When the factors (f) are not correlated, the factor correlation matrix is equal to the identity matrix (i.e.,Φ = Im) and the covariance structure is reduced

...