Multitrait–Multimethod Matrix

In 1959, Donald T. Campbell and Donald W. Fiske published an article in the Psychological Bulletin that, approximately 30 years later, would become the most cited article in the history of the social sciences. By 1992, it had been cited more than 2,000 times by other authors, and a 2005 search of the Social Sciences Citation Index showed more than 4,000 citations. The subject of this article was a statistical tool known as the multitrait–multimethod (MTMM) matrix. An MTMM matrix is a matrix of correlation coefficients computed between each pair of a set of measures (the correlation coefficients indicate how strongly each pair of measures is related).

The correlation matrix is intended to evaluate psychological measures—it is used to help determine how well scores on the measures actually reflect the intended traits. For example, personality traits such as extroversion and conscientiousness are often measured by self-reports, but they could also be measured by reports from others (e.g., friends or coworkers). If all psychological measurements were perfectly accurate, we would not need to consider different methods because all would be identical. But measurements are never perfect; they can be influenced by a variety of factors in addition to the intended traits (e.g., a person's self-assessments of personality might partially reflect that person's idealized view of him- or herself instead of his or her actual personality). The MTMM matrix is designed to evaluate the extent to which measures are influenced by the intended traits versus other systematic factors, commonly referred to as method effects.

Industrial and organizational psychologists have made extensive use of the MTMM matrix. They have conducted large-scale reviews of MTMM studies of job affect and perceptions (using different standard surveys as methods), job performance ratings (using performance dimensions as traits and rating sources such as supervisors and peers as methods), and assessment centers (a set of exercises used to assess potential or current workers; the assessment dimensions serve as traits and the exercises serve as methods). Individual studies have focused on other topics, such as measuring personality. In many cases, the studies indicate substantial method variance—for example, job performance ratings are fairly heavily influenced by the perspective of the particular individual providing the ratings.

Computing the MTMM matrix begins with a study in which multiple traits are measured by multiple methods. This might mean that a sample of people are asked to complete a survey rating their own personality traits, and their personalities are also rated on the same survey by close friends and then again by coworkers. If, for example, five personality traits are measured by these three methods, there would be a total of 15 measures (five traits × three methods). The MTMM matrix can then be computed.

In their original paper, Campbell and Fiske described two main components of validity that, when taken together, provide information on the overall validity of the measures. One component is convergent validity. This means that two measures of the same trait, provided by different methods, should converge on the same conclusion. If ratings of personality are valid, then reports of extroversion by friends and coworkers should tend to agree about how extroverted the person is. A second criterion is discriminant validity. [Page 498]This means that measures of different traits should be distinct. When rating someone's personality, a friend or coworker should distinguish between that person's extroversion and his or her conscientiousness.

...