Multitrait-Multimethod Matrices

Introduction

A multitrait-multimethod matrix (MTMM) refers to a matrix of correlations among multiple theoretical/empirical constructs (traits), each of which is defined operationally by a common set of measurement procedures (methods). Convergence and divergence principle provide a framework underlying the use of MTMM matrices in construct validation research: different operationalizations of a particular construct are expected to measure that construct (convergence) whereas constructs with different labels are expected to refer to different theoretical entities when measured by the same method (divergence). Developed over the past four decades, a large number of analytic methods are now available to analyse MTMM matrices.

Perhaps the most provocative challenge to logical positivism and its core concept of operational definitions of scientific concepts came from the pioneering work of Campbell and Fiske (1959). Definitional operationalism equates a scientific concept to the method (i.e. operation) of obtaining it. Consider, for instance, an IQ score of 110 obtained by using the Wechsler Adult Intelligence Scale (WAIS). The scientific concept of intelligence is defined operationally by the WAIS in this example. Hence, the score of 120 is ‘the intelligence’. Despite the fact that there is no ambiguity as to what the intelligence is in this example, Campbell and Fiske argued convincingly that the score of 120 should be more appropriately labelled as ‘the intelligence measured by WAIS’. The difference between these two labels is profound because there are other ways (i.e. methods) to measure intelligence, e.g. many other intelligence tests in the form of ratings and behavioural observations, neuropsychological measures, and educational attainments that do not necessarily provide congruent information about one's intelligence. After all, most widely used intelligence tests have less than half of their variances in common. Therefore, the influences of methods on test scores, which is considered as imperfect measures of theoretical constructs, need to be investigated empirically.

Trait measurement is an integral part of psychological assessment. To investigate the trait validity of psychological test scores and the potential biasing effect of measurement methods, Campbell and Fiske proposed the multitrait-multimethod matrix (MTMM), which is a matrix of correlations between multiple traits measured by multiple methods. Traits are conceptualized as individual difference variables which can only be observed indirectly, e.g. intelligence, depression, and self-esteem. There are often multiple procedures of measuring traits. Standardized operations defining the processes of trait measurements are usually referred to as methods, e.g. self-report, direct observations, and supervisory ratings. A measure refers to a trait-method unit, e.g. aggression measured by teacher rating. The MTMM matrix, formed by correlating multiple [Page 611]measures, contains information on traits, as well as methods.

Mtmm Matrices

For illustration purposes, an MTMM matrix of four traits, i.e. social problems (soc), attention problems (att), rule-breaking behaviour (rbb), and aggressive behaviour (agg), measured by three methods, i.e. ratings from mother (M), teacher (T), and self (S), appears in Table 1. The MTMM matrix was obtained by the administration of Child Behaviour Checklist, Teacher's Report Form, and Youth Self-Report instruments to 1934 school-age individuals (Achenbach & Rescorla, 2001). The measure labels indicate the method followed by the trait, e.g. T_rbb for teacher's report of rule-breaking behaviour and S_att for self-reported measure of attention problems. Campbell and Fiske classified the MTMM correlations into three sets: (a) different-trait/different method, (b) different-trait/same-method, and (c) same-trait/different-method.

...