Unobtrusive Measures

Introduction

All measures must be considered to be subject to error. We would have no way of knowing if, perchance, a measure turned out to be absolutely accurate. For example, the National Bureau of Standards has a ten gram weight (actually, because of manufacturing error just less than ten grams by about 400 micrograms, the weight of a grain or two of salt) (Freedman, Pisani & Purves, 1991). Despite the most careful weighing, done on a weekly basis, the values actually obtained for the standard weight vary by about 15 micrograms (one microgram is about the weight of a speck of dust) either way from the mean. Presumably, the mean of a long series of measures is the best estimate of the true weight of the standard, but there is no way of being sure of the true weight. Obviously, the way to deal with such errors, usually thought of as random errors, is to do the same measure several times. That is why careful carpenters measure their boards more than once before cutting them especially if the wood is expensive. Random errors are associated with the concept of reliability, the expectation that if a measurement is performed twice under exactly the same conditions, very close agreement should be obtained if the measurement instrument and procedures for using it are dependable.

Another type of measurement error, though, is bias, i.e. consistent deviation from the ‘true’ value of the object or phenomenon of interest. For example, a recent news story recounted speculation that, in order to enhance the prospects of players for careers in professional American football, some collegiate coaches might have arranged the running tracks, on which the players' speed is tested, to be slightly downhill. If that is true, then the running speeds of players from some universities may be biased towards the fast end of the scale. If mothers are asked to estimate the intelligence (or good looks!) of their children, we might expect some bias to be evident, i.e. higher estimates than would be expected from other means of assessment. The bias is not reduced by replicating the measurement. Downhill is still downhill, and mother's love is constant.

The problem of bias has to be dealt with either by knowing the degree of bias so one can allow for it or by using multiple measures that do not share sources of error, i.e. bias (see Sechrest, 1979; Sechrest & Phillips, 1979). A football scout who knows that a player has been tested on a track with a downhill slant can ‘allow’ for that in interpreting reported running speed. We may expect that mothers will exaggerate a bit in describing their children's skills or other virtues and discount the glowing adjectives in the descriptions by some amount. But an even better way to deal with bias is to use other measures that are less biased, or not biased at all in the same way as the original measure. Of course, if a scout knows that a track has a downhill bias, that scout may simply retest [Page 1058]the athlete on a track known to be quite level and disregard the first, presumably biased, report. A scout might also use game films to arrive at a judgement of the speed of the player.

...