Trimmed Mean

There are at least three fundamental concerns when using the mean to summarize data and compare groups. The first is that hypothesis testing methods based on the mean are known to have relatively poor power under general conditions. One of the earliest indications of why power can be poor stems from a seminal paper by John Wilder Tukey published in 1960. Roughly, the sample variance can be greatly inflated by even one unusually large or small value (called an outlier), which in turn can result in low power when using means versus other measures of central tendency that might be used. A second concern is that control over the probability of a Type I error (when the true hypothesis is wrongly rejected) can be poor. It was once thought that Type I error probabilities could be controlled reasonably well under non-normality, but it is now known why practical problems were missed and that serious concerns can arise, even with large sample sizes. A third concern is that when dealing with skewed distributions, the mean can poorly reflect the typical response.

There are two general strategies for dealing with the problem of poor power, with advantages and disadvantages associated with both. The first is to check for outliers, discard any that are found, and use the remaining data to compute a measure of location. The second strategy is to discard a fixed proportion of the largest and smallest observations and average the rest. This latter strategy is called a trimmed mean, which includes the median as a special case, and unlike the first strategy, empirical checks on whether there are outliers are not made.

A 10% trimmed mean is computed by removing the smallest 10% and the largest 10% of the observations and averaging the values that remain. To compute a 20% trimmed mean, remove the smallest 20%, the largest 20%, and compute the usual sample mean using the data that remain. The sample mean and the usual median are special cases that represent two extremes: no trimming and the maximum amount.

A basic issue is deciding how much to trim. In terms of achieving a low standard error relative to other measures of central tendency that might be used, 20% trimming is a reasonably good choice for general use, except when sample sizes are small, in which case it has been suggested that 25% trimming be used instead. This recommendation stems in part from the goal of having a relatively small standard error when sampling from a normal distribution. The median, for example, has a relatively high standard error under normality, which in turn can mean relatively low power when testing hypotheses. This recommendation is counterintuitive based on standard training, but a simple explanation can be found in the books listed under Further Readings.

Moreover, theory and simulations indicate that as the amount of trimming increases, certain known problems with means, when testing hypotheses, diminish. This is not to say that 20% is always optimal; it is not, but it is a reasonable choice for general use. However, if too much trimming is used, with the extreme case being the median, then power can be relatively poor under normality. And when comparing medians, special methods are required for dealing with tied values.

...