Law of Large Numbers

History

Jakob Bernoulli first proposed the Law of Large Numbers in 1713 as his “Golden Theorem.” Since that time, numerous other mathematicians (including Siméon-Denis Poisson who first coined the term Law of Large Numbers in 1837) have proven the theorem and considered its application in games of chance, sampling, and statistical tests. Understanding the Law of Large Numbers is fundamental to understanding the essence of inferential statistics, that is, why one can use samples to estimate population parameters. Despite its primary importance, it is often not fully understood. Consequently, the understanding of the concept has been the topic of numerous studies in mathematics education and cognitive psychology.

Sampling Distributions

Understanding the Law of Large Numbers requires understanding how sampling distributions differ for samples of various sizes. For example, if random samples of 10 men are drawn and their mean heights are calculated so that a frequency distribution of the mean heights can be created, a large amount of variability might be expected between those means. With the mean height of adult men in the United States at about 70 inches (5′10′ or about 177 cm), some samples of 10 men could have means as high as 80 inches (6′6′), whereas others might be as low as 60 inches (5′0′). Although as the central limit theorem suggests the mean of the sampling distribution of the means will be equal to the population mean of 70 inches, the individual sample means will vary substantially. In samples of only 10 randomly selected men, it is easily possible to get an unusually tall group of 10 men or an unusually short group of men. Additionally, in such a small group, one outlier, for example who is 85 inches, can have a large effect on the sample mean. However, if samples of 100 men were drawn from the population, the means of those samples would vary less than the means from the samples of 10 men. It is much more difficult to select 100 tall men randomly from the population than it is to select 10 tall men randomly. Furthermore, if samples of 1,000 men are drawn, it is extremely unlikely that 1,000 tall men will be randomly selected. The mean heights for those samples would vary even less than the means from the samples of 100 men. Thus, as sample sizes increase, the variability between sample statistics decreases. The sample statistics from larger samples are, therefore, better estimates of the true population parameters.

Demonstration

If a fair coin is flipped a million times, we expect that 50% of the flips will result in heads and 50% in tails. Imagine having five people flip a coin 10 times so that we have five samples of 10 flips. Suppose that the five flippers yield the following

...