Shannon, Claude

1916–2001

Mathematician and Scientist

Mathematician, engineer, scientist, inventor, and juggler Claude Shannon is the founder of information theory and is widely regarded as having provided the conceptual framework for modern communications systems.

Born in Gaylord, Michigan, in 1916, Shannon received a B.S. from the University of Michigan and a M.S. and Ph.D. from the Massachusetts Institute of Technology (MIT) in 1940. Shannon's master's thesis, entitled A Symbolic Analysis of Relay and Switching Circuits, used George Boole's algebraic system for manipulating the numbers 0 and 1 to explain electrical switching. Shannon argued that binary values in symbolic logic and electric circuits were essentially identical, and that it would therefore be possible to build a “logic machine” that used switching circuits according to the principles of Boolean algebra.

After completing his doctorate, Shannon went to work for AT&T's Bell Laboratories, where he did some of his most famous work. He was hired to help Bell increase the efficiency of signal transmission down telephone lines. Bell wanted to be able to transmit a maximum number of signals down a single line, thereby using its system to its full capacity. By applying the principles of Boolean algebra to telephone switches, Shannon provided a major breakthrough in resolving this efficiency problem.

In 1948, Shannon published his landmark “Mathematical Theory of Communication” in two parts in the Bell System Technical Journal. In this work, Shannon showed that information could be quantified and therefore analyzed mathematically, providing a conceptual basis for information theory. Many commonplace analytical terms of information science and computer engineering, such as “bit,” come from “Mathematical Theory.”

Shannon wrote that the fundamental unit of information—a bit—is a yes/no situation, which could be represented by a switch that is either on or off. All information, even very complex information, can be built from this basis of zeroes and ones. Complex concepts can be conveyed as in a game of 20 Questions, where the number of bits is sufficient to provide fairly detailed information. As a system uses more bits, it is able to perform more complex computations at an exponential rate. A single bit system has two states: 0 and 1. A 2-bit system has 4 states: 0,0; 0,1; 1,0; and 1,1. A 4-bit system has sixteen states, and so forth.

Shannon's insight was significant for two reasons. First, it quantified information, thereby making it measurable, and therefore useful, for engineering purposes. It also separated information, which could be measured, from meaning, which to this day has not been quantified in any significant way. Scholars interested in the meaning of communicative events have had to look beyond Shannon for their inspiration; but Shannon's work allowed for a totally new way to conceptualize the process of communication in circuits of people or machines.

In Shannon's model, information is a degree of order in a system. But entropy (disorder) is also very important for information theory. If there is too much entropy, then no information is communicated; but if there is no entropy, then no information is communicated and the system remains static. Shannon used mathematical proofs to show that one could use a measure of entropy to define the capacity of a communication channel. So long as the channel stays under capacity, the transmission will be free of errors. This was useful to AT&T, because people needed to be able to hear one another at both ends of a phone line.

...