Teoría Matemática De La Comunicación

A M athematical

Theory

of Communication

B y C. E. SHANNON
INTRODUCTION

T

H E recent development of various methods of modulation such as PCM
and PPM which exchange bandwidth for signal-to-noise ratio has intensified the interest in a general theory of communication.
A basis for
such a theory is contained in the important papers of Nyquist’ and Hartley*
on this subject. In thepresent paper we will extend the theory to include a
number of new factors, in particular the effect of noise in the channel, and
the savings possible due to the statistical structure of the original message
and due to the nature of the final destination of the information.
The fundamental problem of communication is that of reproducing at
one point either exactly or approximately a messageselected at another
point. Frequently the messages have mea&g; that is they refer to or are
correlated according to some system with certain physical or conceptual
entities. These semantic aspects of communication are irrelevant to the
engineering problem. The significant aspect is that the actual message is
one selectedfrom a set of possible messages. The system must be designed
to operatefor each possible selection, not just the one which will actually
be chosen since this is unknown at the time o f d esign.
If the number of messages in the set is finite then this number or any
monotonic function of this number can be regarded as a measure of the information produced when one message is chosen from the set, all choices
being equally likely. As was pointed out by Hartley themost natural
choice is the logarithmic function. Although this definition must be generalized considerably when we consider the influence of the statistics of the
message and when we have a continuous range of messages, we will in all
cases use an essentially logarithmic measure.
The logarithmic measure is more convenient for various reasons:
1. It is practically more useful. Parameters ofengineering importance

Technical
Jwr* Nyquist, H., “Certain Factors Affecting Telegraph Speed,” B ellSysletn
n ol, April 1924, p. 324; “Certain Topics in Telegraph Transmission Theory,” A. I. E. E .
T tans., v. 47, April 1928, p. 617.
Teclanid
Journal,uly
J
1 Hartley, R. V. L., “Transmission of Information,” B ellSystem
1 928,. 5 35.
p

V ol. 27, PP. 379.423, 623.656, J uly,
CopyrightPrinted

1948 by AMERICAN
in U. S. A .

October,
1948
TBLEPIIONE
AND TELEGRAPII

Co.

5

R eissued

December.

1957

C . E. S hannon

6

such as time, bandwidth, number of relays, etc., tend to vary linearly with
the logarithm of the number of possibilities. For example, adding one relay
to a group doubles the number of possible states of the relays. It adds 1
to thebase 2 logarithm of this number. Doubling the time roughly squares
the number of possible messages, or doubles the logarithm, etc.
2. It is nearer to our intuitive feeling as to the proper measure. This is
closely related to (1) since we intuitively measure entities by linear comparison with common standards. One feels, for example, that two punched
cards should have twice the capacity of one forinformation storage, and two
identical channels twice the capacity of one for transmitting information.
3. It is mathematically more suitable. Many of the limiting operations
are simple in terms of the logarithm but would require clumsy restatement in
terms of the number of possibilities.
The choice of a logarithmic base corresponds to the choice of a unit for
measuring information.
If thebase 2 is used the resulting units may be
called binary digits, or more briefly bils, a word suggested by J. W. Tukey.
A device with two stable positions, such as a relay or a flip-flop circuit, can
store one bit of information.
iV such devices can store N bits, since the
total number of possible states is 2N and log,2N = N. If the base 10 is
used the units may be called decimal digits....