Llahsjh

8.1 Signals look like noise!

One of the most important practical questions which arises when we are designing or using an information transmission or processing system is, "What is the Capacity of this system? — i.e. How much information can it transmit or process in a given time?" We formed a rough idea of how to answer this question in an earlier section of this set of webpages. We can nowgo on to obtain more well defined answer by deriving Shannon's Equation. This equation allows us to precisely determine the information carrying capacity of any signal channel.

Consider a signal which is being efficiently communicated (i.e. no redundancy) in the form of a time-dependant analog voltage, . The pattern of voltage variations during a specific time interval, T, allows a receiver toidentify which one of a possible set of messages has actually been sent. At any two moments, & , during a message the voltage will be & .

Using the idea of intersymbol influence we can say that since — there is no redundancy — the values of & will appear to be independent of one another provided that they're far enough apart () to be worth sampling separately. In effect, we can't tellwhat one of the values is just from knowing the other. Of course, for any specific message, both and are determined in advance by the content of that particular message. But the receiver can't know which of all the possible messages has arrived until it has arrived. If the receiver did know in advance which voltage pattern was to be transmitted then the message itself wouldn't provide any newinformation! i.e. the receiver wouldn't know any more after its arrival than before. This leads us to the remarkable conclusion that a signal which is efficiently communicating information will vary from moment to moment in an unpredictable, apparently random, manner. An efficient signal looks very much like random noise!

This, of course, is why random noise can produce errors in a receivedmessage. The statistical properties of an efficiently signalled message are similar to those of random noise. If the signal and noise were obviously different the receiver could easily separate the noise from the signal and avoid making any errors.

To detect and correct errors we therefore have to make the real signal less ‘noise-like’. This is what we're doing when we use parity bits to add redundancyto a signal. The redundancy produces predictable relationships between different sections of the signal pattern. Although this reduces the system's information carrying efficiency it helps us distinguish signal details from random noise. Here, however, we're interested in discovering the maximum possible information carrying capacity of a system. So we have to avoid any redundancy and allow thesignal to have the ‘unpredictable’ qualities which make it statistically similar to random noise.

The amount of noise present in a given system can be represented in terms of its mean noise power

where R is the characteristic impedance of the channel or system and is the rms noise voltage. In a similar manner we can represent a typical message in terms of its average signal power

where is thesignal's rms voltage.

A real signal must have a finite power. Hence for a given set of possible messages there must be some maximum possible power level. This means that the rms signal voltage is limited to some range. It also means that the instantaneous signal voltage must be limited and can't be beyond some specific range, . A similar argument must also be true for noise. Since we areassuming that the signal system is efficient we can expect the signal and noise to have similar statistical properties. This implies that if we watched the signal or noise for a long while we'd find that their level fluctuations had the same peak/rms voltage ratio. We can therefore say that, during a typical message, the noise voltage fluctuations will be confined to some range

where the form...