Feb 28
Continue the discussion of noise that we started last week
by walking through the material in MacKay's book, chapters 8 & 9.
Walk through some of his examples, particularly the "symmetric binary channel"
and "z channel".
Discuss the "noisy channel coding theorem" :
Associated with each discrete memoryless channel, there is a non-negative number
C (called the channel capacity) with the following property. For any ε > 0 and
R < C, for large enough N, there exists a block code of length N and rate ≥ R and
a decoding algorithm, such that the maximal probability of block error is < ε.
C is the maximum of I(X;Y) , which is the mutual information. The idea is that
we can find a binary code of length n to represent the k distinct possible
inputs, with an mistake rate (due to the noise) that approaches zero in
the limit of long messages. The rate of information is k/n . This will work
as long as k/n < C .
