Mar 2
Discuss the homework.
The next topic is error correcting codes, which is chapters 6 & 7 in Biggs, our yellow texbook.
An assignment with readings and exercises from the text is posted - our last one before the break.
To get us going, we'll walk through some of
the chapter 6 material during class ...
but your next mission is to read chapters 6 & 7.
The basic idea is extend the notions from chapter 5
(channel capacity, which we just did)
to codes with words of multiple bits given probabilities
of error per bit. We take only some of the possible
words as legal symbols, and construct a rule for
decoding received words which may have errors in them,
using the "hamming distance".
While there are (again) many definitions and formulas
in these chapters, the underlying ideas are straightforward
once you untangle the symbols.
input -> encoding C -> add noise -> decision decode -> output
decision rule = not all messages need be in code C
mistake = when output not same as input
C = { 010 110 ... } set of code words
= subset of binary words of length n
"extended BSC" ... "binary symmetric channel" with n bits.
hamming distance = d(x,y) = number of bits flipped
entries in noise probability matrix :
gamma(n bits)_xy = epsilon**d (1-epsilon)**(n-d)
where d = d(x,y) = number of bits flipped
Exercises to try in class :
- exercise 6.4 ( one row \( \Gamma \) for a 2-bit code) in class, page 95
- 6.7 (find closest code to codeword with error) on page 99, and any of the following
- top of page 102 (a 1-error correcting code C ⊆ B6 with |C| = 5.), 6.12 to 6.14
Buzz phrases
- "minimum distance decision rule" : pg 99.
- "error detecting"
- "error correcting"
- minimum distance of a code C; neighborhood
- "r-error-correcting code" if delta=min-distance >= 2r+1
Explain the differences between these decision rules for resolving codes with errors. Can you construct a situation in which all three are different?
- the minimum distance rule
- the maximum likelihood rule
- the ideal observer rule (original word with highest probability)
