Mar 28
Begin discussion of ideas from chapter 8 : Linear Codes
"adding" and "multiplying" code words :
- for "+" we'll use bitwise XOR (needs to be closed)
- regular 0,1 multiplication
- then linear algebra still works!
A "linear code" : make code words by "adding" together other codewords.
- Get distances between 'em by grid separations, i.e. basis vectors!
- k basis vectors means 2**k codewords (including k 0's)
Getting codewords from a matrix of column basis vectors
The "E for encoding" matrix :
1 0 0
E = 0 1 0
0 0 1
1 1 0
0 1 1
1 0 1
Data words are (000, 001, ... 111).
Then we get code words by matrix multiplication!
E * [ 0 ] = [ 0 ]
[ 1 ] [ 1 ]
[ 1 ] [ 1 ]
[ 1 ]
[ 0 ]
[ 1 ]
5x3 * 3x1 = 5 x 1
- Do this in class, all together until the ideas makes sense.
- Do 8.7 - 8.9 together depending on time.
For one error correcting codes, after the message is sent, we multiply on the left by the check matrix H , which gives 0 if the message has no errors.
The executive summary of the notation and procedure is :
k = bits in a "block" of data
so 2**k data words which will call "x"
n = bits in error-correcting codewords, n > k ( y )
so 2**n code words which we'll call "y"
ρ = information rate = k / n
δ = minimum hamming distance between codewords
{C} = set of codewords , with |C| = number of codewords = 2**k
The C linear code is a subspace of the 2**n codes+errors.
C is k-dimensional.
E is (n x k = rows x columns) encoding matrix :
y = E x
where |y| = n, |x| = k
H is (n-k x n) check matrix,
H y = 0
We can confirm that a word y has no errors when
H y = 0
When there *is* an error, for 1-error correcting codes,
then (H y) tells us where the error is!
The "cannonical" form for E and H is
E = [ I ] H = [ A I' ]
[ A ]
where
I is k x k identity matrix,
I' is (n-k) x (n-k) identity matrix
A is (n-k) x k 2-ones-per-column, no two the same
Magic!
(I mistakenly claimed earlier that A must have linearly independent columns - that's incorrect.)
Make sure you understand what the variables are,
the dimensions of the various matrices,
and what they're used for.
Section 8.4 gives a simple recipe for
building 1-error correcting codes
using these building blocks.
The text is *very* mathish in chapter 8, but see if you can get the gist
of the procedure by looking at the examples.
There is some very cool linear algebra and group theory going on
behind the scenes here which is making all this work ... which
we will discuss to see if we can make sense of some of the ideas.
The ideas here use two slightly different math notions:
- linear algebra (the matrix E which maps data words to code words is a linear map)
- group theory (the set of codewords is a group under addition)
Here are some starting points for the mathish stuff for those interested.
Some more of the math technical terms :
- A "vector space"
- A set of "basis vectors"
- A "subspace"
- "column space" of a matrix
- "orthogonal" vectors
- matrix as "linear operator"
The upshot set of all codewords (with and without errors) can be broken
into neighborhoods (cosets) which have the same value ("syndrome")
when multiplied by the check matrix. These neighborhoods are also the little circles
in our previous arguments about the packing bounds, namely a message
without an error and those 1 bit flip away (for 1 error correcting codes.)
Examples will hopefully illuminate what's going on.
