Nov 24
chap1 :
- basic concepts & definitions
- matrix math of weights & biases
- sigmoid function
- "gradient" notion (calculus, for backprop)
chap2 :
- backprop in some detail
- lots more matrix math
- use error at last layer to find error in previous layer
- using multivariable calculus
It may be useful to work through a much simpler
math problem to do the same sort of thing that
backprop does, to understand the math better.
For example, set up a neural net with just
a few layers and nodes, assign numbers,
invent a specific final error, and then
"backprop" it by hand to find the changes
in the weights and biases.
chap 3 :
- ways of improving neural net preformance
- cross entropy cost (rather than quadratic)
- discuss "information entropy"
- "regularization" methods to avoid over fitting
- L1, L2 (penalize by sum of weights)
- "dropout" training
- "artificial expansion" of training set
- initializing weights better
- various other
- higher order derivatives in backprop (computationally expensive)
- tanh activation function (-1 to 1, not 0 to 1)
chap 4 & 5 :
