Nov 29
aside
projects
status update : project progress? Presentations in 1 week.
Project rubrik:
- idea / difficulty / planning
- mastery of course material
- follow through / finished & runs without error
- docs / tests / sample output / easy to see how it works
end of term evals
... today or Thur
Hidden Markov Models
last topic this term; discuss this week.
- a very common model in robotics, speech, ...
- related to finite state & context free grammars
- a lot of notation and perhaps confusing math
- ... so you need to try some really simple examples on your own, and do some staring
aima-class.com section 11 is quite good; somewhat specific on "particle filters", clearly a favorite of Thrun's .
- robot localization problem
- hidden part: where am I?
- observations: sensor readings
- particles: distribution of where I might be
- after each observation, use as weights to give probabilities to particles via Bayes
- then update particle positions (state transition matrix or "convolution" with noise & positions)
- repeat
Main point (particle, forward, Virterbi, ...): how to use existing
state probability vector, transition matrix, observations,
and probability of observations (via Bayes) to move state
probability vector forward. (See my math below).
Maybe watch the videos together?
Discuss "stationary" long term behavior of state probabilities.
- Do we need to look at matrix multiplication?
- long term behavior is an eigenvalue problem ...
Browse through the attached "stock market" example,
in particular how the dow.py implements a brute force approach, and
why that blows up quickly.
Finally, do the Viterbi recursive forward approach by hand
for this simple problem. (If need be, finish this on Thursday.)
pg 548 in AMA v2
* ... with slightly different numbers :
P(rain t+1 | rain t) = .7
P(rain t+1 | ! rain t) = .2
P(umbrella | rain) = .9
P(umbrella | ! rain) = .2
* observed umbrella usage days 1 .. 5 = (T, T, F, T T)
* start with P(rain) = .1 on day 0 before umbrella seen
use Trellis diagram and Viterbi algorithm
to find most likely weather sequence.
How much storage and run-time does this
need as function of number of days ?
How many weather sequences are there?
How does that grow as number of days ?
Summary of the Viterbi math:
Pi[i] = initial a priori state probabilities
A[i,j] = state transition = P(state i at time t | state j at time t-1)
B[k,j] = observation probabilities = P(observe k at time t | state j at time t)
Viterbi is a recursive algorithm for finding best path through
Trellis of possible state sequences :
t=1 t=2 t=3
--------------------
rain rain rain
sun sun sun
cloud cloud cloud
--------------------
obs1 obs2 obs3
Look for most likely path from t=1...n.
Then given that, find most likely path to t=n+1
using this (forward) recursion relation
Pr(path to x at t=n+1) = C sum_y Pr(path to y at t=n)*P(x|y)*P(obs|x)
using Bayes to turn P(x|obs) into C*P(obs|x)*P(x) and P(x)=P(x|y)*P(y)
where y is a previous state, and Pr() is a probability of the
entire path of states (that's the Viterbi approach).
To start the recursion off, we get the earliest state vector from
the first observations and an assumption about its prior inital state.
For Thurs: try this out yourself.
![[paper clip]](/courses/source/wiki_images/paper_clip_tilt.png)
name last modified size