
This chapter
covers Markov models and their
hidden version. The former is suitably introduced (though Markovianity is not extended to finite memory,
but restricted to the current
state) and the stationary state of the evolution of regular Markov chains
characterized in terms of the
eigenvectors of the transition matrix. Irregular chains are only very briefly commented. The next
section is dedicated to hidden Markov models. Here Ballard provides a nice motivation to the problem
of trying to infer the unknow ("hidden") state structure of
stochastic processes in the world. Some
basic concepts are introduced, in terms of a
simple urn model, and then the three principal problems
in the hidden Markov approach outlined. Each of these problems are treated at an introductory level, but
yet in some depth, in the following subsections.


