10 Markov Models
    This chapter covers Markov models and their hidden version. The former is suitably introduced (though Markovianity is not extended to finite memory, but restricted to the current state) and the stationary state of the evolution of regular Markov chains characterized in terms of the eigenvectors of the transition matrix. Irregular chains are only very briefly commented. The next section is dedicated to hidden Markov models. Here Ballard provides a nice motivation to the problem of trying to infer the unknow ("hidden") state structure of stochastic processes in the world. Some basic concepts are introduced, in terms of a simple urn model, and then the three principal problems in the hidden Markov approach outlined. Each of these problems are treated at an introductory level, but yet in some depth, in the following subsections.