
This chapter discusses how optimal trajectories can be determined during the dynamical evolution
along the state space. This typically involves defining constraints and objective functions which reflect the aimed goals.
As observed in its abstract, the material in this chapter is a preparation for the learning algorithms to be presented in
subsequent chapters.


Basically, this chapter starts by covering simple minimization algorithms (without constraints),
including algebraic solution of equations and the gradient descent technique. Next,
the Lagrange multipliers approach,
which allows the inclusion of constraints to the solution space, is nicely introduced and exemplified for determining the
distance between a point and a circle. The next topic in this chapter is optimal control. Now, it is felt that the
motivation and mathematical development of this approach will prove to be beyond reach for many typical readers of this book.
In case you are already familiar to the optimal control approach, Ballard's approach may seem elegant and compact to you,
but then this is an introductory book. Consider, for instance, the abrupt introduction
of the Hamiltonian concept in page
122. This is indeed a pity, since not only this is a nice and important physical concept worth some discussion and illustration,
but also because Ballard has excellent potential for conceptually explaining difficult concepts. Although the optimal control
is illustrated in terms of a simple problem in mechanics and the issue of noise in linear systems, this is likely not to be
enough to make the reader feel at ease with the presented concepts.


The final section in this chapter presents and illustrates (for the same mechanical problem addressed
in the previous section) the interesting optimization approach known as dynamic programming, which
relies on the principle of optimality
but is basically restricted to discrete systems. The problem with this approach is observed to be the
curse of dimensionality.


