6 Optimization
    This chapter discusses how optimal trajectories can be determined during the dynamical evolution along the state space. This typically involves defining constraints and objective functions which reflect the aimed goals. As observed in its abstract, the material in this chapter is a preparation for the learning algorithms to be presented in subsequent chapters.
 
    Basically, this chapter starts by covering simple minimization algorithms (without constraints), including algebraic solution of equations and the gradient descent technique. Next, the Lagrange multipliers approach, which allows the inclusion of constraints to the solution space, is nicely introduced and exemplified for determining the distance between a point and a circle. The next topic in this chapter is optimal control. Now, it is felt that the motivation and mathematical development of this approach will prove to be beyond reach for many typical readers of this book. In case you are already familiar to the optimal control approach, Ballard's approach may seem elegant and compact to you, but then this is an introductory book. Consider, for instance, the abrupt introduction of the Hamiltonian concept in page 122. This is indeed a pity, since not only this is a nice and important physical concept worth some discussion and illustration, but also because Ballard has excellent potential for conceptually explaining difficult concepts. Although the optimal control is illustrated in terms of a simple problem in mechanics and the issue of noise in linear systems, this is likely not to be enough to make the reader feel at ease with the presented concepts.
 
    The final section in this chapter presents and illustrates (for the same mechanical problem addressed in the previous section) the interesting optimization approach known as dynamic programming, which relies on the principle of optimality but is basically restricted to discrete systems. The problem with this approach is observed to be the curse of dimensionality.