9 Related Links - Unsupervised Learning
    In this chapter Ballard covers basic concepts in unsupervised learning. It starts revisiting the principal component analysis, firstly outlined in Chapter 4, now observing that this interesting processing can be performed inherently, and without prohibitive computational demands, by some neural networks. This is accomplished by using a two-layer neural network using linear units, with evolution governed in such a way as to find the eigenvector associated to the largest eigenvalue. However, the reader is refereed to the literature for an extension allowing to find more eigenvalues in descending order [Sanger]. The next section introduces and characterizes the concept of competitive learning, followed by a section on topological constraints, which is finely motivated and illustrated with respect to the travelling salesman problem.
    The issue of supervised competitive learning is discussed in the subsequent section. Here, the motivation is to allow supervision in the initial definition of the space partitions in order to secure a more rational and effective classification. The biologically very relevant issue of multimodal data is covered in the next chapter in terms of a technique based on gradient descent over an energy function defined in terms of joint density functions describing disagreement between modalities. This algorithm is outlined in a practical way and illustrated for the case of simultaneous visual and acoustic signals. The last section in this chapter addresses independent components. The beginning of this section provides a good example of a situation which tends to recur from time-to-time, but fortunately not too frequently, in Ballard's book, especially in the last sections in some chapters (see Sections 4.6, 5.3.2 and 7.5). As a matter of fact, the first paragraph in Section 9.7 reads as notes for somebody who is already quite familiar with the covered topic. If not, what is a "cause" or "source" s? In which sense is a cause "hidden" or "independent"? Why should causes be experienced (by who or what?) as data (what does not qualify as data?) because of the transformation x = As? What is x? What is matrix A? Why should a compact code be one that recovers the original causes? It is rather unlikely that such an introduction will make any sense to the novice reader.