This chapter is organized around the
Minimum Description Length - MDL - concept,
which is addressed actually only along its three last pages. All the 17 previous
pages, plus the respective appendix, are intended to serve as a preparation for
this core concept, including Bayes' rule,
information theory, and classification.
As a matter of fact, the most conceptual presentation of MDL in Ballard's book is
to be found from pages 11 to 14. The key idea underlying this concept is to
quantify both the size of the data representation and the complexity of the algorithms
respectively involved for coding and dealing with such representations
The idea of balancing the lengths of the data representations and algorithms is
biologically reasonable since only a finite number of algorithms can accomodated into
the brain, and responses have to be as close to real-time as possible. Thus,
algorithms must be compact and fast, and the data must be filtered to a reasonable
degree. Since the best algorithms for specific data compressions may turn out to
be very complex and slow, it may be better to have suboptimal, but less complex,
algorithms in the brain. This idea of achieving a balance between these two aspects
is expressed in the MDL concept.
Although Ballard goes as far as to state that the brain is organized according to the
MDL concept, which indeed seems to be evolutionary reasonable, it should be stressed
that it is hard to be specific or categorical regarding what are the critical features
to be considered as economical (in terms of brain implementation) because we are not
sure about how information is processed in the brain. After all, the effectiveness
of algorithms depends on the specific computer architecture and data representation.
In other words, the physical nature of biological neurons and cells impose fixed constraints
that can not be changed at will in order to suit optimal entropy.