Self‐Organizing Maps

Citation

Andrew, A.M. (1998), "Self‐Organizing Maps", Kybernetes, Vol. 27 No. 1, pp. 102-104. https://doi.org/10.1108/k.1998.27.1.102.3

This is an important book, dealing with the most successful neural‐net algorithm allowing unsupervised learning, and written and updated by the originator. Much of the treatment closely follows the earlier (1995) edition, but the opportunity has been taken to include a considerable amount of new material. As the author states in his preface to the new edition, the even‐numbered chapters 2, 4, 6, 8 and 10 are essentially the same as before, but the odd‐numbered ones from 1 to 9 are extensively revised and extended. The main reason for revision is to include fresh developments, some of them extensions of the theory and others being new applications.

It is pointed out that mapping is a ubiquitous feature of the nervous system, perhaps most dramatically illustrated by the sensory and motor homunculi of the cerebral cortex, but also seen in many other parts of the brain. In view of this, the author is surprised that the mapping problem had not received attention prior to his work on it. A powerful algorithm is described, in a general form permitting variations in detail, for reorganising the mapping of a set of inputs on to an array of units that is usually a two‐dimensional sheet, though linear and higher‐dimensional arrays are also considered.

The new mapping is such that destinations of similar inputs tend to be adjacent, and the spacing depends on the frequency of activation. The method allows the mapping of multi‐dimensional inputs onto a two‐dimensional sheet, preserving adjacency as an indication of signal similarity. Constraints on the distribution of source signals in the multi‐dimensional space, for example their restriction to a particular multi‐dimensional volume, are also reflected.The mapping does not immediately provide the effect of “clustering”, but when the frequency of activation of the points in the map is transformed into a background grey level, segmentation of the area by “ravines” of low activation frequency is often apparent, and the segments then correspond to significant “clusters”. The mapping process is introduced as corresponding to the needs of hash coding, where codebook vectors, represented by hash addresses, are chosen to cover some range with roughly uniform frequency of activation. A correspondence to the statistical technique of regression analysis is also noted.

Unsupervised learning schemes tend to be strongly dependent on how their inputs are presented, and the basic self‐organising mapping scheme (SOM) is no exception, as the author acknowledges in his preface:

Preprocessing should not be overlooked. The ANN [Artificial Neural Net] algorithms are no sausage machines where raw material (data) is input at one end and results come out at the other. Every problem needs a careful selection of feature variables, which so far is mostly done by hand. We are just at the dawn of automatic feature extraction, using ANN models such as ASSOM.

The abbreviation ASSOM refers to “Adaptive‐Subspace Self‐Organising Map”, which is an important development treated more fully in this edition than in the earlier one. Its discussion begins with the observation that visual recognition schemes generally have to recognise moving objects, and most approaches require preprocessing to provide a steady image. The ASSOM is described as a special kind of SOM in which the various map units adaptively develop into filters of basic invariant features.

In an ASSOM, the “neural” units do not correspond to individual input patterns but to different subspaces defined by basis vectors. It is not a single pattern, but an “episode”, or sequence of patterns, on which the map units compete. Instead of a relatively simple indicator of pattern similarity, an integral that can be termed the “energy of the projections” allows selection of a “representative winner” from the map units. As in the basic SOM, the learning process involves modification of both the representative winner and other units in its neighbourhood. This allows convergence on subspaces that provide the necessary invariance under a set of transformations, such as, in the case of visual inputs, those of translation, rotation and dilatation.

The argument is developed with reference to the earlier work of Gabor on filters to isolate “wavelets”, as a means of achieving a representation that is invariant under at least some of the types of transformation. It is demonstrated that an ASSOM can form Gabor‐type filters automatically.

Kohonen’s ideas have been developed in the context of a wide variety of recognition tasks, including a project on speech recognition. In its context a principle of Learning Vector Quantisation (LVQ) was devised, amounting to a version of SOM that is “supervised” in that it accepts feedback, and is aimed at pattern classification. The input data elements for speech recognition have to consist of phonemes along with a certain amount of context, and the amount of context has to be chosen to eliminate ambiguities and to allow some error‐correction where phonemes may have been wrongly classified. It is unprofitable to include too much context. A technique of Dynamically Expanding Context (DEC) is described, allowing automatic choice of a suitable amount.

The overall treatment starts with a discussion of mapping in the nervous system, and there are references to biology throughout. The fourth chapter (following the introduction of the basic SOM idea) spells out implications for neural modelling, including a case for the existence of neurons serving no other purpose than adjustment of the plasticity, or readiness to accept modification, of other neurons.

The last three chapters deal, respectively, with Applications, Hardware, and an overview of the SOM literature. A wide range of application areas is mentioned. A relatively simple application of the basic SOM is to simple robot‐arm control, where a SOM device can map from the primary arm co‐ordinates, normally consisting of joint angles, on to a plane array. The SOM behaviour is such that the mapping on to this array comes to correspond to the position of the gripper in real space, and a simple version of this example was used to illustrate the SOM principle in the second chapter.

The book is clearly written and is intended to be readable and self‐sufficient. The necessary mathematics are introduced from a fairly elementary level, and although there is a fair sprinkling of equations there is also the attempt to explain the principles in plain language instead of letting the equations speak for themselves. Although the manner of presentation is admirable, there is a sense in which the apparent simplicity is deceptive. The book contains a very large amount of valuable material and comment, and this review can only indicate a part of it. The book is an extremely important contribution to the literature.