Biol. Cybern. 61,241-254 (1989)
Biological
Cybernetics
9 Springer-Verlag 1989
Self-Organizing Semantic Maps
H. Ritter* and T. Kohonen
ttelsinki University of Technology, Laboratory of Computer and Information Science, Rakentajanaukio 2 C, SF-02150 Espoo, Finland
Abstract.
Self-organized formation of topographic
maps for abstract data, such as words, is demonstrated
in this work. The semantic relationships in the data are
reflected by their relative distances in the map. Two
different simulations, both based on a neural network
model that implements the algorithm of the self-
organizing feature maps, are given. For both, an
essential, new ingredient is the inclusion of the con-
texts, in which each symbol appears, into the input
data. This enables the network to detect the "logical
similarity" between words from the statistics of their
contexts. In the first demonstration, the context simply
consists of a set of attribute values that occur in con-
junction with the words. In the second demonstra-
tion, the context is defined by the sequences in which
the words occur, without consideration of any as-
sociated attributes. Simple verbal statements consist-
ing of nouns, verbs, and adverbs have been analyzed in
this way. Such phrases or clauses involve some of the
abstractions that appear in thinking, namely, the most
common categories, into which the words are then
automatically grouped in both of our simulations. We
also argue that a similar process may be at work in the
brain.
1 Hypotheses About Internal Representation
of Linguistic Elements and Structures
1.1 The Objective of this Work
One of the most intriguing problems in the theory of
neural networks, artificial and biological, is to what
extent a simple adaptive system is able to find abstrac-
* On leave from: Institut fiir Physik, Technische Universit~it
M/inchen, James-Franck-Strasse, D-8046 Garching, FRG
Present
address: Beckman Institute, University of Illinois,
Urbana, IL 61801, USA
tions, invariances, and generalizations from raw data.
Many interesting results, e.g., in pattern recognition
(artificial perception of images as well as acoustical and
other patterns) have already been obtained. Extraction
of features from geometrically or physically related
data elements, however, is still a very concrete task, in
principle at least. A much more abstract and enigmatic
object of study is
cognitive information processing
that
deals with elements of consciousness and their relation-
ships; it is frequently identified with the ability to use
languages. The purpose of the present paper is to study
whether it is possible to create in artificial neural
networks abstractions such that they, at least in a
primitive form, would reflect some properties of the
cognitive and linguistic representations and relations.
In particular we are here reporting new results
which demonstrate that a self-organizing process is
indeed able to create over a neural network topo-
graphically or geometrically organized maps that
display semantic relations between symbolic data. It
may be proper to call such representations
self-
organizing semantic maps.
We are also relating our results to the fundamental
basis of cognition, namely,
categorization of observa-
tions.
As the connection of these ideas to fundamental
theories of knowledge might otherwise remain ob-
scure, it may be proper to commence with a short re-
view of the philosophical background, namely, the
theory of categories
as the ultimate framework of ab-
stractions.
1.2 On Categories and their Relation to Linguistic
and Neural Representations
The most general concepts or abstractions that are
needed to interprete the empirical world are called
categories;
such basic reduced elements and forms of
thinking and communication can also be encountered
in all languages, primitive as well as more developed.
