BioSystems 89 (2007) 257–263
Pattern storage in a sparsely coded neural network
with cyclic activation
Julius Stroffek
a
, Eduard Kuriscak
b
, Petr Marsalek
c,∗
a
Charles University Prague, Department of Pathological Physiology, U nemocnice 5, CZ-128 53 Praha 2, Czech Republic
b
Charles University Prague, Department of Physiology, Albertov 5, CZ-128 00 Praha 2, Czech Republic
c
Czech Technical University in Prague, Faculty of Biomedical Engineering, Nam. Sitna 3105, CZ-272 01 Kladno, Czech Republic
Received 18 November 2005; accepted 22 April 2006
Abstract
We investigate an artificial neural network model with a modified Hebb rule. It is an auto-associative neural network similar to
the Hopfield model and to the Willshaw model. It has properties of both of these models. Another property is that the patterns are
sparsely coded and are stored in cycles of synchronous neural activities. The cycles of activity for some ranges of parameter increase
the capacity of the model. We discuss basic properties of the model and some of the implementation issues, namely optimizing of the
algorithms. We describe the modification of the Hebb learning rule, the learning algorithm, the generation of patterns, decomposition
of patterns into cycles and pattern recall.
© 2006 Elsevier Ireland Ltd. All rights reserved.
Keywords: Hebb rule; Auto-associative neural network; Hopfield network
1. Introduction
In his famous book Donald Hebb suggested a mecha-
nism describing how neurons should interact in order to
store information (Hebb, 1949; Sejnowski, 1999). Basi-
cally, activity concurrent in space and time in both pre-
and post-synaptic neuron is required to strengthen the
synapse. The modification of synapses is the mechanism
underlying memory storage and retrieval in both artifi-
cial and biological neural networks (Rolls and Treves,
1998).
The original visionary Hebb formulation is general
enough to exist in many variants of how the pre- and post-
∗
Corresponding author at: Max Planck Institute for the Physics of
Complex Systems, N
¨
othnitzer Str. 38, D-01187 Dresden, Germany.
Tel.: +49 351 871 1221/+420 224 965 901; fax: +420 224 912 834.
E-mail addresses: Julius.Stroffek@lf1.cuni.cz (J. Stroffek),
Eduard.Kuriscak@lf1.cuni.cz (E. Kuriscak), marsalek@cesnet.cz
(P. Marsalek).
synaptic activity are linked. In the case of the feedback
network, the Hebb rule links the forward activity with the
backward, feedback activity. In this case the issue of the
timing within the Hebb rule is especially crucial, because
the feedback strength and the feedback time constant
implying the delay of the feedback are not independent
within the feedback dynamics. One of the first artificial
networks using the Hebb rule is now known as the Hop-
field model. In Hopfield original words the timing issue
of the Hebb rule in his model needed “some appropri-
ate calculation over past history” and indeed in Hopfield
(1982) the Hebb rule and other definitions make it pos-
sible that the network converges using the feedback to a
well defined optimum energy state. This original model
of Hopfield has limitations, however. One particular lim-
itation is the condition for the overall activity of the input
pattern to be in the range around the 50% of activity to
assure the best network functioning. This would require
pattern preprocessing in the case when Hopfield model
is used as an artificial neural network and stores patterns
0303-2647/$ – see front matter © 2006 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.biosystems.2006.04.023