Access the full text.
Sign up today, get DeepDyve free for 14 days.
R. Prager, F. Fallside (1989)
The modified Kanerva model for automatic speech recognitionComputer Speech & Language, 3
Xiaoyan Mu, Mehmet Artiklar, P. Watta, M. Hassoun (2003)
An RCE-based Associative Memory with Application to Human Face RecognitionNeural Processing Letters, 23
A. I, Y. L-L....-I (1990)
Associative memory : on the ( puzzling ) sparse coding limit
A. Papoulis (1965)
Probability, Random Variables and Stochastic Processes
D. Golomb, N. Rubin, H. Sompolinsky (1990)
Willshaw model: Associative memory with sparse coding and low firing rates.Physical review. A, Atomic, molecular, and optical physics, 41 4
A. Knoblauch (2003)
Optimal Matrix Compression Yields Storage Capacity 1 for Binary Willshaw Associative Memory
D. Willshaw, O. Buneman, H. Longuet-Higgens (1969)
Non-Holographic Associative MemoryNature, 222
G. Palm (1987)
On Associative Memories
Jin Shin, Sang Kim (2006)
A Mathematical Theory of Communication
J. Sacramento, A. Wichert (2012)
Binary Willshaw learning yields high synaptic capacity for long-term familiarity memoryBiological Cybernetics, 106
H. Bentz, Michael Hagstroem, G. Palm (1989)
Information storage and effective data retrieval in sparse matricesNeural Networks, 2
W. Little (1974)
The existence of persistent states in the brainBellman Prize in Mathematical Biosciences, 19
G. Palm (1988)
Local Synaptic Rules with Maximal Information Storage Capacity
Panayiota Poirazi, Bartlett Mel (2001)
Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural TissueNeuron, 29
A. Lansner, Örjan Ekeberg (1989)
A One-Layer Feedback Artificial Neural Network with a Bayesian Learning RuleInt. J. Neural Syst., 1
A. Gardner-Medwin (1976)
The recall of events through the learning of associations between their partsProceedings of the Royal Society of London. Series B. Biological Sciences, 194
T. Sejnowski (1977)
Storing covariance with nonlinearly interacting neuronsJournal of Mathematical Biology, 4
A. Lansner (2009)
Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulationsTrends in Neurosciences, 32
G. Palm (1988)
On the asymptotic information storage capacity of neural networks
J. Buckingham, D. Willshaw (1993)
On setting unit thresholds in an incompletely connected associative net Network: ComputInternational Journal of Neural Systems
Gal Chechik, I. Meilijson, E. Ruppin (2001)
Effective Neuronal Learning with Ineffective Hebbian Learning RulesNeural Computation, 13
Christopher Johansson, A. Sandberg, A. Lansner (2002)
A Neural Network with Hypercolumns
J. Albus (1971)
A Theory of Cerebellar FunctionBellman Prize in Mathematical Biosciences, 10
M. Abramowitz, I. Stegun, David Miller (1965)
Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55)Journal of Applied Mechanics, 32
S. Levin (1983)
Lectu re Notes in Biomathematics
T. Kohonen (1972)
Correlation Matrix MemoriesIEEE Transactions on Computers, C-21
S. Amari (1989)
Characteristics of sparsely encoded associative memoryNeural Networks, 2
A. Knoblauch (2009)
THE ROLE OF STRUCTURAL PLASTICITY AND SYNAPTIC CONSOLIDATION FOR MEMORY AND AMNESIA IN A MODEL OF CORTICO-HIPPOCAMPAL INTERPLAY
B. Graham, D. Willshaw (1995)
Improving recall from an associative memoryBiological Cybernetics, 72
J. Knott (1951)
The organization of behavior: A neuropsychological theoryElectroencephalography and Clinical Neurophysiology, 3
Qi Han, X. Liao, Tingwen Huang, Jun Peng, Chuandong Li, Hongyu Huang (2012)
Analysis and design of associative memories based on stability of cellular neural networksNeurocomputing, 97
D. Marr (1969)
A theory of cerebellar cortexThe Journal of Physiology, 202
E. Rolls (1996)
A theory of hippocampal function in memoryHippocampus, 6
Stefano Fusi, P. Drew, L. Abbott (2005)
Cascade Models of Synaptically Stored MemoriesNeuron, 45
D. Willshaw, P. Dayan (1990)
Optimal Plasticity from Matrix Memories: What Goes Up Must Come DownNeural Computation, 2
E. Gardner (1988)
The space of interactions in neural network modelsJournal of Physics A, 21
T. Sejnowski (1977)
Statistical constraints on synaptic plasticity.Journal of theoretical biology, 69 2
T. Cover, Joy Thomas (1991)
Elements of Information Theory
T. Kohonen, E. Oja (1976)
Fast adaptive formation of orthogonalizing filters and associative memory in recurrent networks of neuron-like elementsBiological Cybernetics, 21
G. Shepherd (1979)
The Synaptic Organization of the Brain
R. Sandberg, A. Lansner, K. Petersson
A Palimpsest Memory based on anIn remental Bayesian Learning
R. Henkel, M. Opper (1990)
Distribution of internal fields and dynamics of neural networksEPL, 11
T. Kohonen (1977)
Associative memory. A system-theoretical approachActa Biotheoretica, 28
H. Markram, Maria Toledo-Rodriguez, Yun Wang, Anirudh Gupta, G. Silberberg, Caizhi Wu (2004)
Interneurons of the neocortical inhibitory systemNature Reviews Neuroscience, 5
J. Hopfield (1982)
Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences of the United States of America, 79 8
S. Amari (1977)
Neural theory of association and concept-formationBiological Cybernetics, 26
(1991)
Memory capacities of local rules for synaptic modification: A comparative review
P. Lennie (2003)
The Cost of Cortical ComputationCurrent Biology, 13
A. Lansner, A. Holst (1996)
A Higher order Bayesian Neural Network with Spiking UnitsInternational journal of neural systems, 7 2
G. Laurent (2002)
Olfactory network dynamics and the coding of multidimensional signalsNature Reviews Neuroscience, 3
J.A. Anderson, J. Silverstein, Stephen Ritz, Randall Jones (1977)
Distinctive features, categorical perception, and probability learning: some applications of a neural model
Christopher Johansson, A. Sandberg, A. Lansner (2002)
Attractor Neural Networks with Hypercolumns
C. Shannon (1948)
A mathematical theory of communicationBell Syst. Tech. J., 27
K. Steinbuch (2004)
Die LernmatrixKybernetik, 1
F. Pulvermüller (2003)
The Neuroscience of Language: On Brain Circuits of Words and Serial Order
A. Knoblauch, G. Palm (2001)
Pattern separation and synchronization in spiking associative memories and visual areasNeural networks : the official journal of the International Neural Network Society, 14 6-7
Martin Rehn, Friedrich Sommer (2006)
Storing and restoring visual input with collaborative rank coding and associative memoryNeurocomputing, 69
P. Dayan, D. Willshaw (1991)
Optimising synaptic learning rules in linear associative memoriesBiological Cybernetics, 65
A. Sandberg, A. Lansner, K. Petersson, Örjan Ekeberg (2000)
A palimpsest memory based on an incremental Bayesian learning ruleNeurocomputing, 32-33
S. Waydo, A. Kraskov, R. Quiroga, I. Fried, C. Koch (2006)
Sparse Representation in the Human Medial Temporal LobeThe Journal of Neuroscience, 26
Günther Palm (2004)
On associative memoryBiological Cybernetics, 36
A. Wichert (2006)
Cell assemblies for diagnostic problem-solvingNeurocomputing, 69
Jan Bures, Irwin Kopin, Bruce McEwen, Karl Pribram, Jay Rosenblatt, Lawrence Weiskranz, Maryse Lassonde, Toshiharu Nagatsu, Toshitaka Nabeshima, Richard McCarty, B. Nicholson, R. Faull (2004)
The Basal Ganglia
F. Sommer, P. Dayan (1998)
Bayesian retrieval in associative memories with storage errorsIEEE transactions on neural networks, 9 4
R. Bogacz, Malcolm Brown, C. Giraud-Carrier (2004)
Model of Familiarity Discrimination in the Perirhinal CortexJournal of Computational Neuroscience, 10
(2007)
On the computational benefits of inhibitory neural associative networks (HRI-EU Rep
E. Fransén, A. Lansner (1998)
A model of cortical associative memory based on a horizontal network of connected columns.Network, 9 2
E. Benedetti, M. Budinich (2011)
Neural RelaxNeural Computation, 24
D. Owen (1965)
Handbook of Mathematical Functions with FormulasTechnometrics, 7
J. Borwein (2004)
Mathematics by experiment : plausible reasoning in the 21st century
D. Sterratt, D. Willshaw (2008)
Inhomogeneities in Heteroassociative Memories with Linear Learning RulesNeural Computation, 20
A. Knoblauch (2008)
Neural Associative Memory and the Willshaw--Palm Probability DistributionSIAM J. Appl. Math., 69
D. Greene, Michal Parnas, F. Yao (1994)
Multi-index hashing for information retrievalProceedings 35th Annual Symposium on Foundations of Computer Science
A. Knoblauch (2005)
Neural associative memory for brain modeling and information retrievalInf. Process. Lett., 95
(1987)
An associative network solving the “4-bit adder problem.
(2010)
Comparison of the Lansner/Ekeberg rule to optimal Bayesian learning in neural associative memory (HRI-EU Rep. 10-06)
G. Palm (1982)
Neural Assemblies: An Alternative Approach to Artificial Intelligence
J. Buckingham, D. Willshaw (1992)
Performance characteristics of the associative netNetwork: Computation In Neural Systems, 3
Christopher Johansson, A. Lansner (2007)
Imposing Biological Constraints onto an Abstract Neocortical Attractor Network ModelNeural Computation, 19
J. Anderson (1968)
A memory storage model utilizing spatial correlation functionsKybernetik, 5
C. Huyck, V. Moreira (2005)
Information Retrieval and Categorisation using a Cell Assembly NetworkNeural Computing & Applications, 14
Charles Bishop (1997)
Bayesian Neural NetworksJ. Braz. Comput. Soc., 4
(1990)
Cell assemblies as a guideline for brain research
G. Palm, F. Sommer (1992)
Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory statesNetwork: Computation In Neural Systems, 3
Harry Zhang (2004)
The Optimality of Naive Bayes
Diogo Rendeiro, J. Sacramento, A. Wichert (2014)
Taxonomical Associative MemoryCognitive Computation, 6
Ingrid Welie, J. Hooft, W. Wadman (2004)
Homeostatic scaling of neuronal excitability by synaptic modulation of somatic hyperpolarization-activated Ih channels.Proceedings of the National Academy of Sciences of the United States of America, 101 14
P. Kanerva (1988)
Sparse Distributed Memory
J. Proakis (1985)
Probability, random variables and stochastic processesIEEE Trans. Acoust. Speech Signal Process., 33
(2009)
Zip nets: Neural associative networks with non-linear learning (HRI-EU Rep. 09-03)
A. Stepanyants, P. Hof, D. Chklovskii (2002)
Geometry and Structural Plasticity of Synaptic ConnectivityNeuron, 34
A. Knoblauch (2010)
Zip nets: Efficient associative computation with binary synapsesThe 2010 International Joint Conference on Neural Networks (IJCNN)
A. Knoblauch, G. Palm, F. Sommer (2010)
Memory Capacities for Synaptic and Structural PlasticityNeural Computation, 22
(1999)
Mercator series
G. Turrigiano, K. Leslie, Niraj Desai, L. Rutherford, S. Nelson (1998)
Activity-dependent scaling of quantal amplitude in neocortical neuronsNature, 391
P. Dayan, T. Sejnowski (1993)
The Variance of Covariance Rules for Associative Matrix Memories and Reinforcement LearningNeural Computation, 5
M. Tsodyks, M. Feigel'man (1988)
The Enhanced Storage Capacity in Neural Networks with Low Activity LevelEPL, 6
D. Marr (1971)
Simple memory: a theory for archicortex.Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 262 841
D. MacKay (1991)
Maximum Entropy Connections: Neural Networks
I. Kononenko (1991)
Semi-Naive Bayesian Classifier
E. Gardner, B. Derrida (1988)
Optimal storage properties of neural network modelsJournal of Physics A, 21
G. Palm, F. Sommer (1996)
Associative Data Storage and Retrieval in Neural Networks
J. Hertz, A. Krogh, R. Palmer (1994)
Introduction to the theory of neural computation, 1
Pedro Domingos, M. Pazzani (1997)
On the Optimality of the Simple Bayesian Classifier under Zero-One LossMachine Learning, 29
F. Sommer, G. Palm (1999)
Improved bidirectional retrieval of sparse patterns stored by Hebbian learningNeural networks : the official journal of the International Neural Network Society, 12 2
R. Heim, G. Palm (1978)
Theoretical Approaches to Complex Systems
Neural associative memories are perceptron-like single-layer networks with fast synaptic learning typically storing discrete associations between pairs of neural activity patterns. Previous work optimized the memory capacity for various models of synaptic learning: linear Hopfield-type rules, the Willshaw model employing binary synapses, or the BCPNN rule of Lansner and Ekeberg, for example. Here I show that all of these previous models are limit cases of a general optimal model where synaptic learning is determined by probabilistic Bayesian considerations. Asymptotically, for large networks and very sparse neuron activity, the Bayesian model becomes identical to an inhibitory implementation of the Willshaw and BCPNN-type models. For less sparse patterns, the Bayesian model becomes identical to Hopfield-type networks employing the covariance rule. For intermediate sparseness or finite networks, the optimal Bayesian learning rule differs from the previous models and can significantly improve memory performance. I also provide a unified analytical framework to determine memory capacity at a given output noise level that links approaches based on mutual information, Hamming distance, and signal-to-noise ratio.
Neural Computation – MIT Press
Published: Jun 1, 2011
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.