Venkatesh, Santosh S. | Engineering Papers

NASA NTRS1993

Random interactions in higher order neural networks

Recurrent networks of polynomial threshold elements with random symmetric interactions are studied. Precise asymptotic estimates are derived for the expected number of fixed points as a function of the margin of stability. In particular, it is shown that there is a critical range of margins of stability (depending on the degree of polynomial interaction) such that the expected number of fixed points with margins below the critical range grows exponentially with the number of nodes in the network, while the expected number of fixed points with margins above the critical range decreases exponentially with the number of nodes in the network. The random energy model is also briefly examined and links with higher order neural networks and higher order spin glass models made explicit.

Baldi, Pierre↗

NASA NTRS1987

The capacity of the Hopfield associative memory

Techniques from coding theory are applied to study rigorously the capacity of the Hopfield associative memory. Such a memory stores n-tuple of + or - 1s. The components change depending on a hard-limited version of linear functions of all other components. With symmetric connections between components, a stable state is ultimately reached. By building up the connection matrix as a sum-of-outer products of m fundamental memories, it may be possible to recover a certain one of the m memories by using an initial n-tuple probe vector less than a Hamming distance n/2 away from the fundamental memory. If m fundamental memories are chosen at random, the maximum asymptotic value of m in order that most of the m original memories are exactly recoverable is n/(2 log n). With the added restriction that every one of the m fundamental memories be recoverable exactly, m can be no more than n/(4 log n) asymptotically as n approaches infinity. Extensions are also considered, in particular to capacity under quantization of the outer-product connection matrix. This quantized memory-capacity problem is closely related to the capacity of the quantized Gaussian channel.

Mceliece, Robert J.↗