Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neuronal network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS1995

Introduction to Concepts in Artificial Neural Networks

This introduction to artificial neural networks summarizes some basic concepts of computational neuroscience and the resulting models of artificial neurons. The terminology of biological and artificial neurons, biological and machine learning and neural processing is introduced. The concepts of supervised and unsupervised learning are explained with examples from the power system area. Finally, a taxonomy of different types of neurons and different classes of artificial neural networks is presented.

Artificial Neural↗
NASA NTRS1991

Arbitrary nonlinearity is sufficient to represent all functions by neural networks - A theorem

It is proved that if we have neurons implementing arbitrary linear functions and a neuron implementing one (arbitrary but smooth) nonlinear function g(x), then for every continuous function f(x sub 1,..., x sub m) of arbitrarily many variables, and for arbitrary e above 0, we can construct a network that consists of g-neurons and linear neurons, and computes f with precision e.

Kreinovich, Vladik YA.↗
DOE OSTI2024

Collective dynamics and long-range order in thermal neuristor networks

Abstract In the pursuit of scalable and energy-efficient neuromorphic devices, recent research has unveiled a novel category of spiking oscillators, termed “thermal neuristors.” These devices function via thermal interactions among neighboring vanadium dioxide resistive memories, emulating biological neuronal behavior. Here, we show that the collective dynamical behavior of networks of these neurons showcases a rich phase structure, tunable by adjusting the thermal coupling and input voltage. Notably, we identify phases exhibiting long-range order that, however, does not arise from criticality, but rather from the time non-local response of the system. In addition, we show that these thermal neuristor arrays achieve high accuracy in image recognition and time series prediction through reservoir computing, without leveraging long-range order. Our findings highlight a crucial aspect of neuromorphic computing with possible implications on the functioning of the brain: criticality may not be necessary for the efficient performance of neuromorphic systems in certain computational tasks.

Science & Technology - Other Topics↗
NASA NTRS1992

Fault tolerance of artificial neural networks with applications in critical systems

This paper investigates the fault tolerance characteristics of time continuous recurrent artificial neural networks (ANN) that can be used to solve optimization problems. The principle of operations and performance of these networks are first illustrated by using well-known model problems like the traveling salesman problem and the assignment problem. The ANNs are then subjected to 13 simultaneous 'stuck at 1' or 'stuck at 0' faults for network sizes of up to 900 'neurons'. The effects of these faults is demonstrated and the cause for the observed fault tolerance is discussed. An application is presented in which a network performs a critical task for a real-time distributed processing system by generating new task allocations during the reconfiguration of the system. The performance degradation of the ANN under the presence of faults is investigated by large-scale simulations, and the potential benefits of delegating a critical task to a fault tolerant network are discussed.

Protzel, Peter W.↗
NASA NTRS1997

Emulating the Visual Receptive Field Properties of MST Neurons with a Template Model of Heading Estimation

We have previously proposed a computational neural-network model by which the complex patterns of retinal image motion generated during locomotion (optic flow) can be processed by specialized detectors acting as templates for specific instances of self-motion. The detectors in this template model respond to global optic flow by sampling image motion over a large portion of the visual field through networks of local motion sensors with properties similar to neurons found in the middle temporal (MT) area of primate extrastriate visual cortex. The model detectors were designed to extract self-translation (heading), self-rotation, as well as the scene layout (relative distances) ahead of a moving observer, and are arranged in cortical-like heading maps to perform this function. Heading estimation from optic flow has been postulated by some to be implemented within the medial superior temporal (MST) area. Others have questioned whether MST neurons can fulfill this role because some of their receptive-field properties appear inconsistent with a role in heading estimation. To resolve this issue, we systematically compared MST single-unit responses with the outputs of model detectors under matched stimulus conditions. We found that the basic physiological properties of MST neurons can be explained by the template model. We conclude that MST neurons are well suited to support heading estimation and that the template model provides an explicit set of testable hypotheses which can guide future exploration of MST and adjacent areas within the primate superior temporal sulcus.

Perrone, John A.↗
DOE OSTI2022

Reconfigurable perovskite nickelate electronics for artificial intelligence

Reconfigurable devices offer the ability to program electronic circuits on demand. Here, in this work, we demonstrated on-demand creation of artificial neurons, synapses, and memory capacitors in post-fabricated perovskite NdNiO 3 devices that can be simply reconfigured for a specific purpose by single-shot electric pulses. The sensitivity of electronic properties of perovskite nickelates to the local distribution of hydrogen ions enabled these results. With experimental data from our memory capacitors, simulation results of a reservoir computing framework showed excellent performance for tasks such as digit recognition and classification of electrocardiogram heartbeat activity. Using our reconfigurable artificial neurons and synapses, simulated dynamic networks outperformed static networks for incremental learning scenarios. The ability to fashion the building blocks of brain-inspired computers on demand opens up new directions in adaptive networks.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
NASA NTRS1989

Intelligent Network Management and Functional Cerebellum Synthesis

Transdisciplinary modeling of the cerebellum across histology, physiology, and network engineering provides preliminary results at three organization levels: input/output links to central nervous system networks; links between the six neuron populations in the cerebellum; and computation among the neurons of the populations. Older models probably underestimated the importance and role of climbing fiber input which seems to supply write as well as read signals, not just to Purkinje but also to basket and stellate neurons. The well-known mossy fiber-granule cell-Golgi cell system should also respond to inputs originating from climbing fibers. Corticonuclear microcomplexing might be aided by stellate and basket computation and associate processing. Technological and scientific implications of the proposed cerebellum model are discussed.

Loebner, Egon E.↗
DOE OSTI2024

Scaling neural simulations in STACS

Abstract As modern neuroscience tools acquire more details about the brain, the need to move towards biological-scale neural simulations continues to grow. However, effective simulations at scale remain a challenge. Beyond just the tooling required to enable parallel execution, there is also the unique structure of the synaptic interconnectivity, which is globally sparse but has relatively high connection density and non-local interactions per neuron. There are also various practicalities to consider in high performance computing applications, such as the need for serializing neural networks to support potentially long-running simulations that require checkpoint-restart. Although acceleration on neuromorphic hardware is also a possibility, development in this space can be difficult as hardware support tends to vary between platforms and software support for larger scale models also tends to be limited. In this paper, we focus our attention on Simulation Tool for Asynchronous Cortical Streams (STACS), a spiking neural network simulator that leverages the Charm++ parallel programming framework, with the goal of supporting biological-scale simulations as well as interoperability between platforms. Central to these goals is the implementation of scalable data structures suitable for efficiently distributing a network across parallel partitions. Here, we discuss a straightforward extension of a parallel data format with a history of use in graph partitioners, which also serves as a portable intermediate representation for different neuromorphic backends. We perform scaling studies on the Summit supercomputer, examining the capabilities of STACS in terms of network build and storage, partitioning, and execution. We highlight how a suitably partitioned, spatially dependent synaptic structure introduces a communication workload well-suited to the multicast communication supported by Charm++. We evaluate the strong and weak scaling behavior for networks on the order of millions of neurons and billions of synapses, and show that STACS achieves competitive levels of parallel efficiency.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1993

Cascaded VLSI Chips Help Neural Network To Learn

Cascading provides 12-bit resolution needed for learning. Using conventional silicon chip fabrication technology of VLSI, fully connected architecture consisting of 32 wide-range, variable gain, sigmoidal neurons along one diagonal and 7-bit resolution, electrically programmable, synaptic 32 x 31 weight matrix implemented on neuron-synapse chip. To increase weight nominally from 7 to 13 bits, synapses on chip individually cascaded with respective synapses on another 32 x 32 matrix chip with 7-bit resolution synapses only (without neurons). Cascade correlation algorithm varies number of layers effectively connected into network; adds hidden layers one at a time during learning process in such way as to optimize overall number of neurons and complexity and configuration of network.

Duong, Tuan A.↗
NASA NTRS2000

Neural Network and Response Surface Methodology for Rocket Engine Component Optimization

The goal of this work is to compare the performance of response surface methodology (RSM) and two types of neural networks (NN) to aid preliminary design of two rocket engine components. A data set of 45 training points and 20 test points obtained from a semi-empirical model based on three design variables is used for a shear coaxial injector element. Data for supersonic turbine design is based on six design variables, 76 training, data and 18 test data obtained from simplified aerodynamic analysis. Several RS and NN are first constructed using the training data. The test data are then employed to select the best RS or NN. Quadratic and cubic response surfaces. radial basis neural network (RBNN) and back-propagation neural network (BPNN) are compared. Two-layered RBNN are generated using two different training algorithms, namely solverbe and solverb. A two layered BPNN is generated with Tan-Sigmoid transfer function. Various issues related to the training of the neural networks are addressed including number of neurons, error goals, spread constants and the accuracy of different models in representing the design space. A search for the optimum design is carried out using a standard gradient-based optimization algorithm over the response surfaces represented by the polynomials and trained neural networks. Usually a cubic polynominal performs better than the quadratic polynomial but exceptions have been noticed. Among the NN choices, the RBNN designed using solverb yields more consistent performance for both engine components considered. The training of RBNN is easier as it requires linear regression. This coupled with the consistency in performance promise the possibility of it being used as an optimization strategy for engineering design problems.

Vaidyanathan, Rajkumar↗
NASA NTRS2004

Algorithm for Training a Recurrent Multilayer Perceptron

An improved algorithm has been devised for training a recurrent multilayer perceptron (RMLP) for optimal performance in predicting the behavior of a complex, dynamic, and noisy system multiple time steps into the future. [An RMLP is a computational neural network with self-feedback and cross-talk (both delayed by one time step) among neurons in hidden layers]. Like other neural-network-training algorithms, this algorithm adjusts network biases and synaptic-connection weights according to a gradient-descent rule. The distinguishing feature of this algorithm is a combination of global feedback (the use of predictions as well as the current output value in computing the gradient at each time step) and recursiveness. The recursive aspect of the algorithm lies in the inclusion of the gradient of predictions at each time step with respect to the predictions at the preceding time step; this recursion enables the RMLP to learn the dynamics. It has been conjectured that carrying the recursion to even earlier time steps would enable the RMLP to represent a noisier, more complex system.

Parlos, Alexander G.↗
NASA NTRS1998

Emulating the visual receptive-field properties of MST neurons with a template model of heading estimation

We have proposed previously a computational neural-network model by which the complex patterns of retinal image motion generated during locomotion (optic flow) can be processed by specialized detectors acting as templates for specific instances of self-motion. The detectors in this template model respond to global optic flow by sampling image motion over a large portion of the visual field through networks of local motion sensors with properties similar to those of neurons found in the middle temporal (MT) area of primate extrastriate visual cortex. These detectors, arranged within cortical-like maps, were designed to extract self-translation (heading) and self-rotation, as well as the scene layout (relative distances) ahead of a moving observer. We then postulated that heading from optic flow is directly encoded by individual neurons acting as heading detectors within the medial superior temporal (MST) area. Others have questioned whether individual MST neurons can perform this function because some of their receptive-field properties seem inconsistent with this role. To resolve this issue, we systematically compared MST responses with those of detectors from two different configurations of the model under matched stimulus conditions. We found that the characteristic physiological properties of MST neurons can be explained by the template model. We conclude that MST neurons are well suited to support self-motion estimation via a direct encoding of heading and that the template model provides an explicit set of testable hypotheses that can guide future exploration of MST and adjacent areas within the superior temporal sulcus.

NASA Discipline Neuroscience↗
DOE OSTI2023

A unified and constructive framework for the universality of neural networks

Abstract One of the reasons why many neural networks are capable of replicating complicated tasks or functions is their universal approximation property. Though the past few decades have seen tremendous advances in theories of neural networks, a single constructive and elementary framework for neural network universality remains unavailable. This paper is an effort to provide a unified and constructive framework for the universality of a large class of activation functions including most of the existing ones. At the heart of the framework is the concept of neural network approximate identity (nAI). The main result is as follows: any nAI activation function is universal in the space of continuous functions on compacta. It turns out that most of the existing activation functions are nAI, and thus universal. The framework induces several advantages over the contemporary counterparts. First, it is constructive with elementary means from functional analysis, probability theory, and numerical analysis. Second, it is one of the first unified and constructive attempts that is valid for most of the existing activation functions. Third, it provides new proofs for most activation functions. Fourth, for a given activation and error tolerance, the framework provides precisely the architecture of the corresponding one-hidden neural network with a predetermined number of neurons and the values of weights/biases. Fifth, the framework allows us to abstractly present the first universal approximation with a favorable non-asymptotic rate. Sixth, our framework also provides insights into the developments, and hence providing constructive derivations, of some of the existing approaches.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Switching Dynamics in Vanadium Dioxide-Based Stochastic Thermal Neurons

We report on switching dynamics of individual and coupled vanadium dioxide (VO 2 ) devices subject to voltage pulses as the temperature is systematically varied from room temperature spanning the insulator–metal transition (IMT) temperature. The switching voltage of single devices has a strong relationship with both temperature and voltage pulsewidth. Two-step switching in connected VO 2 devices has been noted in current transient plots and was found to depend on temperature, pulsewidth, and pulse amplitude. Experimental switching behavior measured from VO 2 artificial neurons was implemented into a spiking neural network (SNN). During training, modulating the switching voltage via temperature affords a novel method to implement homeostasis with the coupled devices. Simulation results show the efficacy of the stochastic neuronal characteristics and the proposed homeostasis mechanism on a standard digit recognition task. As a result, these studies contribute to ongoing efforts in neuromorphic computing exploiting collective phase transitions.

42 ENGINEERING↗
DOE OSTI2024

Collective neural network behavior in a dynamically driven disordered system of superconducting loops

Collective properties of complex systems composed of many interacting components such as neurons in our brain can be modeled by artificial networks based on disordered systems. We show that a disordered neural network of superconducting loops with Josephson junctions can exhibit computational properties like categorization and associative memory in the time evolution of its state in response to information from external excitations. Superconducting loops can trap multiples of fluxons in many discrete memory configurations defined by the local free energy minima in the configuration space of all possible states. A memory state can be updated by exciting the Josephson junctions to fire or allow the movement of fluxons through the network as the current through them surpasses their critical current thresholds. Simulations performed with a lumped element circuit model of a 4-loop network show that information written through excitations is translated into stable states of trapped flux and their time evolution. Experimental implementation on a high-Tc superconductor YBCO-based 4-loop network shows dynamically stable flux flow in each pathway characterized by the correlations between junction firing statistics. Neural network behavior is observed as energy barriers separating state categories in simulations in response to multiple excitations, and experimentally as junction responses characterizing different flux flow patterns in the network. The state categories that produce these patterns have different temporal stabilities relative to each other and the excitations. This provides strong evidence for time-dependent (short-to-long-term) memories, that are dependent on the geometrical and junction parameters of the loops, as described with a network model.

Josephson junctions↗
DOE OSTI2023

Compensation of Anisotropy in Spin Hall Devices for Neuromorphic Applications

Spintronic nano-oscillators and diodes with reduced nonlinearity benefit from low phase noise and improved device properties. Moreover, they could offer key advantages for realizing neuromorphic applications, such as spike-based neurons and frequency multiplexing in neural networks. Here, in this work, we experimentally demonstrate the reduction in nonlinearity of a spin Hall nano-oscillator (SHNO) by compensation of its effective magnetic anisotropy. The study involves optimization of Co/Ni multilayer growth to achieve the compensation, followed by spin-diode measurements on patterned microstrips to quantify their anisotropy. The relationship between the second- (H k 2 = 0.47 mT) and first-order ($H^{eff}_{k_1}$ = -0.8 mT) anisotropy fields reveals the existence of an easy cone, thereby validating the presence of compensation. Furthermore, we demonstrate a compensated spin diode that has a fixed frequency when the input power is varied. We then study the current-induced auto-oscillation properties of SHNOs on compensated films by patterning nanoconstrictions of 200 and 100 nm wide. The invariance of the resonance frequency and linewidth of the compensated SHNO with applied dc current indicates the absence of nonlinearity. This independence is maintained irrespective of the applied external fields and their orientations. The compensated SHNO obtained has a linewidth of 1.1 MHz and a peak output power of up to 1 pW/MHz.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Real-Time and Adaptive Reservoir Computing With Application to Profile Prediction in Fusion Plasma

Nuclear fusion is a promising alternative to address the problem of sustainable energy production. The tokamak is an approach to fusion based on magnetic plasma confinement, constituting a complex physical system with many control challenges. Here we study the characteristics and optimization of reservoir computing (RC) for real-time and adaptive prediction of plasma profiles in the DIII-D tokamak. Our experiments demonstrate that RC achieves comparable results to state-of-the-art (deep) convolutional neural networks (CNNs) and long short-term memory (LSTM) models, with a significantly easier and faster training procedure. This efficient approach allows for fast and frequent adaptation of the model to new situations, such as changing plasma conditions or different fusion devices.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Towards a more general understanding of the algorithmic utility of recurrent connections

Lateral and recurrent connections are ubiquitous in biological neural circuits. Yet while the strong computational abilities of feedforward networks have been extensively studied, our understanding of the role and advantages of recurrent computations that might explain their prevalence remains an important open challenge. Foundational studies by Minsky and Roelfsema argued that computations that require propagation of global information for local computation to take place would particularly benefit from the sequential, parallel nature of processing in recurrent networks. Such “tag propagation” algorithms perform repeated, local propagation of information and were originally introduced in the context of detecting connectedness, a task that is challenging for feedforward networks. Here, we advance the understanding of the utility of lateral and recurrent computation by first performing a large-scale empirical study of neural architectures for the computation of connectedness to explore feedforward solutions more fully and establish robustly the importance of recurrent architectures. In addition, we highlight a tradeoff between computation time and performance and construct hybrid feedforward/recurrent models that perform well even in the presence of varying computational time limitations. We then generalize tag propagation architectures to propagating multiple interacting tags and demonstrate that these are efficient computational substrates for more general computations of connectedness by introducing and solving an abstracted biologically inspired decision-making task. Our work thus clarifies and expands the set of computational tasks that can be solved efficiently by recurrent computation, yielding hypotheses for structure in population activity that may be present in such tasks.

59 BASIC BIOLOGICAL SCIENCES↗