Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neuromorphic computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Dendritic Computing with Multigate Ferroelectric Field-Effect Transistors

Although inspired by neuronal systems in the brain, artificial neural networks generally employ point-neurons, which offer computational complexity far less than that of their biological counterparts. Neurons have dendritic arbors that connect to different sets of synapses and offer local nonlinear accumulation – playing a pivotal role in processing and learning. Inspired by this, we propose a novel neuron design based on a multigate ferroelectric field-effect transistor that mimics dendrites. It leverages ferroelectric nonlinearity for local computations within dendritic branches while utilizing the transistor action to generate the neuronal output. The branched architecture enables smaller crossbar arrays in hardware integration, improving efficiency. Using an experimentally calibrated device-circuit-algorithm cosimulation framework, we demonstrate that networks incorporating our dendritic neurons achieve superior performance compared to much larger networks without dendrites (∼ 17× fewer trainable weight parameters). These findings suggest that dendritic hardware can significantly improve computational efficiency and learning capacity of neuromorphic systems optimized for edge applications.

36 MATERIALS SCIENCE↗

Solving sparse finite element problems on neuromorphic hardware

The finite element method (FEM) is one of the most important and ubiquitous numerical methods for solving partial differential equations (PDEs) on computers for scientific and engineering discovery. Applying the FEM to larger and more detailed scientific models has driven advances in high-performance computing for decades. Here we demonstrate that scalable spiking neuromorphic hardware can directly implement the FEM by constructing a spiking neural network that solves the large, sparse, linear systems of equations at the core of the FEM. We show that for the Poisson equation, a fundamental PDE in science and engineering, our neural circuit achieves meaningful levels of numerical accuracy and close to ideal scaling on modern, inherently parallel and energy-efficient neuromorphic hardware, specifically Intel’s Loihi 2 neuromorphic platform. We illustrate extensions to irregular mesh geometries in both two and three dimensions as well as other PDEs such as linear elasticity. Our spiking neural network is constructed from a recurrent network model of the brain’s motor cortex and, in contrast to black-box deep artificial neural network-based methods for PDEs, directly translates the well-understood and trusted mathematics of the FEM to a natively spiking neuromorphic algorithm.

Applied mathematics↗

Physical Compact Model for Three-Terminal SONOS Synaptic Circuit Element

A well-posed physics-based compact model for a three-terminal silicon–oxide–nitride–oxide–silicon (SONOS) synaptic circuit element is presented for use by neuromorphic circuit/system engineers. Based on technology computer aided design (TCAD) simulations of a SONOS device, the model contains a nonvolatile memristor with the state variable QM representing the memristor charge under the gate of the three-terminal element. By incorporating the exponential dependence of the memristance on QM and the applied bias V for the gate, the compact model agrees quantitatively with the results from TCAD simulations as well as experimental measurements for the drain current. The compact model is implemented through VerilogA in the circuit simulation package Cadence Spectre and reproduces the experimental training behavior for the source–drain conductance of a SONOS device after applying writing pulses ranging from –12 V to +11 V, with an accuracy higher than 90%.

42 ENGINEERING↗

Toward memory-efficient melt pool monitoring: a classification framework using event-based imaging and sparse sensing technique

Vision sensors like CMOS and CCD cameras are often used for in-process monitoring of melt pools in laser-based additive and welding processes, but they require transferring large amounts of data and computational processing resources. Event-based neuromorphic imagery, on the other hand, detects only the change in pixel intensity, thus potentially reducing the data amount and latency. With an event imager, this study develops a framework for melt pool condition classification, including image construction, time scale selection, optimal pixel selection, and sparse classification, to achieve a highly memory-efficient scheme. These are based on sparse sensing techniques with singular value decomposition (SVD) and QR pivoting, the two fundamental matrix transformations for linear dimensionality reduction. The framework is then validated by classifying a controlled experiment by exciting various mode shapes of liquid gallium pools of varying depths (3, 6, and 8 mm). At 200 pixels, the classifier can reach overall accuracy of 75%, while at 2000 pixels (0.013% of the total possible pixels), the accuracy is nearly 90% (89.86%). At the same number of pixels, random selection can only achieve 46% and 67%, respectively. The memory savings of the sparsely sampled event data compared to a conventional imager is about 500 times. In addition to performance, implementation and limitations of the framework are also discussed.

42 ENGINEERING↗

Bypassing Fast Time Scales of the Hodgkin-Huxley Neuron Model via a Thresholded Hard Reset

We propose a modification to the Hodgkin-Huxley model to reduce the numerical stiffness of the equations by introducing an explicit voltage threshold. When this threshold is crossed, the voltage and the gating variables are reset to constant values. It is found that, for all of the current stimuli considered, the proposed model accurately reproduces the behavior of the baseline Hodgkin-Huxley model while bypassing the fast dynamics of spiking. Specifically, the model accurately reproduces the spike times and, between spikes, the time courses of the membrane potential and gating variables.

97 MATHEMATICS AND COMPUTING↗

Reimagining Codesign for Advanced Scientific Computing: Report for the ASCR Workshop on Reimagining Codesign

In March 2021, the U.S. Department of Energy’s Advanced Scientific Computing Research program convened the Workshop on Reimagining Codesign. The workshop, also known as ReCoDe, was organized around discussions on eight topic areas: (1) codesign for traditional high-performance computing workloads; (2) codesign of memory/storage systems; (3) codesign of machine learning, neuromorphic, quantum, and other non-von Neumann accelerators; (4) codesign for edge computing and processing at experimental instruments; (5) codesign for security and privacy; (6) hardware design tools and open-source hardware for high-productivity codesign; (7) tools, software stack, and programming languages for high-productivity codesign; and (8) quantitative tools and data collection for modeling and simulation for codesign. The panels identified four Priority Research Directions from these deliberations: (1) breakthrough computing capabilities with targeted heterogeneity and rapid design; (2) software and applications that embrace radical architecture diversity; (3) engineered security and integrity, from transistors to applications; and (4) design with data-rich processes.

97 MATHEMATICS AND COMPUTING↗

Low Power Hardware-In-The-Loop Neuromorphic Training Accelerator

The training process for spiking neural networks can be very computationally intensive. Approaches such as evolutionary algorithms may require evaluating thousands or millions of candidate solutions. In this work, we propose using neuromorphic cores implemented on a Xilinx Zynq system on chip to accelerate and improve the energy efficiency of the evaluation step of an evolutionary training approach. We demonstrate this can significantly reduce the required energy to evolve a network with some cases showing greater than 10 times improvement as compared to a CPU-only system.

Mitchell, Parker↗

Neuromorphic Processing and Sensing for Interception

Interception of a moving and potentially evading target can be a challenging problem, in particular for conditions in which the target may be moving at high speeds and difficult to detect. We have proposed to merge two Sandia LDRD efforts, the SPARR Spiking/Processing Array (neuromorphic event-driven sensing) and the Dragonfly-Inspired Algorithms for Intercept- Trajectory Planning (neural-inspired algorithms for interception) toward a unified system with direct application to national security. Neuromorphic systems demonstrate the most potential for speed and efficiency gains when communication is event-driven and computations are simple but parallelizable. Accordingly, we anticipate fully realizing potential benefits from a neuromorphic interception system if event-driven sensing is combined with processing and acting also implemented on event-driven (spiking) systems. We have successfully translated a neural-inspired interception algorithm to a neural network architecture for evaluation on neuromorphic hardware. Preliminary implementations of the neural network designed for implementation on the Loihi chip are still too immature for conclusive evaluation, but the results of this effort have demonstrated a viable path for a previously developed dragonfly-inspired interception algorithm to be implemented on neuromorphic hardware.

97 MATHEMATICS AND COMPUTING↗

Energy-Efficient Neuromorphic Architectures for Nuclear Radiation Detection Applications

A comprehensive analysis and simulation of two memristor-based neuromorphic architectures for nuclear radiation detection is presented. Both scalable architectures retrofit a locally competitive algorithm to solve overcomplete sparse approximation problems by harnessing memristor crossbar execution of vector–matrix multiplications. The proposed systems demonstrate excellent accuracy and throughput while consuming minimal energy for radionuclide detection. To ensure that the simulation results of our proposed hardware are realistic, the memristor parameters are chosen from our own fabricated memristor devices. Based on these results, we conclude that memristor-based computing is the preeminent technology for a radiation detection platform.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NeuralRW-Loihi: Spiking Discrete Time Markov Chain Simulator for Intel Loihi v

SAND2024-01302O NeuralRW-Loihi: Spiking Discrete Time Markov Chain Simulator for Intel Loihi is a neuromorphic script that generates a random walk simulation of a discrete-time Markov Chain for the Loihi neuromorphic hardware. The code was used to produce the paper "Neuromorphic scaling advantages for energy-efficient random walk computations" by Smith et al., 2021. It was published in Nature Electronics. The software provides code for implementing the discrete time Markov chain random walks on Intel's Loihi Neuromorphic platform. In addition, it provides an example problem through a mini app.

Aimone, James↗

CRCNS22 Learning Rules in the Hippocampus and their Mapping to Neuromorphic Systems (Final Technical Report)

Large scale biologically-realistic computational models are key to investigating the interplay between structure and function in nervous systems, thus paving the way to new clinical methods and neuro-inspired computing solutions. This project focuses on the hippocampus, in particular the CA3-CA1 regions, due to their role in associative learning and memory, pattern separation and completion, and spatial navigation. Investigations into the neuronal organization and learning rule(s) of this circuit can shed light into how declarative memories are formed, stored, recalled and forgotten and inform computational, experimental and clinical neuroscience work. Our project aims at developing a novel data-driven methodology supported by a broad heterogeneous base of neuroscience experimental knowledge and inspired from advances in computer science and engineering. Specifically, this work will benchmark existing and new learning rules within a full-scale spiking neural network simulation of the CA3-CA1 region. The model will be based on an open-source repository, called the Hippocampome, which contains neuronal morphologies, firing patterns, synapse probabilities, and most other required parameters for all known neuron types in the rodent hippocampal formation. The model will be first trained in a supervised fashion for associative memory tasks using backpropagation through time traditionally used in computer science, enhanced with a new technique called the surrogate gradient method. This optimization method will be used to obtain a global loss minimization, but it is not biologically inspired as it assumes the use of data not locally available to the synapses. However, we propose its use as a benchmarking tool, to compare the training performance of local biologically plausible and hardware-mappable learning rules at scale. New rules or combinations will be proposed and tested as needed, based on the obtained results. Progress in this area will also drive the development of novel hardware-mappable algorithms for continual lifelong learning and categorization of new events from few presented examples. This project goes beyond the existing state-of-the-art by looking at large scale realistic neuronal circuits as networks trainable via global optimization methods such as surrogate gradient descent. The objective function of the brain that supports learning is largely unknown, but it is likely that it operates through local learning rules. Studying network trajectories around local minima as proposed in this work represents a useful strategy for understanding whether a network is training by using a specific (set of) learning rule(s). Starting from a completely untrained network is a challenging test since it is difficult to determine how the learning rule affects the trajectory of the network. This interdisciplinary project will help understand what rule governs learning in these regions or if multiple learning rules are involved. The work will develop a robust methodology to measure if the network is converging to the target solution, oscillating around it, or diverging away.

59 BASIC BIOLOGICAL SCIENCES↗

Atypical phase-change alloy Ga 2 Te 3 : atomic structure, incipient nanotectonic nuclei, and multilevel writing

Emerging brain-inspired computing, including artificial optical synapses, photonic tensor cores, neuromorphic networks, etc., needs phase-change materials (PCMs) of the next generation with lower energy consumption and a wider temperature range for reliable long-term operation. Gallium tellurides with higher melting and crystallization temperatures appear to be promising candidates and enable achieving the necessary requirements. Here, using high energy X-ray diffraction and Raman spectroscopy supported by first-principles simulations, we show that vitreous g-Ga 2 Te 3 films essentially have a tetrahedral local structure and sp 3 hybridization, similar to those in the stable fcc Ga 2 Te 3 polymorph and in contrast to a vast majority of typical PCMs. Nevertheless, optical pump–probe laser experiments revealed high-contrast, fast and reversible multilevel SET-RESET transitions raising a question related to the phase change mechanism. A recently observed nanotectonic compression in bulk glassy Ga–Te alloys seems to be responsible for the PCM performance. Incipient nanotectonic nuclei, reminiscent of monoclinic high-pressure HP-Te II and rhombohedral HP-Ga 2 Te 3 , are present as minorities (2–4%) in g-Ga 2 Te 3 but are suggested to grow dramatically with increasing temperature while interacting with appropriate laser pulses. This leads to co-crystallization of HP-polymorphs amplified by a high internal local pressure reaching 4–8 GPa. The metallic HP-forms provide an increasing optical and electrical contrast, favorable for reliable PCM operations, and higher energy efficiency.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

LCA

The Locally Competitive Algorithm (LCA) is a dynamical sparse solver that uses only local computations, allowing for massively parallel implementations on compatible neuromorphic architectures such as Intel's Loihi research chip. In this invention, we show how unsupervised dictionary learning with spiking LCA can be implemented on GPUs and Intel's Loihi research chip.

Parpart, Gavin↗

An Accurate, Error-Tolerant, and Energy-Efficient Neural Network Inference Engine Based on SONOS Analog Memory

In this work, we demonstrate SONOS (silicon-oxide-nitrideoxide- silicon) analog memory arrays that are optimized for neural network inference. The devices are fabricated in a 40nm process and operated in the subthreshold regime for in-memory matrix multiplication. Subthreshold operation enables low conductances to be implemented with low error, which matches the typical weight distribution of neural networks, which is heavily skewed toward near-zero values. This leads to high accuracy in the presence of programming errors and process variations. We simulate the end-to-end neural network inference accuracy, accounting for the measured programming error, read noise, and retention loss in a fabricated SONOS array. Evaluated on the ImageNet dataset using ResNet50, the accuracy using a SONOS system is within 2.16% of floating-point accuracy without any retraining. The unique error properties and high On/Off ratio of the SONOS device allow scaling to large arrays without bit slicing, and enable an inference architecture that achieves 20 TOPS/W on ResNet50, a >10× gain in energy efficiency over state-of-the-art digital and analog inference accelerators.

97 MATHEMATICS AND COMPUTING↗

Backpropagation-based learning with local derivative approximation and memory replay in biologically plausible neural systems

When learning, the brain modifies individual synaptic connections to reach a desired behavior. Animal and human brains have been shown to be incredibly capable of learning complex and varied functions across a wide variety of tasks. In recent years, artificial neural networks, inspired by human and animal brains, have shown great capabilities in learning a wide variety of difficult tasks. However, artificial neural networks primarily teach themselves through the use of backpropagation, a learning method which has no clear analogue within the brain. Additionally, Artificial Neural Networks primarily use continuous activation functions, which differ significantly from the spiking neuronal behavior present in the brain. In this paper, we discuss and demonstrate a biologically plausible learning method that approximates backpropagation through two techniques on Spiking Neural Networks. First, we show that the local temporal derivatives that are necessary for backpropagation can be approximately recovered through reconstruction using spike timings. Second, we show that through learning during a sleep phase, inspired by neuroscience research into memory replay, the localized parallel feedback path can learn to approximate the derivative through the forward path weight matrix, thus solving the weight transport problem. Lastly, we demonstrate that the combination of these two methods can approach or exceed the accuracy of backpropagation-based methods for a variety of neuromorphic vision tasks while maintaining biological plausibility.

42 ENGINEERING↗

Spatial Interactions in Hydrogenated Perovskite Nickelate Synaptic Networks

A key aspect of how the brain learns and enables decision-making processes is through synaptic interactions. Electrical transmission and communication in a network of synapses are modulated by extracellular fields generated by ionic chemical gradients. Emulating such spatial interactions in synthetic networks can be of potential use for neuromorphic learning and the hardware implementation of artificial intelligence. Here, in this work, we demonstrate that in a network of hydrogen-doped perovskite nickelate devices, electric bias across a single junction can tune the coupling strength between the neighboring cells. Electrical transport measurements and spatially resolved diffraction and nanoprobe X-ray and scanning microwave impedance spectroscopic studies suggest that graded proton distribution in the inhomogeneous medium of hydrogen-doped nickelate film enables this behavior. We further demonstrate signal integration through the coupling of various junctions.

36 MATERIALS SCIENCE↗

Scaling neural simulations in STACS

Abstract As modern neuroscience tools acquire more details about the brain, the need to move towards biological-scale neural simulations continues to grow. However, effective simulations at scale remain a challenge. Beyond just the tooling required to enable parallel execution, there is also the unique structure of the synaptic interconnectivity, which is globally sparse but has relatively high connection density and non-local interactions per neuron. There are also various practicalities to consider in high performance computing applications, such as the need for serializing neural networks to support potentially long-running simulations that require checkpoint-restart. Although acceleration on neuromorphic hardware is also a possibility, development in this space can be difficult as hardware support tends to vary between platforms and software support for larger scale models also tends to be limited. In this paper, we focus our attention on Simulation Tool for Asynchronous Cortical Streams (STACS), a spiking neural network simulator that leverages the Charm++ parallel programming framework, with the goal of supporting biological-scale simulations as well as interoperability between platforms. Central to these goals is the implementation of scalable data structures suitable for efficiently distributing a network across parallel partitions. Here, we discuss a straightforward extension of a parallel data format with a history of use in graph partitioners, which also serves as a portable intermediate representation for different neuromorphic backends. We perform scaling studies on the Summit supercomputer, examining the capabilities of STACS in terms of network build and storage, partitioning, and execution. We highlight how a suitably partitioned, spatially dependent synaptic structure introduces a communication workload well-suited to the multicast communication supported by Charm++. We evaluate the strong and weak scaling behavior for networks on the order of millions of neurons and billions of synapses, and show that STACS achieves competitive levels of parallel efficiency.

59 BASIC BIOLOGICAL SCIENCES↗

Reconfigurable neuromorphic components and algorithms for next-generation artificial intelligence

Digital transistor-based general-purpose hardware (e.g., central processing units) is the dominant solution to support both traditional computing (logic, arithmetic, etc.) as well as modern artificial intelligence. State-of-the-art research has shown feasibility of post-digital physics-based neuromorphic hardware, which is hypothesized to support artificial intelligence algorithms with orders-of-magnitude improved time/energy efficiencies. But such research has not been widely deployed mainly because of such novel hardware’s extreme application-specificity, and the dominance of low-cost general-purpose (but inefficient) digital hardware. To make use of the novel algorithms and the superlative performance of physics-based hardware, we need to identify scientific principles that can enable generality in physics-based hardware. This work resulted in two important broad outcomes – first, we demonstrate fully reconfigurable neuromorphic components, and second, we demonstrate a viable artificial intelligence learning algorithm that can exploit the functioning of neuromorphic hardware. We demonstrate up to five orders of magnitude improvement in energy efficiency compared to the best general-purpose digital hardware.

97 MATHEMATICS AND COMPUTING↗