Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “spiking neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

Studies with spike initiators - Linearization by noise allows continuous signal modulation in neural networks

It is shown that noise can be an important element in the translation of neuronal generator potentials (summed inputs) to neuronal spike trains (outputs), creating or expanding a range of amplitudes over which the spike rate is proportional to the generator potential amplitude. Noise converts the basically nonlinear operation of a spike initiator into a nearly linear modulation process. This linearization effect of noise is examined in a simple intuitive model of a static threshold and in a more realistic computer simulation of spike initiator based on the Hodgkin-Huxley (HH) model. The results are qualitatively similar; in each case larger noise amplitude results in a larger range of nearly linear modulation. The computer simulation of the HH model with noise shows linear and nonlinear features that were earlier observed in spike data obtained from the VIIIth nerve of the bullfrog. This suggests that these features can be explained in terms of spike initiator properties, and it also suggests that the HH model may be useful for representing basic spike initiator properties in vertebrates.

Yu, Xiaolong↗
DOE OSTI2020

GRANT: Ground-Roaming Autonomous Neuromorphic Targeter

In this work we describe the design, implementation, and testing of the first neuromorphic robot capable of obstacle avoidance, grid coverage, and targeting controlled by the second generation Dynamic Adaptive Neural Network Array (DANNA2) digital spiking neuromorphic processor. The simplicity of the DANNA2 processor along with the TENNLab hardware/software co-design framework allows for compact spiking networks that can run efficiently on a small, resource-constrained, platform such as a Xilinx Artix-7 field-programmable gate array. Additionally, we present the dynamic reconfigurability of DANNA2 arrays as a method of realizing complex, multi-objective tasks on hardware that is restricted to relatively small networks.

Ambrose, Jonathan↗
NASA NTRS2001

Epicenter location by analysis for interictal spikes

The MEG recording is a quick and painless process that requires no surgery. This approach has the potential to save time, reduce patient discomfort, and eliminates a painful and potentially dangerous surgical step in the treatment procedure.

epilepsy MEG interictal spikes artificial neural n↗
DOE OSTI2021

An introduction to neuromorphic computing and its potential impact for unattended ground sensors

Neuromorphic computers are hardware systems that mimic the brain’s computational process phenomenology. This is in contrast to neural network accelerators, such as the Google TPU or the Intel Neural Compute Stick, which seek to accelerate the fundamental computation and data flows of neural network models used in the field of machine learning. Neuromorphic computers emulate the integrate and fire neuron dynamics of the brain to achieve a spiking communication architecture for computation. While neural networks are brain-inspired, they drastically oversimplify the brain’s computation model. Neuromorphic architectures are closer to the true computation model of the brain (albeit, still simplified). Neuromorphic computing models herald a 1000x power improvement over conventional CPU architectures. Sandia National Labs is a major contributor to the research community on neuromorphic systems by performing design analysis, evaluation, and algorithm development for neuromorphic computers. Space-based remote sensing development has been a focused target of funding for exploratory research into neuromorphic systems for their potential advantage in that program area; SNL has led some of these efforts. Recently, neuromorphic application evaluation has reached the NA-22 program area. This same exploratory research and algorithm development should penetrate the unattended ground sensor space for SNL’s mission partners and program areas. Neuromorphic computing paradigms offer a distinct advantage for the SWaP-constrained embedded systems of our diverse sponsor-driven program areas.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Compensation of Anisotropy in Spin Hall Devices for Neuromorphic Applications

Spintronic nano-oscillators and diodes with reduced nonlinearity benefit from low phase noise and improved device properties. Moreover, they could offer key advantages for realizing neuromorphic applications, such as spike-based neurons and frequency multiplexing in neural networks. Here, in this work, we experimentally demonstrate the reduction in nonlinearity of a spin Hall nano-oscillator (SHNO) by compensation of its effective magnetic anisotropy. The study involves optimization of Co/Ni multilayer growth to achieve the compensation, followed by spin-diode measurements on patterned microstrips to quantify their anisotropy. The relationship between the second- (H k 2 = 0.47 mT) and first-order ($H^{eff}_{k_1}$ = -0.8 mT) anisotropy fields reveals the existence of an easy cone, thereby validating the presence of compensation. Furthermore, we demonstrate a compensated spin diode that has a fixed frequency when the input power is varied. We then study the current-induced auto-oscillation properties of SHNOs on compensated films by patterning nanoconstrictions of 200 and 100 nm wide. The invariance of the resonance frequency and linewidth of the compensated SHNO with applied dc current indicates the absence of nonlinearity. This independence is maintained irrespective of the applied external fields and their orientations. The compensated SHNO obtained has a linewidth of 1.1 MHz and a peak output power of up to 1 pW/MHz.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Easy-plane spin Hall nano-oscillators as spiking neurons for neuromorphic computing

Here we show analytically using a macrospin approximation that easy-plane spin Hall nano-oscillators excited by a spin current polarized perpendicularly to the easy plane have phase dynamics analogous to that of Josephson junctions. Similarly to Josephson junctions, they can reproduce the spiking behavior of biological neurons that is appropriate for neuromorphic computing. To take advantage of typical spin-orbit torques, we use a nanoconstriction geometry, in which the magnetostatic interaction and magnetocrystalline anisotropy are tuned to create an easy plane that includes the interface normal direction. We perform micromagnetic simulations of such oscillators realized in this geometry and show that the easy-plane spiking dynamics is preserved in this experimentally feasible architecture. Finally we simulate two elementary neural network blocks that implement operations essential for neuromorphic computing. First, we show that output spikes energies from two neurons can be summed and injected into a following layer neuron and second, we demonstrate that outputs can be multiplied by synaptic weights implemented by locally modifying the anisotropy.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Reconfigurable Cascaded Thermal Neuristors for Neuromorphic Computing

While the complementary metal-oxide semiconductor (CMOS) technology is the mainstream for the hardware implementation of neural networks, an alternative route is explored based on a new class of spiking oscillators called “thermal neuristors”, which operate and interact solely via thermal processes. Utilizing the insulator-to-metal transition (IMT) in vanadium dioxide, a wide variety of reconfigurable electrical dynamics mirroring biological neurons is demonstrated. Notably, inhibitory functionality is achieved just in a single oxide device, and cascaded information flow is realized exclusively through thermal interactions. To elucidate the underlying mechanisms of the neuristors, a detailed theoretical model is developed, which accurately reflects the experimental results. In conclusion, this study establishes the foundation for scalable and energy-efficient thermal neural networks, fostering progress in brain-inspired computing.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Aftershock Identification Using a Paired Neural Network Applied to Constructed Data

This report is intended to detail the findings of our investigation of the applicability of machine learning to the task of aftershock identification. The ability to automatically identify nuisance aftershock events to reduce analyst workload when searching for events of interest is an important step in improving nuclear monitoring capabilities and while waveform cross - correlation methods have proven successful, they have limitations (e.g., difficulties with spike artifacts, multiple aftershocks in the same window) that machine learning may be able to overcome. Here we apply a Paired Neural Network (PNN) to a dataset consisting of real, high quality signals added to real seismic noises in order to work with controlled, labeled data and establish a baseline of the PNN's capability to identify aftershocks. We compare to waveform cross - correlation and find that the PNN performs well, outperforming waveform cross - correlation when classifying similar waveform pairs, i.e., aftershocks.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Accurate and Accelerated Neuromorphic Network Design Leveraging A Bayesian Hyperparameter Pareto Optimization Approach

Neuromorphic systems allow for extremely efficient hardware implementations for neural networks (NNs). In recent years, several algorithms have been presented to train spiking NNs (SNNs) for neuromorphic hardware. However, SNNs often provide lower accuracy than their artificial NNs (ANNs) counterparts or require computationally expensive and slow training/inference methods. To close this gap, designers typically rely on reconfiguring SNNs through adjustments in the neuron/synapse model or training algorithm itself. Nevertheless, these steps incur significant design time, while still lacking the desired improvement in terms of training/inference times (latency). Designing SNNs that can mimic the accuracy of ANNs with reasonable training times is an exigent challenge in neuromorphic computing. In this work, we present an alternative approach that looks at such designs as an optimization problem rather than algorithm or architecture redesign. We develop a versatile multiobjective hyperparameter optimization (HPO) for automatically tuning HPs of two state-of-the-art SNN training algorithms, SLAYER and HYBRID. We emphasize that, to the best of our knowledge, this is the first work trying to improve SNNs’ computational efficiency, accuracy, and training time using an efficient HPO. We demonstrate significant performance improvements for SNNs on several datasets without the need to redesign or invent new training algorithms/architectures. Our approach results in more accurate networks with lower latency and, in turn, higher energy efficiency than previous implementations. In particular, we demonstrate improvement in accuracy and more than 5× reduction in the training/inference time for the SLAYER algorithm on the DVS Gesture dataset. In the case of HYBRID, we demonstrate 30% reduction in timesteps while surpassing the accuracy of the state-of-the-art networks on CIFAR10. Further, our analysis suggests that even a seemingly minor change in HPs could change the accuracy by 5 - 6×.

Parsa, Maryam↗
DOE OSTI2024

Z-Target Radiography Postprocessing With A Deep Convolution Neural Network

Analyzing X-ray radiographs is crucial for understanding target behavior in Inertial Confinement Fusion (ICF) and High Energy Density (HED) platforms. However, the density of Magneto Raleigh Taylor (MRT) bands and limitations of target materials often obscure relevant spike growth and density information. To address this issue, machine learning postprocessing techniques can be applied to remove darkened regions in radiography images. In this study, a novel method is presented for removing MRT darkened regions from z-target radiographs using a convolutional neural network (CNN). The CNN, consisting of six layers, treats the darkened regions as noise and employs a mixed loss function and end-to-end frameworks to suppress them while preserving sharpness. The six-layer architecture is designed to effectively learn features when provided with a larger volume of learning space. Each layer is optimized using a mixed loss function that combines a standard loss pixel approach with a multi-scaled structural similarity index loss, which considers luminance, contrast, and structure in local neighborhoods. This approach is particularly beneficial for capturing the stochastic structure of MRT limbs. Due to the limited availability of experimental data, training is conducted using synthetic target radiography from 3D Alegra simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1992

SPIKE: Application for ASTRO-D mission planning

SPIKE is a mission planning software system developed by a team of programmers at the STScI for use with the Hubble Space Telescope (HST). SPIKE has been developed for the purpose of automating observatory scheduling to increase the effective utilization and ultimately, scientific return from orbiting telescopes. High-level scheduling strategies using both rule-based and neural network approaches have been incorporated. Graphical displays of activities, constraints, and schedules are an important feature of the system. Although SPIKE was originally developed for the HST, it can be used for other astronomy missions including ground-based observatories. One of the missions that has decided to use SPIKE is ASTRO-D, a Japanese X-ray satellite for which the U.S. is providing a part of the scientific payload. Scheduled to fly in Feb. 1993, its four telescopes will focus X-rays over a wide energy range onto CCD's and imaging gas proportional counters. ASTRO-D will be the first X-ray imaging mission operating over the 0.5-12 keV band with high energy resolution. This combination of capabilities will enable a varied and exciting program of astronomical research to be carried out. ASTRO-D is expected to observe 5 to 20 objects per day and a total of several thousands per year. This requires the implementation of an efficient planning and scheduling system which SPIKE can provide. Although the version of SPIKE that will be used for ASTRO-D mission is almost identical to that used for the HST, there are a few differences. For example, ASTRO-D will use two ground stations for data downlinks, instead of the TDRSS system for data transmission. As a consequence ASTRO-D is constrained by limited on-board data storage capacity to schedule high data-rate observations during periods of frequent high bit rate observations accordingly. We will demonstrate the ASTRO-D version of SPIKE to show what SPIKE can provide and how efficiently it creates an observational schedule.

Isobe, T.↗
DOE OSTI2020

Neuromorphic Processing and Sensing for Interception

Interception of a moving and potentially evading target can be a challenging problem, in particular for conditions in which the target may be moving at high speeds and difficult to detect. We have proposed to merge two Sandia LDRD efforts, the SPARR Spiking/Processing Array (neuromorphic event-driven sensing) and the Dragonfly-Inspired Algorithms for Intercept- Trajectory Planning (neural-inspired algorithms for interception) toward a unified system with direct application to national security. Neuromorphic systems demonstrate the most potential for speed and efficiency gains when communication is event-driven and computations are simple but parallelizable. Accordingly, we anticipate fully realizing potential benefits from a neuromorphic interception system if event-driven sensing is combined with processing and acting also implemented on event-driven (spiking) systems. We have successfully translated a neural-inspired interception algorithm to a neural network architecture for evaluation on neuromorphic hardware. Preliminary implementations of the neural network designed for implementation on the Loihi chip are still too immature for conclusive evaluation, but the results of this effort have demonstrated a viable path for a previously developed dragonfly-inspired interception algorithm to be implemented on neuromorphic hardware.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A Generalized Transformer-Based Pulse Detection Algorithm

Pulse-like signals are ubiquitous in the field of single molecule analysis, e.g., electrical or optical pulses caused by analyte translocations in nanopores. The primary challenge in processing pulse-like signals is to capture the pulses in noisy backgrounds, but current methods are subjectively based on a user-defined threshold for pulse recognition. Here, we propose a generalized machine-learning based method, named pulse detection transformer (PETR), for pulse detection. PETR determines the start and end time points of individual pulses, thereby singling out pulse segments in a time-sequential trace. It is objective without needing to specify any threshold. It provides a generalized interface for downstream algorithms for specific application scenarios. PETR is validated using both simulated and experimental nanopore translocation data. It returns a competitive performance in detecting pulses through assessing them with several standard metrics. Finally, the generalization nature of the PETR output is demonstrated using two representative algorithms for feature extraction.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Deep learning approaches for neural decoding across architectures and recording modalities

Decoding behavior, perception or cognitive state directly from neural signals is critical for brain–computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. Here, we describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1990

Spike: Artificial intelligence scheduling for Hubble space telescope

Efficient utilization of spacecraft resources is essential, but the accompanying scheduling problems are often computationally intractable and are difficult to approximate because of the presence of numerous interacting constraints. Artificial intelligence techniques were applied to the scheduling of the NASA/ESA Hubble Space Telescope (HST). This presents a particularly challenging problem since a yearlong observing program can contain some tens of thousands of exposures which are subject to a large number of scientific, operational, spacecraft, and environmental constraints. New techniques were developed for machine reasoning about scheduling constraints and goals, especially in cases where uncertainty is an important scheduling consideration and where resolving conflicts among conflicting preferences is essential. These technique were utilized in a set of workstation based scheduling tools (Spike) for HST. Graphical displays of activities, constraints, and schedules are an important feature of the system. High level scheduling strategies using both rule based and neural network approaches were developed. While the specific constraints implemented are those most relevant to HST, the framework developed is far more general and could easily handle other kinds of scheduling problems. The concept and implementation of the Spike system are described along with some experiments in adapting Spike to other spacecraft scheduling domains.

Johnston, Mark↗
DOE OSTI2023

Spatiotemporal analysis of 3D human iPSC-derived neural networks using a 3D multi-electrode array

While there is a growing appreciation of three-dimensional (3D) neural tissues (i.e., hydrogel-based, organoids, and spheroids), shown to improve cellular health and network activity to mirror brain-like activity in vivo , functional assessment using current electrophysiology techniques (e.g., planar multi-electrode arrays or patch clamp) has been technically challenging and limited to surface measurements at the bottom or top of the 3D tissue. As next-generation MEAs, specifically 3D MEAs, are being developed to increase the spatial precision across all three dimensions (X, Y, Z), development of improved computational analytical tools to discern region-specific changes within the Z dimension of the 3D tissue is needed. In the present study, we introduce a novel computational analytical pipeline to analyze 3D neural network activity recorded from a “bottom-up” 3D MEA integrated with a 3D hydrogel-based tissue containing human iPSC-derived neurons and primary astrocytes. Over a period of ~6.5 weeks, we describe the development and maturation of 3D neural activity (i.e., features of spiking and bursting activity) within cross sections of the 3D tissue, based on the vertical position of the electrode on the 3D MEA probe, in addition to network activity (identified using synchrony analysis) within and between cross sections. Then, using the sequential addition of postsynaptic receptor antagonists, bicuculline (BIC), 2-amino-5-phosphonovaleric acid (AP-5), and 6-cyano-5-nitroquinoxaline-2,3-dione (CNQX), we demonstrate that networks within and between cross sections of the 3D hydrogel-based tissue show a preference for GABA and/or glutamate synaptic transmission, suggesting differences in the network composition throughout the neural tissue. The ability to monitor the functional dynamics of the entire 3D reconstructed neural tissue is a critical bottleneck; here we demonstrate a computational pipeline that can be implemented in studies to better interpret network activity within an engineered 3D neural tissue and have a better understanding of the modeled organ tissue.

3D culture↗
DOE OSTI2023

A leaf-level spectral library to support high-throughput plant phenotyping: predictive accuracy and model transfer

Abstract Leaf-level hyperspectral reflectance has become an effective tool for high-throughput phenotyping of plant leaf traits due to its rapid, low-cost, multi-sensing, and non-destructive nature. However, collecting samples for model calibration can still be expensive, and models show poor transferability among different datasets. This study had three specific objectives: first, to assemble a large library of leaf hyperspectral data (n=2460) from maize and sorghum; second, to evaluate two machine-learning approaches to estimate nine leaf properties (chlorophyll, thickness, water content, nitrogen, phosphorus, potassium, calcium, magnesium, and sulfur); and third, to investigate the usefulness of this spectral library for predicting external datasets (n=445) including soybean and camelina using extra-weighted spiking. Internal cross-validation showed satisfactory performance of the spectral library to estimate all nine traits (mean R2=0.688), with partial least-squares regression outperforming deep neural network models. Models calibrated solely using the spectral library showed degraded performance on external datasets (mean R2=0.159 for camelina, 0.337 for soybean). Models improved significantly when a small portion of external samples (n=20) was added to the library via extra-weighted spiking (mean R2=0.574 for camelina, 0.536 for soybean). The leaf-level spectral library greatly benefits plant physiological and biochemical phenotyping, whilst extra-weight spiking improves model transferability and extends its utility.

59 BASIC BIOLOGICAL SCIENCES↗