Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Spatiotemporal analysis of 3D human iPSC-derived neural networks using a 3D multi-electrode array

While there is a growing appreciation of three-dimensional (3D) neural tissues (i.e., hydrogel-based, organoids, and spheroids), shown to improve cellular health and network activity to mirror brain-like activity in vivo , functional assessment using current electrophysiology techniques (e.g., planar multi-electrode arrays or patch clamp) has been technically challenging and limited to surface measurements at the bottom or top of the 3D tissue. As next-generation MEAs, specifically 3D MEAs, are being developed to increase the spatial precision across all three dimensions (X, Y, Z), development of improved computational analytical tools to discern region-specific changes within the Z dimension of the 3D tissue is needed. In the present study, we introduce a novel computational analytical pipeline to analyze 3D neural network activity recorded from a “bottom-up” 3D MEA integrated with a 3D hydrogel-based tissue containing human iPSC-derived neurons and primary astrocytes. Over a period of ~6.5 weeks, we describe the development and maturation of 3D neural activity (i.e., features of spiking and bursting activity) within cross sections of the 3D tissue, based on the vertical position of the electrode on the 3D MEA probe, in addition to network activity (identified using synchrony analysis) within and between cross sections. Then, using the sequential addition of postsynaptic receptor antagonists, bicuculline (BIC), 2-amino-5-phosphonovaleric acid (AP-5), and 6-cyano-5-nitroquinoxaline-2,3-dione (CNQX), we demonstrate that networks within and between cross sections of the 3D hydrogel-based tissue show a preference for GABA and/or glutamate synaptic transmission, suggesting differences in the network composition throughout the neural tissue. The ability to monitor the functional dynamics of the entire 3D reconstructed neural tissue is a critical bottleneck; here we demonstrate a computational pipeline that can be implemented in studies to better interpret network activity within an engineered 3D neural tissue and have a better understanding of the modeled organ tissue.

3D culture↗

Applying Physics-Informed Neural Networks to Solve Navier–Stokes Equations for Laminar Flow around a Particle

In recent years, Physics-Informed Neural Networks (PINNs) have drawn great interest among researchers as a tool to solve computational physics problems. Unlike conventional neural networks, which are black-box models that “blindly” establish a correlation between input and output variables using a large quantity of labeled data, PINNs directly embed physical laws (primarily partial differential equations) within the loss function of neural networks. By minimizing the loss function, this approach allows the output variables to automatically satisfy physical equations without the need for labeled data. The Navier–Stokes equation is one of the most classic governing equations in thermal fluid engineering. This study constructs a PINN to solve the Navier–Stokes equations for a 2D incompressible laminar flow problem. Flows passing around a 2D circular particle are chosen as the benchmark case, and an elliptical particle is also examined to enrich the research. The velocity and pressure fields are predicted by the PINNs, and the results are compared with those derived from Computational Fluid Dynamics (CFD). Additionally, the particle drag force coefficient is calculated to quantify the discrepancy in the results of the PINNs as compared to CFD outcomes. The drag coefficient maintained an error within 10% across all test scenarios.

Hu, Beichao (ORCID:0009000163215151)↗

Modeling the Impacts of Material Properties on Oscillatory Neuron Behavior

In this study, neuromorphic computing, which mimics the functions of biological brains, offers improvements in both latency and energy efficiency over typical von Neumann computing architectures. Spiking neural networks can be especially power-efficient because they encode information temporally and can use more sparse electrical inputs. Here, we study the design of volatile memristors (variable resistors with memory) for neuronal devices, with particular consideration toward the feasibility of all-on-chip oscillation using built-in capacitance. We use circuit simulations to model the behavior of oscillator neurons with a range of realistic material properties. We find that energy inputs increase with insulating-phase resistivity, thermal conductivity, and device aspect ratio. However, we also find that the minimum capacitance needed for oscillation decreases with increasing insulating-phase resistivity, which opposes the constraints for power efficiency. Based on published data on NbO 2 , VO 2 , and EuNiO 3 , we find that existing materials can be engineered for all-on-chip spiking using their parasitic capacitance.

36 MATERIALS SCIENCE↗

DeePKS + ABACUS as a Bridge between Expensive Quantum Mechanical Models and Machine Learning Potentials

Recently, the development of machine learning (ML) potentials has made it possible to perform large-scale and long-time molecular simulations with the accuracy of quantum mechanical (QM) models. However, for different levels of QM methods, such as density functional theory (DFT) at the meta-GGA level and/or with exact exchange, quantum Monte Carlo, etc., generating a sufficient amount of data for training an ML potential has remained computationally challenging due to their high cost. In this work, we demonstrate that this issue can be largely alleviated with Deep Kohn–Sham (DeePKS), an ML-based DFT model. DeePKS employs a computationally efficient neural network-based functional model to construct a correction term added upon a cheap DFT model. Upon training, DeePKS offers closely matched energies and forces compared with high-level QM method, but the number of training data required is orders of magnitude less than that required for training a reliable ML potential. As such, DeePKS can serve as a bridge between expensive QM models and ML potentials: one can generate a decent amount of high-accuracy QM data to train a DeePKS model and then use the DeePKS model to label a much larger amount of configurations to train an ML potential. Further, this scheme for periodic systems is implemented in a DFT package ABACUS, which is open source and ready for use in various applications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Exponential Escape Rate of Filamentary Incubation in Mott Spiking Neurons

Mott materials such as vanadium oxides, when subject to a strong applied voltage, present an inhomogeneous insulator-to-metal transition with formation of metallic filaments within the insulating bulk. This property is enabling the development of compact and power-efficient neuromorphic devices known as Mott neurons. However, the nature of the transition has not been fully understood yet, as it may be attributed to different effects, including Joule self-heating and hot-carrier injection. Moreover, the experimental determination of the threshold voltage needed to induce the transition has proven to be challenging, as the transition becomes increasingly unpredictable when the threshold is approached. The physical understanding of these issues would not only deepen our understanding of Mott insulators, but would also be an important step toward the realization of neuromorphic devices based on such materials. In this work we use numerical simulations based on the Mott resistor network model to study the nature of the filament incubation and formation process. Here, we show that both electronic and thermal effects, in the form of current density focusing and Joule self-heating, respectively, contribute to the filamentary incubation and growth. Remarkably, we find that the percolation of the metallic filaments near the threshold is intrinsically stochastic, qualitatively similar to the familiar Arrhenius activated behavior and to the stochastic firing of biological neurons. More precisely, we characterize the filament percolation as a Poisson point process, which has the same probability distribution as mathematical models of neuronal firing with an exponential escape rate. Finally, we support the numerical simulation results by performing experiments in VO 2 that are in agreement with the exponential escape rate behavior. Thus, we establish a functionality of Mott insulators that opens a path toward implementing neuromorphic hardware with quantum materials.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Citation network datasets for benchmarking spiking graph neural networks on experimental neuromorphic hardware

Spiking neural networks (SNNs) running on neuromorphic computers offer an energy-efficient alternative for AI tasks. Recently, spiking graph neural networks (S-GNNs) have been shown to produce encouraging results on benchmark citation network datasets such as Cora, CiteSeer, and PubMed for node classification tasks. These S-GNNs were run on SNN simulators only because they contain up to tens of thousands of neurons and up to millions of synapses, translating poorly to neuromorphic hardware. Therefore, in this paper, we create a suite of benchmark datasets from the CiteSeer dataset that can be accommodated on current neuromorphic hardware platforms. Our contribution consists of a collection of three datasets. First, we have an induced subgraph of CiteSeer, which we call MiniSeer, containing 2110 papers, 3604 binary features, and 6 topics. Second, MicroSeer is a very small dataset consisting of 84 papers, 1227 features, and 6 topics. Lastly, BiteSeer is a collection of 15 binary classification datasets. We present creation of these datasets along with accuracies, running times, and spike counts when simulated. We believe that our results in this paper will be used by the neuromorphic community to benchmark, test, and develop neuromorphic hardware and simulators.

Zhu, Kevin [George Mason University, Virginia]↗

Predicting Band-Gap of Inorganic Materials Using Neuromorphic Graph Learning

Predicting properties of inorganic materials is a heavily researched topic, with several new prediction approaches emerging as competitors. One such competitor is graph neural networks, which leverage the structure of the graph to aid in the prediction process. In this work, we propose integration of neuromorphic computation into the graph neural network pipeline. We call this approach Neuromorphic Graph Learning (NGL). We utilize the NGL approach to leverage evolutionary algorithms and a novel Spike Pipeline for Raster Analysis (SPIRE) for the prediction of band gap in inorganic materials.

Mulet, Ian [University of Tennessee (UT)]↗

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗

Cost function dependent barren plateaus in shallow parametrized quantum circuits

Variational quantum algorithms (VQAs) optimize the parameters θ of a parametrized quantum circuit V(θ) to minimize a cost function C. While VQAs may enable practical applications of noisy quantum computers, they are nevertheless heuristic methods with unproven scaling. Here, we rigorously prove two results, assuming V(θ) is an alternating layered ansatz composed of blocks forming local 2-designs. Our first result states that defining C in terms of global observables leads to exponentially vanishing gradients (i.e., barren plateaus) even when V(θ) is shallow. Hence, several VQAs in the literature must revise their proposed costs. On the other hand, our second result states that defining C with local observables leads to at worst a polynomially vanishing gradient, so long as the depth of V(θ) is Ο(logn). Our results establish a connection between locality and trainability. We illustrate these ideas with large-scale simulations, up to 100 qubits, of a quantum autoencoder implementation.

97 MATHEMATICS AND COMPUTING↗

Exploring End-to-end Deep Learning Applications for Event Classification at CMS

An essential part of new physics searches at the Large Hadron Collider (LHC) at CERN involves event classification, or distinguishing potential signal events from those coming from background processes. Current machine learning techniques accomplish this using traditional hand-engineered features like particle 4-momenta, motivated by our understanding of particle decay phenomenology. While such techniques have proven useful for simple decays, they are highly dependent on our ability to model all aspects of the phenomenology and detector response. Meanwhile, powerful deep learning algorithms are capable of not only training on high-level features, but of performing feature extraction. In computer vision, convolutional neural networks have become the state-of-the-art for many applications. Motivated by their success, we apply deep learning algorithms to low-level detector data from the 2012 CMS Simulated Open Data to directly learn useful features, in what we call, end-to-end event classification. We demonstrate the power of this approach in the context of a physics search and offer solutions to some of the inherent challenges, such as image construction, image sparsity, combining multiple sub-detectors, and de-correlating the classifier from the search observable, among others.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Comparison of machine learning systems trained to detect Alfvén eigenmodes using the CO 2 interferometer on DIII-D

Abstract A Machine-Learning (ML) based detection scheme that automatically detects Alfvén Eigenmodes (AE) in a labelled DIII-D database is presented here. Controlling AEs is important for the success of planned burning plasma devices such as ITER, since resonant fast ions can drive AEs unstable and degrade the performance of the plasma or damage the first walls of the machine vessel. Artificial Intelligence could be useful for real-time detection and control of AEs in steady-state plasma scenarios by implementing ML-based models into control algorithms that drive actuators for mitigation of AE impacts. Thus, the objective is to compare differences in performance between using two different recurrent neural network systems (Reservoir Computing Network and Long Short Term Memory Network) and two different representations of the C O 2 phase data (simple and crosspower spectrograms). All C O 2 interferometer chords are used to train both models, but only one is processed during each training step. The results from the model and data comparison show higher performance for the RCN model (True Positive Rate = 90% and False Positive Rate = 14%), and that using simple magnitude spectrograms is sufficient to detect AEs. Also, the vertical C O 2 interferometer chord passing near the center is better for ML-based detection of AEs.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Improving Progressive Retrieval for HPC Scientific Data using Deep Neural Network

As the disparity between compute and I/O on high-performance computing systems has continued to widen, it has become increasingly difficult to perform post-hoc data analytics on full-resolution scientific simulation data due to the high I/O cost. Error-bounded data decomposition and progressive data retrieval framework has recently been developed to address such a challenge by performing data decomposition before storage and reading only part of the decomposed data when necessary. However, the performance of the progressive retrieval framework has been suffering from the over-pessimistic error control theory, such that the achieved maximum error of recomposed data is significantly lower than the required error. Therefore, more data than required is fetched for recomposition, incurring additional I/O overhead. In order to tackle this issue, we propose a DNN-based progressive retrieval framework that can better identify the minimum amount of data to be retrieved. Our contributions are as follows: 1) We provide an in-depth investigation of the recently developed progressive retrieval framework; 2) We propose two designs of prediction models (named D-MGARD and E-MGARD) to estimate the amount of retrieved data size based on error bounds. 3) We evaluate our proposed solutions using scientific datasets generated by real-world simulations from two domains. Evaluation results demonstrate the effectiveness of our solution in accurately predicting the amount of retrieval data size, as well as the advantages of our solution over the traditional approach to reducing the I/O overhead. Based on our evaluation, our solution is shown to read significantly less data (5% - 40% with D-MGARD, 20% - 80% with E-MGARD).

Wang, Jinzhen↗

Rebooting Computers as Learning Machines

Artificial neural networks could become the technological driver that replaces Moore's law, boosting computers' utlity through a process akin to automatic programming--although physics and computer architecture would are also a factor.

97 MATHEMATICS AND COMPUTING↗

Targeted Energy Management System Toolset

TEMST integrates advanced sensor systems with advanced computational intelligence, e.g. neural networks and fuzzy arithmetic, to enable effective energy management while minimizing negative human comfort consequences and promoting improved occupant satisfaction. Preliminary work suggests that energy cost reductions above ten percent are easily achievable. In addition to optimizing existing Energy Management Systems (EMS), the proposed toolset will be both resilient and secure in order to proactively maintain a safe level of operational normalcy in response to anomalies and malicious actions.

Rieger, CraigG.↗

2D material platform for overcoming the amplitude–phase tradeoff in ring resonators

Compact and high-speed electro-optic phase modulators play a vital role in various large-scale applications including optical computing, quantum and neural networks, and optical communication links. Conventional electro-refractive phase modulators such as silicon (Si), III-V and graphene on Si suffer from a fundamental tradeoff between device length and optical loss that limits their scaling capabilities. High-finesse ring resonators have been traditionally used as compact intensity modulators, but their use for phase modulation has been limited due to the high insertion loss associated with the phase shift. Here, we show that high-finesse resonators can achieve a strong phase shift with low insertion loss by simultaneous modulation of the real and imaginary parts of the refractive index, to the same extent, i.e., Δ n Δ k ∼1. To implement this strategy, we demonstrate an active hybrid platform that combines a low-loss SiN ring resonator with 2D materials such as graphene and transition metal dichalcogenide [tungsten disulphide (WSe 2 )], which induces a strong change in the imaginary and real parts of the index. Our platform consisting of a 25 µm long Gr-Al 2 O 3 -WSe 2 capacitor embedded on a SiN ring of 50 µm radius (∼8% ring coverage) achieves a continuous phase shift of (0.46±0.05) π radians with an insertion loss (IL) of 3.18±0.20 dB and a transmission modulation (Δ T Ring ) of 1.72±0.15dB at a probe wavelength ( λ p ) of 1646.18 nm. We find that our Gr-Al 2 O 3 -WSe 2 capacitor exhibits a phase modulation efficiency ( V π 2 ⋅ L ) of 0.530±0.016V⋅cm and can support an electro-optic bandwidth of 14.9±0.1GHz. We further show that our platform can achieve a phase shift of π radians with an IL of 5 dB and a minimum Δ T of 0.046 dB. We demonstrate the broadband nature of the binary phase response, by measuring a phase shift of (1.00±0.10) π radians, with an IL of 5.20±0.31dB and a minimal Δ T Ring of 0.015±0.006dB for resonances spanning from 1564 to 1650 nm. This SiN–2D hybrid platform provides the design for compact and high-speed reconfigurable circuits with graphene and transition metal dichalcogenide (TMD) monolayers that can enable large-scale photonic systems.

Datta, Ipshita (ORCID:0000000318842882)↗

Optical computing and image processing using photorefractive gallium arsenide

Recent experimental results on matrix-vector multiplication and multiple four-wave mixing using GaAs are presented. Attention is given to a simple concept of using two overlapping holograms in GaAs to do two matrix-vector multiplication processes operating in parallel with a common input vector. This concept can be used to construct high-speed, high-capacity, reconfigurable interconnection and multiplexing modules, important for optical computing and neural-network applications.

Cheng, Li-Jen↗

AlxGa1-xAs Single-Quantum-Well Surface-Emitting Lasers

Surface-emitting solid-state laser contains edge-emitting Al0.08Ga0.92As single-quantum-well (SQW) active layer sandwiched between graded-index-of-refraction separate-confinement-heterostructure (GRINSCH) layers of AlxGa1-xAs, includes etched 90 degree mirrors and 45 degree facets to direct edge-emitted beam perpendicular to top surface. Laser resembles those described in "Pseudomorphic-InxGa1-xAs Surface-Emitting Lasers" (NPO-18243). Suitable for incorporation into optoelectronic integrated circuits for photonic computing; e.g., optoelectronic neural networks.

Kim, Jae H.↗

Neurocomputing

The present work is intended to give technologists, research scientists, and mathematicians a graduate-level overview of the field of neurocomputing. After exploring the relationship of this field to general neuroscience, attention is given to neural network building blocks, the self-adaptation equations of learning laws, the data-transformation structures of associative networks, and the multilayer data-transformation structures of mapping networks. Also treated are the neurocomputing frontiers of spatiotemporal, stochastic, and hierarchical networks, 'neurosoftware', the creation of neural network-based computers, and neurocomputing applications in sensor processing, control, and data analysis.

Hecht-Nielsen, Robert↗