Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97

Improving neutrino energy estimation of charged-current interaction events with recurrent neural networks in MicroBooNE

We present a deep learning-based method for estimating the neutrino energy of charged-current neutrino-argon interactions. We employ a recurrent neural network (RNN) architecture for neutrino energy estimation in the MicroBooNE experiment, utilizing liquid argon time projection chamber (LArTPC) detector technology. Traditional energy estimation approaches in LArTPCs, which largely rely on reconstructing and summing visible energies, often experience sizable biases and resolution smearing because of the complex nature of neutrino interactions and the detector response. The estimation of neutrino energy can be improved after considering the kinematics information of reconstructed final-state particles. Utilizing kinematic information of reconstructed particles, the deep learning-based approach shows improved resolution and reduced bias for the muon neutrino Monte Carlo simulation sample compared to the traditional approach. In order to address the common concern about the effectiveness of this method on experimental data, the RNN-based energy estimator is further examined and validated with dedicated data-simulation consistency tests using MicroBooNE data. We also assess its potential impact on a neutrino oscillation study after accounting for all statistical and systematic uncertainties and show that it enhances physics sensitivity. This method has good potential to improve the performance of other physics analyses. Published by the American Physical Society 2024

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Neuromorphic Accelerator for Deep Spiking Neural Networks with NVM Crossbar Arrays

In this paper, we present a scalable digital hardware accelerator based on non-volatile memory arrays capable of realizing deep convolutional spiking neural networks (SNNs). Our design studies are conducted using a compact model for spin-transfer torque random access memory (STT-RAM) devices. Large networks are realized by tiling multiple cores which communicate by transmitting spike packets via an on-chip routing network. Compared to an equivalent SRAM based core design, we show that the STT-RAM based design achieves nearly 15X higher GSOPS (Synaptic Operations per Second) per Watt per mm 2 making it a promising platform for realizing systems with significant area and power limitations.

Kulkarni, Shruti↗

Real-time expert system and neural network for the classification of remotely sensed data

The paper examines software techniques for classifying remotely sensed data such that the number of computational steps and the amount of resources are bounded. The combination of both neural network and expert system methodology for classifying these data based on land use/land cover categories is examined. The method involves pipelining images through a neural net for initial classification and then through the expert system which resolves the ambiguous classifications. As with any pipeline, every component must have approximately equivalent run-times or otherwise a bottleneck will occur. If real-time is a requirement, each of the components must execute within a bounded number of steps. Attention is focused on the real-time system technique, which is argued to prevent a bottleneck for this data classification application.

Short, Nicholas, Jr.↗

Neural Network for Principle of Least Action

The principle of least action is the cornerstone of classical mechanics, theory of relativity, quantum mechanics, and thermodynamics. Here, we describe how a neural network (NN) learns to find the trajectory for a Lennard-Jones (LJ) system that maintains balance in minimizing the Onsager–Machlup (OM) action and maintaining the energy conservation. The phase-space trajectory thus calculated is in excellent agreement with the corresponding results from the “ground-truth” molecular dynamics (MD) simulation. Furthermore, we show that the NN can easily find structural transformation pathways for LJ clusters, for example, the basin-hopping transformation of an LJ 38 from an incomplete Mackay icosahedron to a truncated face-centered cubic octahedron. Unlike MD, the NN computes atomic trajectories over the entire temporal domain in one fell swoop, and the NN time step is a factor of 20 larger than the MD time step. The NN approach to OM action is quite general and can be adapted to model morphometrics in a variety of applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

When in-memory computing meets spiking neural networks—A perspective on device-circuit-system-and-algorithm co-design

This review explores the intersection of bio-plausible artificial intelligence in the form of spiking neural networks (SNNs) with the analog in-memory computing (IMC) domain, highlighting their collective potential for low-power edge computing environments. Through detailed investigation at the device, circuit, and system levels, we highlight the pivotal synergies between SNNs and IMC architectures. Additionally, we emphasize the critical need for comprehensive system-level analyses, considering the inter-dependencies among algorithms, devices, circuit, and system parameters, crucial for optimal performance. An in-depth analysis leads to the identification of key system-level bottlenecks arising from device limitations, which can be addressed using SNN-specific algorithm–hardware co-design techniques. This review underscores the imperative for holistic device to system design-space co-exploration, highlighting the critical aspects of hardware and algorithm research endeavors for low-power neuromorphic solutions.

Physics↗

Solving the nuclear pairing model with neural network quantum states

In this work, we present a variational Monte Carlo method that solves the nuclear many-body problem in the occupation number formalism exploiting an artificial neural network representation of the groundstate wave function. A memory-efficient version of the stochastic reconfiguration algorithm is developed to train the network by minimizing the expectation value of the Hamiltonian. We benchmark this approach against widely used nuclear many-body methods by solving a model used to describe pairing in nuclei for different types of interaction and different values of the interaction strength. Despite its polynomial computational cost, our method outperforms coupled-cluster and provides energies that are in excellent agreement with the numerically-exact full configuration interaction values.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Advancing Cyber-Attack Detection in Power Systems: A Comparative Study of Machine Learning and Graph Neural Network Approaches

This paper explores the detection and localization of cyber-attacks on power systems, focusing on comparing conventional machine learning (ML) and deep learning methods, and graph neural network (GNN)-based techniques. We assess the detection accuracy of these approaches and their potential to pinpoint the locations of specific buses under attack. Given the demonstrated success of GNNs in other time series anomaly detection applications, we aim to evaluate their performance within the context of power systems cyber-attack. Utilizing the IEEE 68-bus system, we simulated four types of attacks to test the selected approaches. Our results indicate that GNN-based methods outperform conventional machine learning and deep learning models in detection. Additionally, GNNs show promise in accurately localizing attacks for simple scenarios, although they still face challenges in more complex cases.

artificial intelligence↗

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗

Shape Optimization of Supersonic Turbines Using Response Surface and Neural Network Methods

Turbine performance directly affects engine specific impulse, thrust-to-weight ratio, and cost in a rocket propulsion system. A global optimization framework combining the radial basis neural network (RBNN) and the polynomial-based response surface method (RSM) is constructed for shape optimization of a supersonic turbine. Based on the optimized preliminary design, shape optimization is performed for the first vane and blade of a 2-stage supersonic turbine, involving O(10) design variables. The design of experiment approach is adopted to reduce the data size needed by the optimization task. It is demonstrated that a major merit of the global optimization approach is that it enables one to adaptively revise the design space to perform multiple optimization cycles. This benefit is realized when an optimal design approaches the boundary of a pre-defined design space. Furthermore, by inspecting the influence of each design variable, one can also gain insight into the existence of multiple design choices and select the optimum design based on other factors such as stress and materials considerations.

Papila, Nilay↗

Discovering CO Adsorption and Desorption Pathways from Chemical Reaction Neural Network Modeling of Transient Kinetics Spectroscopy

Here, we demonstrate a data-driven approach to interpreting surface reactions by combining time-resolved gas pulsing infrared spectroscopy with chemical reaction neural networks (CRNNs). Using CO adsorption and desorption on Pd(111) at 460–490 K as a model system, we show how transient kinetic data can reveal detailed reaction mechanisms. Starting with a simple one-species model, we systematically evaluate increasingly complex mechanisms involving hollow and bridge site adsorption. Despite the similar goodness of fit to the same experimental absorbance data, our models predict distinct coverage dynamics for different adsorption sites. Through analysis of spectral peak stability and predicted dynamics, we identify a mechanism in which CO primarily adsorbs on bridge sites followed by rapid conversion to hollow sites as being the most physically consistent with experimental observations. This work provides a framework for extracting mechanistic insights from limited experimental data, demonstrating how machine learning can bridge the gap between transient kinetic measurements and a molecular-level understanding of surface reactions.

36 MATERIALS SCIENCE↗

A Two-Tier Convolutional Neural Network for Combined Detection and Segmentation in Biological Imagery

Deep learning techniques have been useful for modern microscopy imaging techniques to further study and analyze biological structures and organs. Convolutional neural networks (CNN) have improved 2D object detection, localization, and segmentation. For imagery containing biological structures with depth, it is especially desirable to perform these tasks in 3D. Traditionally, performing these tasks simultaneously in 3D has proven to be computationally expensive. Currently available methodologies thus largely work to segment 3D objects from 2D images (without context from captured 3D volumes). In this work, we present a novel approach to perform fast and accurate localization, detection, and segmentation of volumes containing cells. Specifically, in our method, we modify and tune two state-of-the-art CNNs, namely 2D YOLOv2 and 3D U-Net, and combine them with a new fusion and image processing algorithms. Annotated volumes in this space are limited, and we have created synthetic data that mimics actual structures for training and testing our proposed approach. Promising results on this test data demonstrate the value of the technique and offers a methodology for 3D cell analysis in real microscopy imagery.

Ziabari, Amir↗

Ductile failure and damage localization in Al6061-T6 characterized by in situ X-ray computed tomography and neural network segmentation

Damage initiation and localization in uniaxial tension tests of an Al6061-T6 alloy are evaluated and quantified using in situ X-ray computed tomography and a neural network segmentation strategy. This segmentation strategy offers significant advantages over traditional thresholding in quantifying microstructural and damage evolution metrics. Void nucleation was observed to occur throughout the deformation process, whereas void volume did not grow significantly until later stages of deformation. Localized analysis determined that void nucleation begins in and around the necked region during the early stages of deformation, whereas limited damage nucleation occurs outside of the neck until late stages of deformation. Lastly, inhomogeneities in void and particle spacing in the initial undeformed state of the material are observed to correlate well with the location of damage localization and necking during deformation. These results provide new insights into the ductile failure of Al alloys and how preexisting defects influence damage and rupture.

36 MATERIALS SCIENCE↗

Phase I Closeout Report: Invoking Artificial Neural Networks to Measure Insider Threat Mitigation

Researchers from Sandia National Laboratories (Sandia) and the University of Texas at Austin (UT) conducted this study to explore the effectiveness of commercial artificial neural network (ANN) software to improve insider threat detection and mitigation (ITDM). This study hypothesized that ANNs could be "trainee to learn patterns of organizational behaviors, detect off-normal (or anomalous) deviations from these patterns, and alert when certain types, frequencies, or quantities of deviations emerge. The ReconaSense ANN system was installed at UT's Nuclear Engineering Teaching Laboratory (NETL) and collected 13,653 access control data points and 694 intrusion sensor data points over a three-month period. Preliminary analysis of this baseline data demonstrated regularized patterns of life in the facility, and that off-normal behaviors are detectable under certain situations -- even for a facility with anticipated highly non-routine, operational behaviors. Completion of this pilot study demonstrated how the ReconaSense ANN could be used to identify expected operational patterns and detect unexpected anomalous behaviors in support of a data-analytic approach to ITDM. While additional studies are needed to fully understand and characterize this system, the results of this initial study are overall very promising for demonstrating a new framework for ITDM utilizing ANNs and data analysis techniques.

97 MATHEMATICS AND COMPUTING↗

Fast, High-fidelity Lyα Forests with Convolutional Neural Networks

Full-physics cosmological simulations are powerful tools for studying the formation and evolution of structure in the universe but require extreme computational resources. Here, we train a convolutional neural network to use a cheaper N-body-only simulation to reconstruct the baryon hydrodynamic variables (density, temperature, and velocity) on scales relevant to the Lyα forest, using data from Nyx simulations. We show that our method enables rapid estimation of these fields at a resolution of ~20 kpc, and captures the statistics of the Lyα forest with much greater accuracy than existing approximations. Because our model is fully convolutional, we can train on smaller simulation boxes and deploy on much larger ones, enabling substantial computational savings. Furthermore, as our method produces an approximation for the hydrodynamic fields instead of Lyα flux directly, it is not limited to a particular choice of ionizing background or mean transmitted flux.

79 ASTRONOMY AND ASTROPHYSICS↗

New graph-neural-network flavor tagger for Belle II and measurement of sin 2⁢𝜙 1 in 𝐵 0 → 𝐽/𝜓⁢𝐾$^0_ S$ decays

We present GFlaT, a new algorithm that uses a graph-neural-network to determine the flavor of neutral 𝐵 mesons produced in ϒ⁡(4⁢𝑆) decays. It improves previous algorithms by using the information from all charged final-state particles and the relations between them. We evaluate its performance using 𝐵 decays to flavor-specific hadronic final states reconstructed in a 362 fb −1 sample of electron-positron collisions collected at the ϒ⁡(4⁢𝑆) resonance with the Belle II detector at the SuperKEKB collider. We achieve an effective tagging efficiency of (37.40 ± 0.43 ± 0.36%), where the first uncertainty is statistical and the second systematic, which is 18% better than the previous Belle II algorithm. Demonstrating the algorithm, we use 𝐵 0 →𝐽/𝜓⁢𝐾$^0_ S$ decays to measure the mixing-induced and direct 𝐶⁢𝑃 violation parameters, 𝑆 = (0.724 ± 0.035 ± 0.009) and 𝐶 = (−0.035 ± 0.026 ± 0.029).

CP violation↗