Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,405 records · Page 78
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

3D reconstruction of an inertial-confinement fusion implosion with neural networks using multiple heterogeneous data sources

3D asymmetries are major degradation mechanisms in inertial-confinement fusion implosions at the National Ignition Facility (NIF). These asymmetries can be diagnosed and reconstructed with the neutron imaging system (NIS) on three lines of sight around the NIF target chamber. Conventional tomographic reconstructions are used to reconstruct the 3D morphology of the implosion using NIS [Volegov et al., J. Appl. Phys. 127, 083301 (2020)], but the problem is ill-posed with only three imaging lines of sight. Asymmetries can also be diagnosed with the real-time neutron activation diagnostics (RTNAD) and the neutron time-of-flight (nToF) suite. Since the NIS, RTNAD, and nToF each sample a different part of the implosion using different physical principles, we propose that it is possible to overcome the limitations of too few imaging lines of sight by performing 3D reconstructions that combine information from all three heterogeneous data sources. This work presents a new machine learning-based reconstruction technique to do just this. By using a simple physics model and group of neural networks to map 3D morphologies to data, this technique can easily account for data of multiple different types. A simple proof-of-principle is presented, demonstrating that this technique can accurately reconstruct a hot-spot shape using synthetic primary neutron images and a hot-spot velocity vector. In particular, the hot-spot’s asymmetry, quantified as spherical harmonic coefficients, is reconstructed to within ±4% of the radius in 90% of test cases. In the future, this technique will be applied to actual NIS, RTNAD, and nToF data to better understand 3D asymmetries at the NIF.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

A Graph Dynamical neural network approach for decoding dynamical states in ferroelectrics.

Ferroelectric materials such as BaTiO 3 show tremendous potential for emerging advances in memory devices, particular neuromorphic type devices. High density of memory can be obtained by stabilising polar domain walls at the nanoscale, regions of discontinuity between the well-defined polarization order parameter, but little is known about what controls their structure and dynamics in real nanoscale materials. Indeed, chiral polar domain walls have been observed in heterogeneous ferroelectrics, such as oxygen-deficient BaTiO 3 , but very little is known about how such polar-domains walls interact with defects. Indeed, a critical understanding of how dynamics of domain-walls depend on point-defects is crucial to create engineered ferroelectric memory devices. For this work, we perform large-scale simulations of nansocale domain-wall dynamics in pristine and defective BaTiO 3 using reactive force-field developed by us earlier (Phys. Chem. Chem. Phys., 2019, 21, 18240–18249), and capture their dynamical dependence on point defects using a graph dynamical neural-network approach, which we adapted to interrogate solids with well-defined order-parameters, and implemented using Pytorch based libraries. Our machine learning (ML) approach goes beyond the traditional post-processing methods to capture both spatial and temporal heterogeneities of large-scale molecular dynamics simulations of complex defective ferroelectric oxide materials. We crucially find that isolated oxygen vacancies introduce very localized spatial regions (~1–2 unit-cell in length) that show slow dipole relaxation due to formation of defect-dipoles, and that these defect-dipoles in turn slow the intrinsic dynamics of domain walls. Further, the roughness of domain walls, also influenced by vacancies, introduce dynamic heterogeneity along the domain-wall. As such we find a novel mechanism by which quenched disorder due to defects introduce dynamic heterogeneity thereby influencing response to external fields (particularly time varying fields) in a ferroelectric. Our study also emphasizes the need for creating digital twins of dynamical quantities to achieve autonomous in operando control of nanoscale switching.

42 ENGINEERING↗
DOE OSTI2023

Demonstration of acoustic monitoring for structural health of microreactors: Through use of neural networks and resonant ultrasound spectroscopy

Nuclear microreactors prioritize modularity and portability and are intended to be a cost-effective technology for non-conventional nuclear markets. As such, the development of microreactors into a safe and feasible solution for energy security applications will necessitate the development of non-destructive technologies to monitor the integrity of inaccessible reactors components during operation. This demonstration applies linear and nonlinear acoustic techniques, in combination with machine learning, to detect and classify mechanical changes (stress and damage) in a test article which are broadly representative of potential operating challenges within a functioning microreactor. All necessary data has been collected for this demonstration, with minor experimental issues identified that can be addressed in follow-on work. Motivated by the expected conditions within a functioning microreactor, we have demonstrated our monitoring techniques on a core-block-like test article using an unstructured excitation source that approximates the noisy acoustic environment expected during reactor operation. At all stress states mechanically applied to the test article, a machine learning model using an artificial neural network was able to classify with 100% accuracy whether a 3D laser vibrometry point measurement was made on an intact or artificially defective test article. Further, model predictions about whether the defect interface was rough or smooth were 95% accurate, indicating the ability of acoustic techniques to recover defect characteristics. Resonant ultrasound spectroscopy (RUS) was also applied to the dataset to provide further quantitative insights about material properties. RUS analysis was ultimately hampered by several minor experimental and data issues, limiting results to certain cases for this demonstration. Last, analysis using nonlinear RUS exhibited sensitivity to changing levels of applied stresses for each intact and defective state. As presented in this demonstration, acoustic monitoring exhibits sensitivity to stress changes, which are of concern due to high thermal gradients expected during startup and operation. Further, our techniques distinguish between measurements made on intact and damaged test articles.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

An Artificial Neural Network System for Photon-Based Active Interrogation Applications

Active interrogation (AI) is a promising technique to detect shielded special nuclear materials (SNMs). At the University of Michigan, we are developing a photon-based AI system that uses bremsstrahlung radiation from an electron linear accelerator (linac) as an ionizing source and stilbene organic scintillating detectors for neutron detection. Stilbene scintillators are sensitive to fast neutrons and photons and have excellent pulse shape discrimination (PSD) capabilities. The traditional charge integration (CI) method commonly used for PSD analysis eliminates piled-up pulses and relies on a particle discrimination line to separate neutrons and photons. The presence of the intense photon flux during AI creates a significant number of piled-up events in the stilbene scintillator, thereby posing a great challenge to the traditional CI method. Identifying true single neutron pulses becomes challenging due to the presence of a pile-up cloud and overlapping neutron, photon and pile-up clouds in the PSD analysis. To mitigate the effect of pulse pile up and identify true single neutron pulses from stilbene scintillators, an artificial neural network (ANN) system is developed. The developed ANN system identifies single neutron pulses and neutron-photon combinations from piled-up events. The results obtained from a 252Cf measurement in the presence of the intense photon flux show that the developed ANN system outperforms the traditional CI method. Since many piled-up events lie above the particle discrimination line, they get misclassified as neutrons by the traditional CI method resulting in 25% overestimation of the net neutron count rate during the linac pulse. The overall net neutron count rate (single and restored neutrons) during the linac pulse, estimated by the ANN system is 60% of the ground truth. Energy spectroscopy of the ANN attributed single neutron pulses further provides evidence on the detection of prompt fission neutrons from the 252Cf fission source.

42 ENGINEERING↗
DOE OSTI2024

Using spatio-temporal graph neural networks to estimate fleet-wide photovoltaic performance degradation patterns

Accurate estimation of photovoltaic (PV) system performance is crucial for determining its feasibility as a power generation technology and financial asset. PV-based energy solutions offer a viable alternative to traditional energy resources due to their superior Levelized Cost of Energy (LCOE). A significant challenge in assessing the LCOE of PV systems lies in understanding the Performance Loss Rate (PLR) for large fleets of PV systems. Estimating the PLR of PV systems becomes increasingly important in the rapidly growing PV industry. Precise PLR estimation benefits PV users by providing real-time monitoring of PV module performance, while explainable PLR estimation assists PV manufacturers in studying and enhancing the performance of their products. However, traditional PLR estimation methods based on statistical models have notable drawbacks. Firstly, they require user knowledge and decision-making. Secondly, they fail to leverage spatial coherence for fleet-level analysis. Additionally, these methods inherently assume the linearity of degradation, which is not representative of real world degradation. To overcome these challenges, we propose a novel graph deep learning-based decomposition method called the Spatio-Temporal Graph Neural Network for fleet-level PLR estimation (PV-stGNN-PLR). PV-stGNN-PLR decomposes the power timeseries data into aging and fluctuation components, utilizing the aging component to estimate PLR. PV-stGNN-PLR exploits spatial and temporal coherence to derive PLR estimation for all systems in a fleet and imposes flatness and smoothness regularization in loss function to ensure the successful disentanglement between aging and fluctuation. We have evaluated PV-stGNN-PLR on three simulated PV datasets consisting of 100 inverters from 5 sites. Experimental results show that PV-stGNN-PLR obtains a reduction of 33.9% and 35.1% on average in Mean Absolute Percent Error (MAPE) and Euclidean Distance (ED) in PLR degradation pattern estimation compared to the state-of-the-art PLR estimation methods.

14 SOLAR ENERGY↗
NASA NTRSDate not supplied

A Deep Neural Network for Achieving Spectrally Consistent and Seamless Infrared Radiance Measurements Across Geostationary Satellite Domains

The NASA Clouds and the Earth's Radiant Energy System (CERES) project provides the scientific community with observed top-of-atmosphere (TOA) shortwave and longwave fluxes for climate monitoring and climate model validation. To achieve this goal, CERES relies on TOA broadband fluxes derived from geostationary satellite (GEO) imagery to account for the diurnal flux variations between the CERES observation intervals. Consistent global flux derivation depends on accurate and consistent cloud retrievals. Scene-dependent spectral measurement inconsistency of the instruments that make up the contiguous ring of GEO observations (GEO-Ring), as well as limb darkening effects, can cause discontinuities in derived cloud properties and radiative fluxes at the boundaries of adjacent imager domains. Although the algorithms utilize radiative transfer models to account for instrument-band-dependent atmospheric correction and viewing zenith angle (VZA) dependency, small discontinuities may persist due to uncertainties inherent to the multiple imager-specific algorithms. Furthermore, while hyperspectral-instrument-based spectral band adjustment factors may effectively account for spectrally induced bias, they are less effective at reducing variance owed to the specific composition of the viewed scene, which is challenging to robustly characterize. As such, this article highlights the use of a deep neural network (DNN) to resolve spectral-and VZA-induced biases between GEO-Ring imagers. The DNN uses available infrared (IR) channels from the GEO instruments, along with viewing and solar illumination geometry, to estimate homogenized, VIIRS-like IR radiances for use in the GEO cloud algorithm. This approach is effective at mitigating scene-dependent spectral variance and VZA dependency, resulting in consistent radiance measurements across the GEO-Ring, thereby leading toward a more seamless global cloud assessment.

deep learning↗
DOE OSTI2022

Evaluation of artificial neural network performance for classification of potato plants infected with potato virus Y using spectral data on multiple varieties and genotypes

Potato virus Y (Potyviridae, PVY) is a plant virus that poses a significant threat to potato producers on a global basis. The pathogen has disrupted seed potato supplies and negatively impacted yield and quality of commercial potato crops. The potato industry currently manages PVY infection levels via insecticide applications, regional seed certification programs that rely on field scouting to visually assess individual plants for infection status, and destructive and costly tissue sampling coupled with laboratory assays. Despite these efforts, PVY continues to confound potato industry stakeholders resulting in economic harm. Remote sensing and machine learning provide for the development of new tools to more accurately detect and spatially quantify PVY-infected plants versus the current state of the art. However, there is a need to understand how the occurrence of many different potato varieties impact the dynamics of developing models to detect potato plants impacted with PVY and their potential effectiveness. This study evaluates classification modelling outcomes using spectral datasets collected in different temporal and spatial environments (greenhouse and a production field) on multiple potato varieties consisting of labelled instances of plants infected with PVY and those not infected with the virus. A modelling framework was developed to support iterative modelling runs using artificial neural network (ANN) architectures configured as binary classifiers to develop sample populations to support statistical analysis on model performance using specific spectral subsets. When using spectral data to detect PVY-infected plants, ANN models achieved the highest mean accuracy of 0.894 on a single variety. Conversely, the same ANN model architecture only achieved a mean accuracy of 0.575 on a spectral data set representing 29 potato breeding lines. Additionally, statistical analysis indicates spectral regions including the red edge, near infrared and shortwave infrared contain more important spectral features for the ANN classifier introduced in this research.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

A detailed study of interpretability of deep neural network based top taggers

Abstract Recent developments in the methods of explainable artificial intelligence (XAI) allow researchers to explore the inner workings of deep neural networks (DNNs), revealing crucial information about input–output relationships and realizing how data connects with machine learning models. In this paper we explore interpretability of DNN models designed to identify jets coming from top quark decay in high energy proton–proton collisions at the Large Hadron Collider. We review a subset of existing top tagger models and explore different quantitative methods to identify which features play the most important roles in identifying the top jets. We also investigate how and why feature importance varies across different XAI metrics, how correlations among features impact their explainability, and how latent space representations encode information as well as correlate with physically meaningful quantities. Our studies uncover some major pitfalls of existing XAI methods and illustrate how they can be overcome to obtain consistent and meaningful interpretation of these models. We additionally illustrate the activity of hidden layers as neural activation pattern diagrams and demonstrate how they can be used to understand how DNNs relay information across the layers and how this understanding can help to make such models significantly simpler by allowing effective model reoptimization and hyperparameter tuning. These studies not only facilitate a methodological approach to interpreting models but also unveil new insights about what these models learn. Incorporating these observations into augmented model design, we propose the particle flow interaction network model and demonstrate how interpretability-inspired model augmentation can improve top tagging performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A multiscale recurrent neural network model for predicting energy production from geothermal reservoirs

Optimization of energy production from geothermal reservoirs requires reliable prediction of energy production performance under alternative operation and development scenarios. Traditionally, reservoir simulation models are used for the evaluation and screening of alternative production and development plans. However, simulation models require extensive data collection and modeling efforts and are time-consuming to build, run, and update. Data-driven predictive models, on the other hand, can serve as efficient prediction tools that can be used for decision support and management of daily operations and surveillance activities. Data-driven models become particularly attractive when a reservoir simulation model for a field does not exist and/or is difficult to build. Machine learning (ML)-based data-driven models that have recently become popular in several fields exploit statistical patterns and relations in training data to generate predictions. As such, they tend to perform better in interpolation problems (that is, prediction within the training data range) than when they are used to extrapolate beyond the training data. Production data from geothermal reservoirs tend to exhibit short-term variabilities as well as long-term trends, such as monotonically declining production temperatures. Capturing both short-term features and long-term trends with ML-based models is not trivial. We evaluate the use of recurrent neural networks (RNN) for the prediction of energy production from geothermal reservoirs. RNN is a class of ML architectures that are used to represent and predict sequential/dynamic data. Thus, it can be challenging to apply RNN to problems where long-term trends must be captured and extrapolation beyond the training data range is needed. We introduce the multiscale RNN architecture to extend the application of RNN to detect and predict both short-term variabilities and long-term trends in geothermal data. The developed architecture consists of a long-term component to only capture low-frequency data patterns, and a short-term component to detect features with higher frequency and more nonlinearity. The final prediction is obtained by combining the long-term and short-term predictions. Both synthetic and field data are used to evaluate the presented multiscale RNN model. The prediction performance of the multiscale RNN is compared against those obtained from the regular RNN and the autoregressive (AR) model. The results suggest that the multiscale architecture improves the long-term prediction performance of the regular RNN and enhances its robustness against noise.

15 GEOTHERMAL ENERGY↗
NASA NTRS1990

Toward modeling a dynamic biological neural network

The organization of a mammalian macular endorgan is interpreted on physical and engineering principles. Having accomplished this task, it is possible to mathematically and symbolically model information processing by the macular neural network. A complex symbolic model is produced through use of mathematical notations that describe the functioning system. The model, consisting of six tiers, is constructed to mimic the neural system. On the basis of initial simulations, it is concluded that the network functions best when some of the detecting elements (i.e., type I hair cells) are weakly inhibitory. The simulations also illustrate the importance of disinhibition of receptors located in the third tier in shaping nerve discharge patterns at the sixth tier in the model system.

Ross, M. D.↗
DOE OSTI2023

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Abstract We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

Whiting, Wes (ORCID:0000000247505060)↗
DOE OSTI2025

Quantum Time Dynamics Mediated by the Yang–Baxter Equation and Artificial Neural Networks

Quantum computing shows great potential, but errors pose a significant challenge. This study explores new strategies for mitigating quantum errors using artificial neural networks (ANNs) and the Yang–Baxter equation (YBE). Unlike traditional error mitigation methods, which are computationally intensive, we investigate artificial error mitigation. We developed a novel method that combines ANNs for noise mitigation combined with the YBE to generate noisy data. This approach effectively reduces noise in quantum simulations, enhancing the accuracy of the results. The YBE rigorously preserves quantum correlations and symmetries in spin chain simulations in certain classes of integrable lattice models, enabling effective compression of quantum circuits while retaining linear scalability with the number of qubits. This compression facilitates both full and partial implementations, allowing the generation of noisy quantum data on hardware alongside noiseless simulations using classical platforms. By introducing controlled noise through the YBE, we enhance the data set for error mitigation. We train an ANN model on partial data from quantum simulations, demonstrating its effectiveness in mitigating errors in time-evolving quantum states, providing a scalable framework to enhance quantum computation fidelity, particularly in noisy intermediate-scale quantum (NISQ) systems. We demonstrate the efficacy of this approach by performing quantum time dynamics simulations using the Heisenberg XY Hamiltonian on real quantum devices.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Convolutional Neural Network–Aided Temperature Field Reconstruction: An Innovative Method for Advanced Reactor Monitoring

In this study, the capabilities of a physics-informed convolutional neural network (CNN) for reconstructing the temperature field from a limited set of measurements taken at the boundaries of internal flows are demonstrated. Such an approach enables the development of less invasive monitoring methods for real-time plant diagnostics. As a test case, a Molten Salt Fast Reactor (MSFR) design was selected. This circulating fuel reactor has received interest from both scientific and industrial communities due to its intrinsic safety and sustainability. Molten salt flows in such reactors, however, can present highly localized temperature peaks that can induce significant thermal stresses onto the vessel walls. At these local maxima, the salt temperature may exceed a thousand kelvins, which makes a direct measurement challenging or even unfeasible. The proposed CNN algorithm allows one to detect indirectly such discontinuities through an accurate, albeit indirect, temperature measurement method during reactor operation. The datasets employed to train and test the machine learning models in the present work were generated with Nek5000, a computational fluid dynamics (CFD) code developed at Argonne National Laboratory. The CNN algorithm is trained with CFD results that span a set of MSFR operational power and flow ranges. Here, to demonstrate the efficacy of the algorithm, predictions are made for test cases contained within the training range but for which the CFD data were not used when training. Results demonstrate that the proposed technique properly characterizes temperature peaks and distributions within the domain for a broad range of scenarios.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Prediction of electron density and pressure profile shapes on NSTX-U using neural networks

A new model for prediction of electron density and pressure profile shapes on NSTX and NSTX-U has been developed using neural networks. The model has been trained and tested on measured profiles from experimental discharges during the first operational campaign of NSTX-U. By projecting profiles onto empirically derived basis functions, the model is able to efficiently and accurately reproduce profile shapes. In order to project the performance of the model to upcoming NSTX-U operations, a large database of profiles from the operation of NSTX is used to test performance as a function of available data. The rapid execution time of the model is well suited to the planned applications, including optimization during scenario development activities, and real-time plasma control. Finally, a potential application of the model to real-time profile estimation is demonstrated.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Data optimization for large batch distributed training of deep neural networks

Distributed training in deep learning (DL) is common practice as data and models grow. The current practice for distributed training of deep neural networks faces the challenges of communication bottlenecks when operating at scale, and model accuracy deterioration with an increase in global batch size. Present solutions focus on improving message exchange efficiency as well as implementing techniques to tweak batch sizes and models in the training process. The loss of training accuracy typically happens because the loss function gets trapped in a local minima. We observe that the loss landscape minimization is shaped by both the model and training data and propose a data optimization approach that utilizes machine learning to implicitly smooth out the loss landscape resulting in fewer local minima. Our approach filters out data points which are less important to feature learning, enabling us to speed up the training of models on larger batch sizes to improved accuracy.

Gahlot, Shubhankar↗
DOE OSTI2021

Prediction of electron density and pressure profile shapes on NSTX-U using neural networks

A new model for prediction of electron density and pressure profile shapes on NSTX and NSTX-U has been developed using neural networks. The model has been trained and tested on measured profiles from experimental discharges during the first operational campaign of NSTX-U. By projecting profiles onto empirically derived basis functions, the model is able to efficiently and accurately reproduce profile shapes. In order to project the performance of the model to upcoming NSTX-U operations, a large database of profiles from the operation of NSTX is used to test performance as a function of available data. The rapid execution time of the model is well suited to the planned applications, including optimization during scenario development activities, and real-time plasma control. A potential application of the model to real-time profile estimation is demonstrated.

Boyer, Mark↗
DOE OSTI2021

Confronting Domain Shift in Trained Neural Networks

SAND2021-15138 O This code accompanies a paper published in the Proceedings of Machine Learning Research (PMLR), “Confronting Domain Shift in Trained Neural Networks.” Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Martinez, Carianne↗
NASA NTRS1993

WNN 92; Proceedings of the 3rd Workshop on Neural Networks: Academic/Industrial/NASA/Defense, Auburn Univ., AL, Feb. 10-12, 1992 and South Shore Harbour, TX, Nov. 4-6, 1992

The present conference discusses such neural networks (NN) related topics as their current development status, NN architectures, NN learning rules, NN optimization methods, NN temporal models, NN control methods, NN pattern recognition systems and applications, biological and biomedical applications of NNs, VLSI design techniques for NNs, NN systems simulation, fuzzy logic, and genetic algorithms. Attention is given to missileborne integrated NNs, adaptive-mixture NNs, implementable learning rules, an NN simulator for travelling salesman problem solutions, similarity-based forecasting, NN control of hypersonic aircraft takeoff, NN control of the Space Shuttle Arm, an adaptive NN robot manipulator controller, a synthetic approach to digital filtering, NNs for speech analysis, adaptive spline networks, an anticipatory fuzzy logic controller, and encoding operations for fuzzy associative memories.

Padgett, Mary L.↗