Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Parametrizing the detector response with neural networks

In high energy physics, characterizing the response of a detector to radiation is one of the most important and fundamental experimental tasks. In many cases, this task is accomplished by parameterizing summary statistics of the full detector response probability density. The parameterized detector response can then be used for calibration as well as for directly improving physics analysis sensitivity. This paper reports on how to parameterize summary statistics of the detector response using neural networks. Specifically, neural networks are powerful tools for incorporating multidimensional data and the loss function used during training determines which summary statistic is learned. One common summary statistic that has not been combined with deep learning (as far as the authors are aware) is the mode. A neural network-based approach to mode learning is proposed and empirically demonstrated in the context of high energy jet calibrations. Altogether, the neural network-based toolkit for detector response parameterization can enhance the utility of data collected at high energy physics experiments and beyond.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Neural entropy-stable conservative flux form neural networks for learning hyperbolic conservation laws

We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior knowledge of the governing equations or assume a fixed discretization. Our approach removes this dependency by embedding entropy-stable design principles into the learning process itself, enabling the discovery of physically consistent dynamics in a fully data-driven setting. By jointly learning both the flux function and a corresponding entropy, NESCFN promotes conservation and entropy dissipation, which is critical for long-term stability and fidelity in the system of hyperbolic conservation laws. Furthermore, numerical results demonstrate that the method achieves stability and conservation over extended time horizons and accurately captures shock propagation speeds, even without oracle access to future-time solution profiles in the training data.

Conservative flux form↗
DOE OSTI2026

Recurrent convolutional neural networks for modeling nonadiabatic dynamics of quantum-classical systems

Recurrent neural networks (RNNs) have recently been extensively applied to model the time evolution in fluid dynamics, weather predictions, and even chaotic systems due to their ability to capture temporal dependencies and sequential patterns in data. Here we present an RNN model based on convolutional neural networks for modeling the nonlinear nonadiabatic dynamics of hybrid quantum-classical systems. The dynamical evolution of the hybrid systems is governed by equations of motion for classical degrees of freedom and von Neumann equation for electrons. The Physics-Aware Recurrent Convolution (PARC) neural network structure incorporates a differentiator-integrator architecture that inductively models the spatiotemporal dynamics of generic physical systems. Here, we apply our RNN approach to learn the space-time evolution of a one-dimensional semiclassical Holstein model after an interaction quench. For shallow quenches (small changes in electron-lattice coupling), the deterministic dynamics can be accurately captured using a single-CNN-based recurrent network. In contrast, deep quenches induce chaotic evolution, making long-term trajectory prediction significantly more challenging. Nonetheless, we demonstrate that the PARC-CNN architecture can effectively learn the statistical climate of the Holstein model under deep-quench conditions.

Holstein model↗
DOE OSTI2022

Hybrid Quantum-Classical Neural Networks

Deep learning is one of the most successful and far-reaching strategies used in machine learning today. However, the scale and utility of neural networks is still greatly limited by the current hardware used to train them. These concerns have become increasingly pressing as conventional computers are soon expected to approach the physical limitations that will slow their performance improvements in the near future. For these reasons, scientists have begun to explore alternative computing platforms, like quantum computers, for training neural networks. In recent years, variational quantum circuits have emerged as one of the most successful approaches to quantum deep learning on noisy intermediate scale quantum devices. We propose a hybrid quantum-classical neural network architecture where each neuron is a variational quantum circuit. We empirically analyze the performance of this hybrid neural network on a series of binary classification data sets using a simulated IBM universal quantum computer and a state-of-the-art IBM universal quantum computer. On the simulated hardware, we observe that the hybrid neural network achieves around 10% higher classification accuracy and 20% better minimization of the cost function than an individual variational quantum circuit. On the quantum hardware, we observe that each model only performs well when the qubit and gate count is sufficiently small.

Arthur, Davis↗
DOE OSTI2024

Physics-informed State-space Neural Networks for transport phenomena

This work introduces Physics -informed State -space neural network Models (PSMs), a novel solution to achieving real-time optimization, flexibility, and fault tolerance in autonomous systems, particularly in transportdominated systems such as chemical, biomedical, and power plants. Traditional data -driven methods fall short due to a lack of physical constraints like mass conservation; PSMs address this issue by training deep neural networks with sensor data and physics -informing using components' Partial Differential Equations (PDEs), resulting in a physics -constrained, end -to -end differentiable forward dynamics model. Further, through two in silico experiments - a heated channel and a cooling system loop - we demonstrate that PSMs offer a more accurate approach than a purely data -driven model. In the former experiment, PSMs demonstrated significantly lower average root -mean -square errors across test datasets compared to a purely data -driven neural network, with reductions of 44 %, 48 %, and 94 % in predicting pressure, velocity, and temperature, respectively. Beyond accuracy, PSMs demonstrate a compelling multitask capability, making them highly versatile. In this work, we showcase two: supervisory control of a nonlinear system through a sequentially updated state -space representation and the proposal of a diagnostic algorithm using residuals from each of the PDEs. The former demonstrates PSMs' ability to handle constant and time -dependent constraints, while the latter illustrates their value in system diagnostics and fault detection.

42 ENGINEERING↗
DOE OSTI2022

GFINNs: GENERIC formalism informed neural networks for deterministic and stochastic dynamical systems

Here we propose the GENERIC formalism informed neural networks (GFINNs) that obey the symmetric degeneracy conditions of the GENERIC formalism. GFINNs comprise two modules, each of which contains two components. We model each component using a neural network whose architecture is designed to satisfy the required conditions. The component-wise architecture design provides flexible ways of leveraging available physics information into neural networks. We prove theoretically that GFINNs are sufficiently expressive to learn the underlying equations, hence establishing the universal approximation theorem. We demonstrate the performance of GFINNs in three simulation problems: gas containers exchanging heat and volume, thermoelastic double pendulum and the Langevin dynamics. In all the examples, GFINNs outperform existing methods, hence demonstrating good accuracy in predictions for both deterministic and stochastic systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A Novel Method for Controlling Crud Deposition in Nuclear Reactors Using Optimization Algorithms and Deep Neural Network Based Surrogate Models

This work presents the use of a high-fidelity neural network surrogate model within a Modular Optimization Framework for treatment of crud deposition as a constraint within light-water reactor core loading pattern optimization. The neural network was utilized for the treatment of crud constraints within the context of an advanced genetic algorithm applied to the core design problem. This proof-of-concept study shows that loading pattern optimization aided by a neural network surrogate model can optimize the manner in which crud distributes within a nuclear reactor without impacting operational parameters such as enrichment or cycle length. Several analysis methods were investigated. Analysis found that the surrogate model and genetic algorithm successfully minimized the deviation from a uniform crud distribution against a population of solutions from a reference optimization in which the crud distribution was not optimized. Strong evidence is presented that shows boron deposition in crud can be optimized through the loading pattern. This proof-of-concept study shows that the methods employed provide a powerful tool for mitigating the effects of crud deposition in nuclear reactors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Precision calibration of calorimeter signals in the ATLAS experiment using an uncertainty-aware neural network

The ATLAS experiment at the Large Hadron Collider explores the use of modern neural networks for a multi-dimensional calibration of its calorimeter signal defined by clusters of topologically connected cells (topo-clusters). The Bayesian neural network (BNN) approach not only yields a continuous and smooth calibration function that improves performance relative to the standard calibration but also provides uncertainties on the calibrated energies for each topo-cluster. The results obtained by using a trained BNN are compared to the standard local hadronic calibration and to a calibration provided by training a deep neural network. The uncertainties predicted by the BNN are interpreted in the context of a fractional contribution to the systematic uncertainties of the trained calibration. They are also compared to uncertainty predictions obtained from an alternative estimator employing repulsive ensembles.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Radar Super Resolution using a Deep Convolutional Neural Network

Super-resolution involves synthetically increasing the resolution of gridded data beyond its native resolution. Typically, this is done using interpolation schemes, which estimate sub-grid scale values from neighboring data, and perform the same operation everywhere regardless of the large-scale context, or by requiring a network of radars with overlapping fields of view. Recently, significant progress has been made in single image super resolution using convolutional neural networks. Conceptually, a neural network may be able to learn relations between large scale precipitation features and the associated sub-pixel scale variability and outperform interpolation schemes. Here, we use a deep convolutional neural network to artificially enhance the resolution of NEXRAD PPI scans. The model is trained on 6-months of reflectivity observations from the Langley Hill WA (KLGX) radar, and we find that it substantially outperforms common interpolation schemes for x4 and x8 resolution increases based on several objective error and perceptual quality metrics.

radar, machine learning, super resolution, Remote ↗
DOE OSTI2021

Multilevel Training of Neural Networks

MTNN is a software library for applying multilevel algorithms on training neural networks. Based on PyTorch, it is designed to be an extensible framework to enable researchers to implement multilevel training algorithms on a variety of neural network models. This release includes a culmination of preliminary work in applying the multilevel Full Approximation Scheme technique on fully connected and convolutional neural networks to solve PDE problems (pending publication submission).

Li, Ruipeng↗
DOE OSTI2024

Exact enforcement of temporal continuity in sequential physics-informed neural networks

The use of deep learning methods in scientific computing represents a potential paradigm shift in engineering problem solving. One of the most prominent developments is Physics-Informed Neural Networks (PINNs), in which neural networks are trained to satisfy partial differential equations (PDEs). While this method shows promise, the standard version has been shown to struggle in accurately predicting the dynamic behavior of time-dependent problems. To address this challenge, methods have been proposed that decompose the time domain into multiple segments, employing a distinct neural network in each segment and directly incorporating continuity between them in the loss function of the minimization problem. In this work we introduce a method to exactly enforce continuity between successive time segments via a solution ansatz. This hard constrained sequential PINN (HCS-PINN) method is simple to implement and eliminates the need for any loss terms associated with temporal continuity. The method is tested for a number of benchmark problems involving both linear and non-linear PDEs. Examples include various first order time dependent problems in which traditional PINNs struggle, namely advection, Allen–Cahn, and Korteweg–de Vries equations. Furthermore, second and third order time-dependent problems are demonstrated via wave and Jerky dynamics examples, respectively. Notably, the Jerky dynamics problem is chaotic, making the problem especially sensitive to temporal accuracy. Finally, the numerical experiments conducted with the proposed method demonstrated superior convergence and accuracy over both traditional PINNs and the soft-constrained counterparts.

42 ENGINEERING↗
DOE OSTI2020

hp -VPINNs: Variational physics-informed neural networks with domain decomposition

We formulate a general framework for hp-variational physics-informed neural networks (hp-VPINNs) based on the nonlinear approximation of shallow and deep neural networks and hp-refinement via domain decomposition and projection onto the space of high-order polynomials. The trial space is the space of neural network, which is defined globally over the entire computational domain, while the test space contains piecewise polynomials. Specifically in this study, the hp-refinement corresponds to a global approximation with a local learning algorithm that can efficiently localize the network parameter optimization. Here, we demonstrate the advantages of hp-VPINNs in both accuracy and training cost for several numerical examples of function approximation and in solving differential equations.

42 ENGINEERING↗
DOE OSTI2023

Spatial-Temporal Recurrent Graph Neural Networks for Fault Diagnostics in Power Distribution Systems

Fault diagnostics are extremely important to decide proper actions toward fault isolation and system restoration. The growing integration of inverter-based distributed energy resources imposes strong influences on fault detection using traditional overcurrent relays. This paper utilizes emerging graph learning techniques to build new temporal recurrent graph neural network models for fault diagnostics. The temporal recurrent graph neural network structures can extract the spatial-temporal features from data of voltage measurement units installed at the critical buses. From these features, fault event detection, fault type/phase classification, and fault location are performed. Compared with previous works, the proposed temporal recurrent graph neural networks provide a better generalization for fault diagnostics. Moreover, the proposed scheme retrieves the voltage signals instead of current signals so that there is no need to install relays at all lines of the distribution system. Therefore, the proposed scheme is generalizable and not limited by the number of relays installed. The effectiveness of the proposed method is comprehensively evaluated on the Potsdam microgrid and IEEE 123-node system in comparison with other neural network structures.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Artificial to Spiking Neural Networks Conversion with Calibration in Scientific Machine Learning

Here, we introduce a method to convert physics-informed neural networks (PINNs), commonly used in scientific machine learning, to spiking neural networks (SNNs), which are expected to have higher energy efficiency compared to traditional artificial neural networks (ANNs). We first extend the calibration technique of SNNs to arbitrary activation functions beyond ReLU, making it more versatile, and we prove a theorem that ensures the effectiveness of the calibration. We successfully convert PINNs to SNNs, enabling computational efficiency for diverse regression tasks in solving multiple differential equations, including the unsteady Navier–Stokes equations. We demonstrate great gains in terms of overall efficiency, including separable PINNs (SPINNs), which accelerate the training process. Overall, this is the first work of this kind and the proposed method achieves relatively good accuracy with low spike rates.

PINN↗
DOE OSTI2020

FLEET: Flexible Efficient Ensemble Training for Heterogeneous Deep Neural Networks

Parallel training of an ensemble of Deep Neural Networks (DNN) on a cluster of nodes is an effective approach to shorten the process of neural network architecture search and hyper-parameter tuning for a given learning task. Prior efforts have shown that data sharing, where the common preprocessing operation is shared across the DNN training pipelines, saves computational resources and improves pipeline efficiency. Data sharing strategy, however, performs poorly for a heterogeneous set of DNNs where each DNN has varying computational needs and thus different training rate and convergence speed. This paper proposes FLEET, a flexible ensemble DNN training framework for efficiently training a heterogeneous set of DNNs. We build FLEET via several technical innovations. We theoretically prove that an optimal resource allocation is NP-hard and propose a greedy algorithm to efficiently allocate resources for training each DNN with data sharing. We integrate data-parallel DNN training into ensemble training to mitigate the differences in training rates and introduce checkpointing into this context to address the issue of different convergence speeds. Experiments show that FLEET significantly improves the training efficiency of DNN ensembles without compromising the quality of the result.

Guan, Hui↗
DOE OSTI2020

Adaptive Activation Functions Accelerate Convergence in Deep and Physics-informed Neural Networks

We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima.

machine leaning, Bad minima, Inverse problems, Phy↗
DOE OSTI2023

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

Models that predict brain responses to stimuli provide one measure of understanding of a sensory system and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models and, thus, how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on 4 different tasks. Most tested models outpredicted standard spectromporal filter-bank models of auditory cortex and exhibited systematic model-brain correspondence: Middle stages best predicted primary auditory cortex, while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Upsampling Monte Carlo Reactor Simulation Tallies in Depleted Sodium-Cooled Fast Reactor Assemblies Using a Convolutional Neural Network

The computational demand of neutron Monte Carlo transport simulations can increase rapidly with the spatial and energy resolution of tallied physical quantities. Convolutional neural networks have been used to increase the resolution of Monte Carlo simulations of light water reactor assemblies while preserving accuracy with negligible additional computational cost. Here, we show that a convolutional neural network can also be used to upsample tally results from Monte Carlo simulations of sodium-cooled fast reactor assemblies, thereby extending the applicability beyond thermal systems. The convolutional neural network model is trained using neutron flux tallies from 300 procedurally generated nuclear reactor assemblies simulated using OpenMC. Validation and test datasets included 16 simulations of procedurally generated assemblies, and a realistic simulation of a European sodium-cooled fast reactor assembly was included in the test dataset. We show the residuals between the high-resolution flux tallies predicted by the neural network and high-resolution Monte Carlo tallies on relative and absolute bases. The network can upsample tallies from simulations of fast reactor assemblies with diverse and heterogeneous materials and geometries by a factor of two in each spatial and energy dimension. The network’s predictions are within the statistical uncertainty of the Monte Carlo tallies in almost all cases. This includes test assemblies for which burnup values and geometric parameters were well outside the ranges of those in assemblies used to train the network.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗