Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Detecting optical transients using artificial neural networks and reference images from different surveys

ABSTRACT We present a technique to detect optical transients based on an artificial neural networks method. We describe the architecture of two networks capable of comparing images of the same part of the sky taken by different telescopes. One image corresponds to the epoch in which a potential transient could exist; the other is a reference image of an earlier epoch. We use data obtained by the Dr. Cristina V. Torres Memorial Astronomical Observatory and archival reference images from the Sloan Digital Sky Survey. We trained a convolutional neural network and a dense layer network on simulated source samples and then tested the trained networks on samples created from real image data. Autonomous detection methods replace the standard process of detecting transients, which is normally achieved by source extraction of a difference image followed by human inspection of the detected candidates. Replacing the human inspection component with an entirely autonomous method would allow for a rapid and automatic follow-up of interesting targets of opportunity. The toy-model pipeline that we present here is not yet able to replace human inspection, but it might provide useful hints to identify potential candidates. The method will be further expanded and tested on telescopes participating in the Transient Optical Robotic Observatory of the South Collaboration.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

A dimensionality reduction algorithm for mapping tokamak operational regimes using a variational autoencoder (VAE) neural network

Abstract A variational autoencoder (VAE) is a type of unsupervised neural network which is able to learn meaningful data representations in a reduced dimensional space. We present an application of VAE in identifying the operational stability boundary of tokamak plasma discharges. This model was implemented using a dataset of over 3000 discharges from the high beta tokamak-extended pulse (HBT-EP) device. We found the VAE model to be capable of forming a continuous low-dimensional operational space map and identifying the operational boundaries using a specified warning time window. By projecting the operational parameters onto the same reduced space, this provides an intuitive way for the machine operator or an automated control system to perform disruption avoidance using a relevant control actuator as a discharge approaches a boundary. Pre-programmed GPU control experiments were conducted to demonstrate this control technique using HBT-EP’s saddle control coils as a horizontal position actuator, showing the ability to avoid the oncoming disruptive event and extend the duration of the discharge.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Solving differential equations using deep neural networks

Recent work on solving partial differential equations (PDEs) with deep neural networks (DNNs) is presented. The paper reviews and extends some of these methods while carefully analyzing a fundamental feature in numerical PDEs and nonlinear analysis: irregular solutions. First, the Sod shock tube solution to the compressible Euler equations is discussed and analyzed. This analysis includes a comparison of a DNN-based approach with conventional finite element and finite volume methods, and demonstrates that the DNN is competitive in terms of degrees of freedom required for a given accuracy. Further, the DNN-based approach is extended to consider performance improvements and simultaneous parameter space exploration. Next, a shock solution to compressible magnetohydrodynamics (MHD) is solved for, and used in a scenario where experimental data is utilized to enhance a PDE system that is a priori insufficient to validate against the observed/experimental data. This is accomplished by enriching the model PDE system with source terms that are then inferred via supervised training with synthetic experimental data. The resulting DNN framework for PDEs enables straightforward system prototyping and natural integration of large data sets (be they synthetic or experimental), all while simultaneously enabling single-pass exploration of an entire parameter space.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Ps and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference

Efficient machine learning implementations optimized for inference in hardware have wide-ranging benefits, depending on the application, from lower inference latency to higher data throughput and reduced energy consumption. Two popular techniques for reducing computation in neural networks are pruning, removing insignificant synapses, and quantization, reducing the precision of the calculations. In this work, we explore the interplay between pruning and quantization during the training of neural networks for ultra low latency applications targeting high energy physics use cases. Techniques developed for this study have potential applications across many other domains. We study various configurations of pruning during quantization-aware training, which we term quantization-aware pruning, and the effect of techniques like regularization, batch normalization, and different pruning schemes on performance, computational complexity, and information content metrics. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task. Further, quantization-aware pruning typically performs similar to or better in terms of computational efficiency compared to other neural architecture search techniques like Bayesian optimization. Surprisingly, while networks with different training configurations can have similar performance for the benchmark application, the information content in the network can vary significantly, affecting its generalizability.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Training quantum neural networks using the quantum information bottleneck method

Abstract We provide in this paper a concrete method for training a quantum neural network to maximize the relevant information about a property that is transmitted through the network. This is significant because it gives an operationally well founded quantity to optimize when training autoencoders for problems where the inputs and outputs are fully quantum. We provide a rigorous algorithm for computing the value of the quantum information bottleneck quantity within error ε that requires O ( log 2 ⁡ ( 1 / ϵ ) + 1 / δ 2 ) queries to a purification of the input density operator if its spectrum is supported on { 0 } ⋃ [ δ , 1 − δ ] for δ > 0 and the kernels of the relevant density matrices are disjoint. We further provide algorithms for estimating the derivatives of the QIB function, showing that quantum neural networks can be trained efficiently using the QIB quantity given that the number of gradient steps required is polynomial.

Çatlı, Ahmet Burak (ORCID:0000000152294141)↗
DOE OSTI2023

1-D coupled surface flow and transport equations revisited via the physics-informed neural network approach

The de Saint-Venant equation (SVE) and advection–diffusion equation (ADE) are commonly employed to solve solute transport problems in surface water. In this work, we propose a mesh-free method based on the physics-informed neural network (PINN) to solve the one dimensional (1-D) SVE, ADE, and the coupled SVE and ADE (SVE-ADE) under various initial and boundary conditions. The PINN model extends the architecture of deep neural networks (DNNs) with implementation of loss function, which are additionally subject to constraints imposed by the physical laws of SVE and ADE, along with their initial and boundary conditions. In such a manner, PINNs can be quickly steered to the true solution while obeying the physical laws. The results of PINN model are compared with the analytical and/or numerical solutions under various conditions to investigate its accuracy and efficiency in solving the SVE, ADE, and SVE-ADE. Our results indicate PINN can accurately simulate the shock wave morphology and avoid numerical dissipation in unsteady flow condition. The PINN method outweighed traditional numerical methods in several aspects, including its ability to function with small amounts of data, no grid discretization, and random selection of sampling points, etc. Additionally, the PINN method is also suitable for solving inverse problems with sparse and noisy data. With 1% noise and 2000 initial and boundary condition points (N u ), the errors of the estimated flow rate (v) and diffusion coefficient (D) are 0.003% and 0.105%, respectively, which indicate the accuracy and robustness of the proposed method. Finally, our results indicate the capability and robustness of the proposed PINN methodology for solving multi-physics problems, irrespective of the presence of sparse and noisy data in the training dataset.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Solving inverse problems in stochastic models using deep neural networks and adversarial training

Inverse problems associated with stochastic models constitute a significant portion of scientific and engineering applications. In such cases the unknown quantities are distributions. The applicability of traditional methods is limited because of their demanding assumptions or prohibitive computational consumption; for example, maximum likelihood methods require closed-form density functions, and Markov Chain Monte Carlo needs a large number of simulations. We propose a new method that estimates the unknown distribution by matching the statistical properties between observed and simulated random processes. We leverage the expressive power of neural networks to approximate the unknown distribution and use a discriminative neural network for computing the statistical discrepancies between the observed and simulated random processes. Here we demonstrated numerically that the proposed methods can estimate both the model parameters and learn complicated unknown distributions.

42 ENGINEERING↗
NASA NTRSDate not supplied

Variance Decomposition of MEDLI2 Reconstructed Heating Using Neural Networks

The Mars Entry, Descent, and Landing Instrumentation (MEDLI2) sensor suite collected data during entry of the Mars 2020 Perseverance rover into Mars’ atmosphere. This suite included a network of MEDLI2 Instrumented Sensor Plugs (MISPs). Each MISP was comprised of a cylinder made of Thermal Protection System (TPS) material with 1-3 embedded thermocouples (TCs), and it was flush mounted into the heatshield or backshell. Data from these in-depth TCs were used to reconstruct the aeroheating environment of the vehicle throughout entry. Surface heating was posed as an inverse problem, with the goal of estimating the surface heating by minimizing an objective function of the difference between MISP temperature measurements during flight and the temperature predictions derived from the Fully Implicit Ablation and Thermal response (FIAT) program. Given an aerothermal environment, FIAT calculates the material response and provides in-depth temperatures throughout the TPS material. To achieve the reverse, an internal tool called FIAT_Opt runs through multiple different environments until the output temperature at the TC depth closely matches the flight data. 95% confidence intervals on the reconstructed surface heating were obtained using Monte Carlo analysis, in which uncertainties in the thermocouple depth and the TPS material properties (e.g., density, thermal conductivity, heat capacity, emissivity) based on flight-lot material testing were included. A variance decomposition method using Sobol indices was employed to assess the sensitivity of the reconstructed peak heating to the TC placement and material property uncertainties. Variance decomposition was found to require tens of thousands of FIAT_Opt runs in order for the Sobol indices to converge. With a single FIAT_Opt run taking on the order of 40 minutes, the required number of computations would take months to complete, even if using multiple CPUs. To mitigate this problem, three machine learning models (ridge regression with cross-validation, random forest regression, and a deep neural network) were trained and tested using the 2000 Monte Carlo runs that were already completed. A subset of 1600 runs were used to train the model (i.e., training set), while the remaining 400 runs were used as the test set. The predictions from the deep neural network (DNN) on the test set showed nearly perfect agreement to the actual values computed with FIAT_Opt (R2 > 0.99). Using the DNN as a surrogate model, the variance decomposition using 50,000 runs was completed within minutes. The resulting Sobol indices showed that the reconstructed peak surface heating was most sensitive to the uncertainties in the thermal conductivity (ST = 0.37) and heat capacity (ST = 0.26). This method can be leveraged to provide requirements for material property measurements needed to improve the accuracy of surface heating prediction and ultimately lead to the reduction of design margins in the future. This presentation will include background on the MEDLI2 suite; the method used for inverse heating estimation; the way that material property uncertainties were accounted for using Monte Carlo analysis; a brief background on variance decomposition; the motivation for using machine learning in this context; how a neural network was trained on the data to enable variance decomposition in a fraction of the time; and the variance decomposition results for one of the MISPs.

Hannah Alpert↗
DOE OSTI2021

Analyzing acoustic emission data to identify cracking modes in cement paste using an artificial neural network

This research is focused on the identification of cracking mechanisms for cement paste using acoustic emission data, recorded from compression and notched four-point bending tests. A procedure is developed for analyzing the data by employing an agglomerative hierarchical clustering method, an artificial neural network, and a ray-tracing source location algorithm. An agglomerative hierarchical clustering method is utilized to cluster the AE data from a compression test using frequency-dependent features. A neural network is trained using the compression test data and applied to the AE data emitted during the four-point bending test. The clustered data from the four-point bending test is localized using a ray-tracing algorithm. Based on the occurrence and locations of the clustered events and signal feature analyses, potential cracking mechanisms are identified and assigned.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Efficient human activity recognition with spatio-temporal spiking neural networks

In this study, we explore Human Activity Recognition (HAR), a task that aims to predict individuals' daily activities utilizing time series data obtained from wearable sensors for health-related applications. Although recent research has predominantly employed end-to-end Artificial Neural Networks (ANNs) for feature extraction and classification in HAR, these approaches impose a substantial computational load on wearable devices and exhibit limitations in temporal feature extraction due to their activation functions. To address these challenges, we propose the application of Spiking Neural Networks (SNNs), an architecture inspired by the characteristics of biological neurons, to HAR tasks. SNNs accumulate input activation as presynaptic potential charges and generate a binary spike upon surpassing a predetermined threshold. This unique property facilitates spatio-temporal feature extraction and confers the advantage of low-power computation attributable to binary spikes. We conduct rigorous experiments on three distinct HAR datasets using SNNs, demonstrating that our approach attains competitive or superior performance relative to ANNs, while concurrently reducing energy consumption by up to 94%.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1991

Ferroelectric Memory Capacitors For Neural Networks

Thin-film ferroelectric capacitors proposed as nonvolatile analog memory devices. Intended primarily for use as synaptic connections in electronic neural networks. Connection strengths (synaptic weights) stored as nonlinear remanent polarizations of ferroelectric films. Ferroelectric memory and interrogation capacitors combined into memory devices in vertical or lateral configurations. Photoconductive layer modulated by light provides variable resistance to alter bias signal applied to memory capacitor. Features include nondestructive readout, simplicity, and resistance to ionizing radiation. Interrogated without destroying stored analog data. Also amenable to very-large-scale integration. Allows use of ac coupling, eliminating errors caused by dc offsets in amplifier circuits of neural networks.

Thakoor, Sarita↗
DOE OSTI2025

Allosteric prediction via convolutional neural networks and protein structural and dynamical features

Allostery is the phenomenon whereby a binding event or covalent modification at one site in a protein modulates function at a distal site, thus changing a protein’s functional state. As such, it is a ubiquitous aspect of protein functional regulation. Computationally predicting allosteric states is important as part of the broader challenge of functional annotation, but it also has practical implications for drug development, as targeting an allosteric site often affords greater specificity compared with targeting an orthosteric site. This study introduces a machine learning approach to predict the allosteric functional state using the small G-protein KRas as the model system, due to its implication in many types of cancer and being well studied as a result with many x-ray crystallographic structures of KRas available with different mutations and ligands bound. Using structural and dynamical features that can be cast as images, namely interatomic distances, contact maps, covariance, and mutual information, supervised learning was performed using convolutional neural networks. Two pretrained convolutional neural network architectures, GoogLeNet and ResNet18, were fine-tuned to classify KRas into active or inactive states based on these features. Across training regimes, atomic contact maps emerged as the most effective structural feature, whereas linearized mutual information outperformed covariance in capturing dynamical correlations relevant to allostery. Models achieved significant validation accuracy, with atomic contact maps yielding up to 90% accuracy. In conclusion, the findings suggest that integrating global structural rearrangements and correlated motion patterns with deep learning can reliably predict protein allosteric states, offering a promising framework for understanding allosteric regulation and developing targeted therapeutics.

Rajeshwar T., Rajitha [Oak Ridge National Laborato↗
DOE OSTI2025

Emulation of the calculations of final r -process abundance patterns with a neural network

This work explores the construction of a fast emulator for the calculation of the final pattern of nucleosynthesis in the rapid neutron capture process (the r-process). An emulator is built using a feed-forward artificial neural network (ANN). We train the ANN with nuclear data and relative abundance patterns. We take as input the β-decay half-lives and the one-neutron separation energy of the nuclei in the rare-earth region. The output is the final isotopic abundance pattern. In this work, we focus on the nuclear data and abundance patterns in the rare-earth region to reduce the dimension of the input and output space. We show that the ANN can capture the effect of the changes in the nuclear physics inputs on the final r-process abundance pattern in the adopted astrophysical conditions. We employ the deep ensemble method to quantify the prediction uncertainty of the neural network emulator. The emulator achieves a speed-up by a factor of about 20 000 in obtaining a final abundance pattern in the rare-earth region. The emulator may be utilized in statistical analyses such as uncertainty quantification, inverse problems, and sensitivity analysis.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble

In this work we explore the capability of physics-informed neural networks (PINNs) to discover multiple solutions. Many real-world phenomena governed by nonlinear differential equations (DEs), such as fluid flow, exhibit multiple solutions under the same conditions, yet capturing this solution multiplicity remains a significant challenge. A key difficulty lies in providing appropriate initial conditions or guesses, as widely used time-marching schemes and Newton’s method are highly sensitive to these choices when solving complex computational problems. While machine learning models, particularly PINNs, have shown promise in solving DEs, their ability to capture multiple solutions remains underexplored. In this work, we propose a simple and practical approach using PINNs to learn and discover multiple solutions. We first demonstrate that PINNs, when combined with random initialization and deep ensemble method—originally developed for uncertainty quantification—can effectively uncover multiple solutions to nonlinear ordinary and partial DEs. Although training large ensembles of PINNs may appear computationally demanding, this can be done efficiently using vectorization techniques supported by modern deep learning frameworks, allowing many networks to be trained simultaneously. Our approach highlights the critical role of initialization in shaping solution diversity, addressing an often-overlooked aspect of machine learning for scientific computing. Furthermore, we propose utilizing PINN-generated solutions as initial conditions or initial guesses for conventional numerical solvers to enhance accuracy and efficiency in capturing multiple solutions. Extensive numerical experiments, including the Allen–Cahn equation and cavity flow, where our approach successfully identifies both stable and unstable solutions, validate the effectiveness of our method. These findings establish a general and efficient framework for addressing solution multiplicity in nonlinear DEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Frame invariant neural network closures for Kraichnan turbulence

Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in developing parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In recent years, machine learning has been successful in extracting complex patterns from high-resolution spatio-temporal data, leading to improved parameterization models, and ultimately better coarse grid prediction. However, the inability to satisfy known physics and poor generalization hinders the application of these models for real-world problems. In this work, we put forth a frame invariant closure approach to improve the accuracy and generalizability of deep learning-based subgrid scale closure models by embedding physical symmetries directly into the structure of the neural network. Specifically, we utilized specialized layers within the convolutional neural network in such a way that desired constraints are theoretically guaranteed without the need for any regularization terms. We demonstrate our framework for a two-dimensional decaying turbulence test case mostly characterized by the forward enstrophy cascade. We show that our frame invariant SGS model (i) accurately predicts the subgrid scale source term, (ii) respects the physical symmetries such as translation, Galilean, and rotation invariance, and (iii) is numerically stable when implemented in coarse-grid simulation with generalization to different initial conditions and Reynolds number. This work opens up a possibility of connecting physics-based theories and data-driven modeling paradigms, and thus represents a promising step towards the development of physically consistent data-driven turbulence closure models.

42 ENGINEERING↗
DOE OSTI2021

Enabling complete multichannel nonadiabatic dynamics: A global representation of the two-channel coupled, 1,2 1 A and 1 3 A states of NH 3 using neural networks

Global coupled three-state two-channel potential energy and property/interaction (dipole and spin–orbit coupling) surfaces for the dissociation of NH 3 (Ã) into NH + H 2 and NH 2 + H are reported. The permutational invariant polynomial-neural network approach is used to simultaneously fit and diabatize the electronic Hamiltonian by fitting the energies, energy gradients, and derivative couplings of the two coupled lowest-lying singlet states as well as fitting the energy and energy gradients of the lowest-lying triplet state. The key issue in fitting property matrix elements in the diabatic basis is that the diabatic surfaces must be smooth, that is, the diabatization must remove spikes in the original adiabatic property surfaces attributable to the switch of electronic wavefunctions at the conical intersection seam. Here, we employ the fit potential energy matrix to transform properties in the adiabatic representation to a quasi-diabatic representation and remove the discontinuity near the conical intersection seam. The property matrix elements can then be fit with smooth neural network functions. The coupled potential energy surfaces along with the dipole and spin–orbit coupling surfaces will enable more accurate and complete treatment of optical transitions, as well as nonadiabatic internal conversion and intersystem crossing.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗