Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “physical learning, networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Deep convolutional neural networks for multi-scale time-series classification and application to disruption prediction in fusion devices

The multi-scale, mutli-physics nature of fusion plasmas makes predicting plasma events challenging. Recent advances in deep convolutional neural network architectures (CNN) utilizing dilated convolutions enable accurate predictions on sequences which have long-range, multi-scale characteristics, such as the time-series generated by diagnostic instruments observing fusion plasmas. Here we apply this neural network architecture to the popular problem of disruption prediction in fusion tokamaks, utilizing raw data from a single diagnostic, the Electron Cyclotron Emission imaging (ECEi) diagnostic from the DIII-D tokamak. ECEi measures a fundamental plasma quantity (electron temperature) with high temporal resolution over the entire plasma discharge, making it sensitive to a number of potential pre-disruptions markers with different temporal and spatial scales. Promising, initial disruption prediction results are obtained training a deep CNN with large receptive field ({$\sim$}30k), achieving an $F_1$-score of {$\sim$}91\% on individual time-slices using only the ECEi data.

convolutional neural networks↗
DOE OSTI2021

Machine learning methods for probabilistic locked-mode predictors in tokamak plasmas

A rotating tokamak plasma can interact resonantly with the external helical magnetic perturbations, also known as error fields. This can lead to locking and then to disruptions. We leverage machine learning (ML) methods to predict the locking events. We use a coupled third-order nonlinear ordinary differential equation model to represent the interaction of the magnetic perturbation and the plasma rotation with the error field. This model is sufficient to describe qualitatively the locking and unlocking bifurcations. Here, we explore using ML algorithms with the simulation data and experimental data, focusing on the methods that can be used with sparse datasets. These methods lead to the possibility of the avoidance of locking in real-time operations. We describe the operational space in terms of two control parameters: the magnitude of the error field and the rotation frequency associated with the momentum source that maintains the plasma rotation. The outcomes are quan- tified by order parameters that completely characterize the state, whether locked or unlocked. We use unsupervised ML methods to classify locked/unlocked states and note the usefulness of a certain normalization of the order parameters. Three supervised ML classifiers are used in suite to estimate the probability of locking in the region of control parameter space with hysteresis, i.e., the set of control parameters for which both locked and unlocked states can exist. The results show that a neural network gives the best estimate of the locking probability. An analogy of the present locking model with the van der Waals equation of state is also provided.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗
DOE OSTI2023

The influence of physical and algorithmic factors on simulated far-field waveforms and source–time functions of underground explosions using unsupervised machine learning

SUMMARY Characterizing explosion sources and differentiating between earthquake and underground explosions using distributed seismic networks becomes non-trivial when explosions are detonated in cavities or heterogeneous ground material. Moreover, there is little understanding of how changes in subsurface physical properties affect the far-field waveforms we record and use to infer information about the source. Simulations of underground explosions and the resultant ground motions can be a powerful tool to systematically explore how different subsurface properties affect far-field waveform features, but there are added variables that arise from how we choose to model the explosions that can confound interpretation. To assess how both subsurface properties and algorithmic choices affect the seismic wavefield and the estimated source functions, we ran a series of 2-D axisymmetric non-linear numerical explosion experiments and wave propagation simulations that explore a wide array of parameters. We then inverted the synthetic far-field waveform data using a linear inversion scheme to estimate source–time functions (STFs) for each simulation case. We applied principal component analysis (PCA), an unsupervised machine learning method, to both the far-field waveforms and STFs to identify the most important factors that control variance in the waveform data and differences between cases. For the far-field waveforms, the largest variance occurs in the shallower radial receiver channels in the 0–50 Hz frequency band. For the STFs, both peak amplitude and rise times across different frequencies contribute to the variance. We find that the ground equation of state (i.e. lithology and rheology) and the explosion emplacement conditions (i.e. tamped versus cavity) have the greatest effect on the variance of the far-field waveforms and STFs, with the ground yield strength and fracture pressure being secondary factors. Differences in the PCA results between the far-field waveforms and STFs could possibly be due to near-field non-linearities of the source that are not accounted for in the estimation of STFs and could be associated with yield strength, fracture pressure, cavity radius and cavity shape parameters. Other algorithmic parameters are found to be less important and cause less variance in both the far-field waveforms and STFs, meaning algorithmic choices in how we model explosions are less important, which is encouraging for the further use of explosion simulations to study how physical Earth properties affect seismic waveform features and estimated STFs.

58 GEOSCIENCES↗
DOE OSTI2023

Physics–Informed Neural Networks of the Saint–Venant Equations for Downscaling a Large–Scale River Model

Large-scale river models are being refined over coastal regions to improve the scientific understanding of coastal processes, hazards and responses to climate change. However, coarse mesh resolutions and approximations in physical representations of tidal rivers limit the performance of such models at resolving the complex flow dynamics near the river-ocean interface, resulting in inaccurate simulations of flood inundation. In this research, we propose a machine learning (ML) framework based on the state-of-the-art physics-informed neural network (PINN) to simulate the downscaled flow at the subgrid scale. First, we demonstrate that PINN is able to assimilate observations of various types and solve the one-dimensional (1-D) Saint-Venant equations (SVE) directly. We perform the flow simulations over a floodplain and along an open channel in several synthetic case studies. The PINN performance is evaluated against analytical solutions and numerical models. Our results indicate that the PINN solutions of water depth have satisfactory accuracy with limited observations assimilated. In the case of flood wave propagation induced by storm surge and tide, a new neural network architecture is proposed based on Fourier feature embeddings that seamlessly encodes the periodic tidal boundary condition in the PINN's formulation. Furthermore, we show that the PINN-based downscaling can produce more reasonable subgrid solutions of the along-channel water depth by assimilating observational data. The PINN solution outperforms the simple linear interpolation in resolving the topography and dynamic flow regimes at the subgrid scale. This study provides a promising path towards improving emulation capabilities in large-scale models to characterize fine-scale coastal processes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Benchmarking the performance of uncertainty quantification methods for neural network-based interatomic potentials

Machine-learned interatomic potentials (ML-IAPs) continue to gain popularity as accurate, computationally efficient replacements for traditional, physics-based interatomic potentials and expensive ab initio methods. Uncertainty quantification (UQ) of ML-IAPs is a growing area of research as UQ is critical in many applications of IAPs, such as developing curated datasets, active learning-based data augmentation, self-improving models, and estimating the uncertainty of molecular dynamics simulations. In this paper, we construct and benchmark a series of different neural network potentials (NNPs) with varying network architectures to determine the performance of these models with respect to both the mean and uncertainty calibration error. Each NNP method is specifically designed to predict either epistemic or aleatoric uncertainty with particular focus on the differences in behavior between the epistemic and aleatoric uncertainty estimates. We benchmark these methods using multiple datasets common in the ML-IAP literature. The results show that the aleatoric uncertainty from single-shot model architectures is a competitive alternative to ensemble-based epistemic uncertainty predictions in regions of sufficient data-density. However, in regions where the representative data is sparse, aleatoric uncertainty models tend to overpredict and epistemic methods tend to underpredict the actual model error. We conclude that the type of UQ is crucial when discussing performance of probabilistic model results as different methods have different performance characteristics depending on the regime in which they are evaluated. Therefore, the type of UQ method should be carefully evaluated against both the data characteristics and requirements for the intended application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Sparse Convolutional Neural Networks for particle classification in ProtoDUNE-SP events

Deep Learning (DL) methods and Computer Vision are becoming important tools for event reconstruction in particle physics detectors. In this work, we report on the use of submanifold sparse convolutional neural networks (SparseNets) for the classification of track and shower hits from a DUNE prototype liquid-argon detector at CERN (ProtoDUNE-SP). By taking advantage of the three-dimensional nature of the problem we use a set of nine input features to classify sparse and locally dense hits associated to track or shower particles. The SparseNet has been trained on a test sample and shows promising results: efficiencies and purities greater than 90%. This has also been achieved with a considerable speedup and substantially less resource utilization with respect to other DL networks such as graph neural networks. This method offers great scalability advantages for future large neutrino detectors such as the planned DUNE experiment.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Hybrid data-driven and model-informed online tool wear detection in milling machines

Precision machining tool wear is responsible for low product throughput and quality. Monitoring the tool wear online is vital to prevent degradation in machining quality. However, direct real-time tool wear measurement is not practical. This paper presents residual-based anomaly detection models, combining a hybrid model comprised of a physics-based model and a data-driven model (a decision tree or a neural network) to predict signals of interest (e.g., power or forces) under nominal conditions, followed by Page’s cumulative sum test for detecting tool wear on-line using the computer numerical control machine measurements. The most informative features are ranked using dynamic programming and its approximation variants from real-time measurements and machine settings, such as the width of cut, depth of cut, feed rate and spindle speed, that serve as inputs to the predictive models. The baseline nominal model is incrementally updated with experimental data via a gradient boosted adaptation model to generate the residuals that account for discrepancies between the actual machine data under normal conditions and the baseline nominal model predictions. The hybrid model is validated against 20 Mazak milling machine experimental tests and one Haas run-to-failure experiment. The proposed anomaly detector is applied to synthetic data from simulations of the physics-based model at different operating conditions, measurement noise levels, and tool wear levels, and the methods were able to achieve an overall 92% accuracy in data with 1% noise. The anomaly detection methods based on hybrid model reduced the false alarms of either the data-driven or physical-based models alone, and are found to be capable of good online detection of tool wear.

42 ENGINEERING↗
DOE OSTI2023

Deep-learning-aided extraction of optical constants in scanning near-field optical microscopy

Scanning near-field optical microscopy is one of the most effective techniques for spectroscopy of nanoscale systems. However, inferring optical constants from the measured near-field signal can be challenging because of a complicated and highly nonlinear interaction between the scanned probe and the sample. Conventional fitting methods applied to this problem often suffer from the lack of convergence or require human intervention. Here, we develop an alternative approach where the optical parameter extraction is automated by a deep learning network. The network provides an initial estimate that is subsequently refined by a traditional fitting algorithm. In conclusion, we show that this method demonstrates superior accuracy, stability against noise, and computational speed when applied to simulated near-field spectra.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Acoustic-based monitoring and machine learning of component status for microreactor applications

This report provides a description and assessment of recent efforts to couple acoustic-based experimental measurements and characterization with machine learning models in order to enhance structural health monitoring capabilities for nuclear microreactors. With resilient embedded sensors in development by others supported by programs funded by the US Department of Energy’s Office of Nuclear Energy, the work described herein builds upon ongoing efforts to improve non-destructive testing technology that relates measured acoustic signatures to component stresses and/or structural defects, using a combination of new experimental measurements and machine learning architectures. The experimental procedure remained similar to that developed for the previous year’s demonstration of damage detection by the authors, with the same damaged sample tested under similar applied stress conditions. Notably, a new mounting fixture was designed and implemented to improve measurement consistency and a more sophisticated laser Doppler vibrometer was employed to make high-fidelity vibration measurements. Two nominally identical sets of training data were collected for each experimental setup to better understand the repeatability of the experiment and to better test the generality of trained neural network models. Additionally, we obtained new high-quality 3D mode shapes of the damaged test article at various stress and excitation levels, providing greater insights into the physical response of the sample during testing. Previously, we demonstrated that a machine learning model based on a convolutional neural network can predict structural details of an artificially introduced interface (intact, rough cut, smooth cut), and the applied torque level. In this study, we have transitioned to graph-based neural network architectures to better develop and test a flexible framework that is more suitable to being transferred away from controlled benchtop experiments and into more applied settings where less-structured data inputs may be expected. In general, performance testing of a graph neural network on frequency-domain representations of the data indicates strong and consistent identification of test conditions for datasets recorded on damaged components. With goals of predicting damage location and other changing experimental conditions using limited datasets, predictive models using a graph neural network architecture correctly predicted the applied torque level with an accuracy of 85% using only a single measurement point and predicted within one torque level in 95% of test windows. Predictions of damage location had limited success due to the symmetry and minimal number of the damage scenarios presented during model training. Results were ambiguous as to whether the model could detect the location of the artificial damage, or if it was instead learning the location of a given measurement point on the part and subsequently detecting which points were closest to the location of the damage. This finding will be factored into upcoming planned work on damaged graphite components, where new experimental tests with a larger number and variety of damage scenarios are expected to provide improved validation of recent developments in monitoring methodology.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Application of a Physics-Informed Convolutional Neural Network for Monitoring the Temperature Fields in High-Temperature Gas Reactors

Here, this work presents current advances in applying a physics-informed convolutional neural network (CNN) to evaluate temperature distributions in advanced reactors. Our goal is to demonstrate that the CNN can reconstruct temperature fields within the solid region of a prismatic fuel assembly in a high-temperature gas reactor (HTGR) with sensor data available in only a few cooling channels. Before that, we showcase the superior performance of the physics-informed CNN in comparison to a purely data-driven multilayer perceptron (MLP), considering a canonical heated channel setup. This analysis shows the advantages of our approach and justifies its choice. The datasets employed here are obtained upon numerical simulations performed with codes under the Nuclear Energy Advanced Modeling and Simulation program. This work is important, as industry experience indicates that the assembly material in HTGR concepts is prone to large thermal-mechanical loads nearing operational limits. This makes it crucial to characterize peak temperatures and their distributions near hot spots. Modern thermocouples are unreliable in these types of harsh environments because of the high neutron fluxes and elevated temperatures involved. The CNN-based field reconstruction represents an attractive solution, enabling sensor arrays in less aggressive locations and augmenting indirect predictions for less accessible regions. The results show that the CNN reduces prediction errors by orders of magnitude in comparison to the MLP, considering the simple yet well-representative heated channel case. In the case of the HTGR fuel assembly, the CNN can successfully reconstruct temperature fields over various cooling regimes. Furthermore, we also explore the algorithm’s ability to detect abnormalities. Interestingly, the CNN proves it has the capacity to detect blockage in one of the noninstrumented cooling channels.

Machine learning↗
DOE OSTI2025

Three-dimensional reconstruction of inertial confinement fusion hot-spot plasma from x-ray and nuclear diagnostics on OMEGA

Multidimensional effects degrade the neutron yield and the compressed areal density of laser-direct-drive inertial confinement fusion implosions of layered deuterium–tritium cryogenic targets on the OMEGA Laser System with respect to 1D radiation-hydrodynamic simulation predictions. A comprehensive physics-informed 3D reconstruction effort is under way to infer hot-spot and shell conditions at stagnation from four x-ray and seven neutron detectors distributed around the OMEGA target chamber. Neutron diagnostics, providing measurements of the neutron yield, hot-spot flow velocity, and apparent ion-temperature distribution, are used to infer the mode-1 perturbation at stagnation. The x-ray imagers record the shape of the hot-spot plasma to diagnose mode-1 and mode-2 perturbations. A deep-learning convolutional neural network trained on an extensive set of 3D radiation-hydrodynamic simulations is used to interpret the x-ray and nuclear measurements to infer the 3D profiles of the hot-spot plasma conditions and the amount of laser energy coupled to the hot-spot plasma. A 3D simulation database shows that larger mode-1 asymmetries are correlated with higher hot-spot flow velocities and reduced laser-energy coupling and neutron yield. Three-dimensional hot-spot reconstructions from x-ray measurements indicate that higher amounts of residual kinetic energy are correlated with higher measured hot-spot flow velocities, consistent with 3D simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Frame invariant neural network closures for Kraichnan turbulence

Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in developing parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In recent years, machine learning has been successful in extracting complex patterns from high-resolution spatio-temporal data, leading to improved parameterization models, and ultimately better coarse grid prediction. However, the inability to satisfy known physics and poor generalization hinders the application of these models for real-world problems. In this work, we put forth a frame invariant closure approach to improve the accuracy and generalizability of deep learning-based subgrid scale closure models by embedding physical symmetries directly into the structure of the neural network. Specifically, we utilized specialized layers within the convolutional neural network in such a way that desired constraints are theoretically guaranteed without the need for any regularization terms. We demonstrate our framework for a two-dimensional decaying turbulence test case mostly characterized by the forward enstrophy cascade. We show that our frame invariant SGS model (i) accurately predicts the subgrid scale source term, (ii) respects the physical symmetries such as translation, Galilean, and rotation invariance, and (iii) is numerically stable when implemented in coarse-grid simulation with generalization to different initial conditions and Reynolds number. This work opens up a possibility of connecting physics-based theories and data-driven modeling paradigms, and thus represents a promising step towards the development of physically consistent data-driven turbulence closure models.

42 ENGINEERING↗
DOE OSTI2022

Rethinking the ill-posedness of the spectral function reconstruction — Why is it fundamentally hard and how Artificial Neural Networks can help

Reconstructing hadron spectral functions through Euclidean correlation functions are of the important missions in lattice QCD calculations. However, in a Källen–Lehmann (KL) spectral representation, the reconstruction is observed to be ill-posed in practice. It is usually ascribed to the fewer observation points compared to the number of points in the spectral function. In this paper, by solving the eigenvalue problem of continuous KL convolution, we show analytically that the ill-posedness of the inversion is fundamental and it exists even for continuous correlation functions. We discussed how to introduce regulators to alleviate the predicament, in which include the Artificial Neural Networks (ANNs) representations recently proposed by the Authors in another study. The uniqueness of solutions using ANNs representations is manifested analytically and validated numerically. Reconstructed spectral functions using different regularization schemes are also demonstrated, together with their eigen-mode decomposition. We observe that components with large eigenvalues can be reliably reconstructed by all methods, whereas those with low eigenvalues need to be constrained by regulators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Moist Physics Parameterization Based on Deep Learning

Abstract Current moist physics parameterization schemes in general circulation models (GCMs) are the main source of biases in simulated precipitation and atmospheric circulation. Recent advances in machine learning make it possible to explore data‐driven approaches to developing parameterization for moist physics processes such as convection and clouds. This study aims to develop a new moist physics parameterization scheme based on deep learning. We use a residual convolutional neural network (ResNet) for this purpose. It is trained with 1‐year simulation from a superparameterized GCM, SPCAM. An independent year of SPCAM simulation is used for evaluation. In the design of the neural network, referred to as ResCu, the moist static energy conservation during moist processes is considered. In addition, the past history of the atmospheric states, convection, and clouds is also considered. The predicted variables from the neural network are GCM grid‐scale heating and drying rates by convection and clouds, and cloud liquid and ice water contents. Precipitation is derived from predicted moisture tendency. In the independent data test, ResCu can accurately reproduce the SPCAM simulation in both time mean and temporal variance. Comparison with other neural networks demonstrates the superior performance of ResNet architecture. ResCu is further tested in a single‐column model for both continental midlatitude warm season convection and tropical monsoonal convection. In both cases, it simulates the timing and intensity of convective events well. In the prognostic test of tropical convection case, the simulated temperature and moisture biases with ResCu are smaller than those using conventional convection and cloud parameterizations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Molecular dynamics studies of knotted polymers

Molecular dynamics calculations have been used to explore the influence of knots on the strength of a polymer strand. In particular, the mechanism of breaking 31, 41, 51, and 52 prime knots has been studied using two very different models to represent the polymer: (1) the generic coarse-grained (CG) bead model of polymer physics and (2) a state-of-the-art machine learned atomistic neural network (NN) potential for polyethylene derived from electronic structure calculations. While there is a broad overall agreement between the results on the influence of the pulling rate on chain rupture based on the CG and atomistic NN models, for the simple 31 and 41 knots, significant differences are found for the more complex 51 and 52 knots. Notably, in the latter case, the NN model more frequently predicts that these knots can break not only at the crossings at the entrance/exit but also at one of the central crossing points. The relative smoothness of the CG potential energy surface also leads to stabilization of tighter knots compared to the more realistic NN model.

DelloStritto, Mark (ORCID:0000000206785860)↗
DOE OSTI2019

Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs

Uncertainty quantification for forward and inverse problems is a central challenge across physical and biomedical disciplines. We address this challenge for the problem of modeling subsurface flow at the Hanford Site by combining stochastic computational models with observational data using physics-informed GAN models. The geographic extent, spatial heterogeneity, and multiple correlation length scales of the Hanford Site require training a computationally intensive GAN model to thousands of dimensions. We develop a highly optimized implementation that scales to {\herogpucount} NVIDIA Volta GPUs. We develop a hierarchical scheme based on a multi-player game-theoretic approach for exploiting domain parallelism, map discriminators and generators to multiple GPUs, and employ efficient communication schemes to ensure training stability and convergence. Our implementation scales to {\heronodecount} nodes on the Summit supercomputer with a \weakscalingeff\% scaling efficiency, achieving peak and sustained half-precision rates of {\heropeakperf} PF/s and {\herosustainedperf} PF/s.

physics informed ML, machine learning, generative ↗