Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Physics Informed Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

Autonomous Inversion of In Situ Deformation Measurement Data for Injection-Induced Stress Change

Geologic carbon storage (GCS) is likely to play a key part of the global effort to dramatically reduce CO2 emissions and perhaps even reduce atmospheric CO2 concentrations through carbon negative operations. A critical part of effort to commercialize and widely deploy this technology is developing the capability to rapidly assimilate real-time monitoring data into a form that will enable site operators to make decisions to manage the safe and efficient operations. Two of the risks associate with GCS are the risk of inducing fractures in the sealing formations that can create leakage pathways and the risk of inducing earthquakes of sufficient magnitude to cause public concern, property damage, or safety risks. To properly manage these risks the site operator needs to know the initial state of stress, the change in stress induced by injection, and the relationship between operational parameters such as injection rate and pressure and the change in stress. Current methods of estimating the change in stress require choosing the type of constitutive model and the model parameters based on core, log, and geophysical data during the characterization phase, with little feedback from operational observations to validate or refine these choices. These characterization methods interrogate the geologic formations using length scales, loading rates or magnitudes that are quite different from those encountered by the actual storage system. It is shown that errors in the assumed constitutive response, even when informed by laboratory tests on core samples, are likely to be common, large, and underestimate the magnitude of stress change caused by injection. Recent advances in borehole-based strain instruments and borehole and surface-based tilt and displacement instruments have now enabled monitoring of the deformation of the storage system throughout its operational lifespan. This data can enable validation and refinement of the knowledge of the geomechanical properties and state of the system, but brings with it a challenge to transform the raw data into actionable knowledge. We demonstrate a method that uses automatic differentiation and a finite-element based geomechanical model perform a gradient-based deterministic inversion of geomechanical monitoring data. This approach allows autonomous integration of the instrument data without the need for time consuming manual interpretation and selection of updated model parameters. Furthermore, only isotropic linear elasticity is considered in this paper, the approach presented is very flexible as to what type of geomechanical constitutive response can be used. The approach is easily adaptable to nonlinear physics-based constitutive models to account for common rock behaviors such as creep and plasticity. The approach also enables training of machine learning-based constitutive models by allowing back propagation of errors through the finite element calculations. This enables strongly enforcing known physics, such as conservation of momentum and continuity, while allowing data-driven models to learn the truly unknown physics such as the constitutive or petrophysical responses.

Burghardt, Jeffrey A.↗

Evaluating Meteorological Dust Events and Machine-Learning Based Dust Identification in Geostationary Satellite Imagery

NASA scientists in the Short-term Prediction Research and Transition Center (SPoRT) developed a physically-based machine learning approach to identify dust in satellite imagery with a focus on night-time dust detection (Berndt et al. 201; DustTracker-AI). NASA/NOAA Geostationary Environmental Operational Satellite-16 (GOES-16) imagery was used for training and model inputs. The training, testing and validation data set consists of 28 events in the Southwest United States, capturing dust and null events in the region from 2018-2020.With 83 distinct images and millions of pixels a random forest model was trained and validated, correctly labeling 85% of dust pixels.For the first time, the model was run in near-real time production during the spring of 2022 and dust probability visualizations were made available to NOAA National Weather Service (NWS) forecasters to assess its utility for dust forecasting. Results indicated the model helped increase the confidence in the presence of dust and enabled dust tracking for a longer period of time into the night-time hours. Forecaster assessment and running the model in near real-time allowed for the team to determine the types of events missed, captured, and false alarms. To gain additional context on model performance,the SPoRT team sought to gather more detailed information on the training database(e.g., meteorological characteristics and drivers). The goal of this project was to identify the meteorological drivers for the dust events and create a database which synthesized information from observations, forecaster discussions, and analyses pertaining to the dust events to understand the types of events currently used to train the model. A more detailed meteorological synopsis was created for each dust event in the training, testing, and validation datasets. Following the completion of the database and documentation, the classification details revealed that 88% of the dust events were synoptically driven while mesoscale events were less prevalent in model datasets. Meteorological conditions found such as mixing layer depth and wind velocity had mean values of 645mb and 21kt respectively.With conditions of deep mixed layers and moderate to strong surface winds a mesoscale thunderstorm outflow event was considered and subsequently added to the model training data set to test the impact of additional mesoscale training data. The model was retrained and then qualitatively tested on a sample thunderstorm outflow case that the original model was unable to identify. Preliminary results showed potential that the addition of more mesoscale events included in the training data could help to better identify indistinct and localized dust events.

Connor Welch↗

Enhancing the Quality and Reliability of Machine Learning Interatomic Potentials through Better Reporting Practices

Recent developments in machine learning interatomic potentials (MLIPs) have empowered even nonexperts in machine learning to train MLIPs for accelerating materials simulations. However, reproducibility and independent evaluation of presented MLIP results is hindered by a lack of clear standards in current literature. In this Perspective, we aim to provide guidance on best practices for documenting MLIP use while walking the reader through the development and deployment of MLIPs including hardware and software requirements, generating training data, training models, validating predictions, and MLIP inference. We also suggest useful plotting practices and analyses to validate and boost confidence in the deployed models. Finally, we provide a step-by-step checklist for practitioners to use directly before publication to standardize the information to be reported. Altogether, we hope that our work will encourage the reliable and reproducible use of these MLIPs, which will accelerate their ability to make a positive impact in various disciplines including materials science, chemistry, and biology, among others.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Safeguards-Informed Hybrid Imagery Dataset [Poster]

Deep Learning computer vision models require many thousands of properly labelled images for training, which is especially challenging for safeguards and nonproliferation, given that safeguards-relevant images are typically rare due to the sensitivity and limited availability of the technologies. Creating relevant images through real-world staging is costly and limiting in scope. Expert-labeling is expensive, time consuming, and error prone. We aim to develop a data set of both realworld and synthetic images that are relevant to the nuclear safeguards domain that can be used to support multiple data science research questions. In the process of developing this data, we aim to develop a novel workflow to validate synthetic images using machine learning explainability methods, testing among multiple computer vision algorithms, and iterative synthetic data rendering. We will deliver one million images – both real-world and synthetically rendered – of two types uranium storage and transportation containers with labelled ground truth and associated adversarial examples.

97 MATHEMATICS AND COMPUTING↗

Hierarchical reconstruction of 3D well-connected porous media from 2D exemplars using statistics-informed neural network

The relationships between porous microstructures and transport properties are of fundamental importance in various scientific and engineering applications. Due to the intricacy, stochasticity and heterogeneity of porous media, reliable characterization and modeling of transport properties often require a complete dataset of internal microstructure samples. However, it is often an unbearable cost to acquire sufficient 3D digital microstructures by purely using microscopic imaging systems. Herein this paper presents a machine learning-based technique to hierarchically reconstruct 3D well-connected porous microstructures from one isotropic or several anisotropic low-cost 2D exemplar(s). To compactly characterize the large-scale microstructural features, a Gaussian image pyramid is built for each 2D exemplar. Local morphology patterns are collected from the Gaussian image pyramids, and then they serve as the training data to embed the 2D morphological statistics into feed-forward neural networks at multiple length levels. By using a specially-developed morphology integration scheme, the 3D morphological statistics at different levels can be inferred from the statistics-informed neural networks. Gibbs sampling is adopted to hierarchically reconstruct 3D microstructures by using multi-level 3D morphological statistics, where the large-scale, regional and local morphological patterns are statistically generated and successively added to the same 3D random field. The proposed method is tested on a series of porous media with distinct morphologies, and the statistical equivalence between the reconstructed and the real microstructures is systematically evaluated by comparing morphological descriptors and transport properties. The results demonstrate that the proposed 2D-to-3D microstructure reconstruction method is a universal and efficient approach to generating morphologically and physically realistic samples of porous media.

42 ENGINEERING↗

On the use of neural networks for energy reconstruction in high-granularity calorimeters

We contrasted the performance of deep neural networks — Convolutional Neural Network (CNN) and Graph Neural Network (GNN) — to current state of the art energy regression methods in a finely 3D-segmented calorimeter simulated by GEANT4. This comparative benchmark gives us some insight to assess the particular latent signals neural network methods exploit to achieve superior resolution. A CNN trained solely on a pure sample of pions achieved substantial improvement in the energy resolution for both single pions and jets over the conventional approaches. It maintained good performance for electron and photon reconstruction. We also used the Graph Neural Network (GNN) with edge convolution to assess the importance of timing information in the shower development for improved energy reconstruction. We implement a simple simulation based correction to the energy sum derived from the fraction of energy deposited in the electromagnetic shower component. This serves as an approximate dual-readout analogue for our benchmark comparison. Although this study does not include the simulation of detector effects, such as electronic noise, the margin of improvement seems robust enough to suggest these benefits will endure in real-world application. We also find reason to infer that the CNN/GNN methods leverage latent features that concur with our current understanding of the physics of calorimeter measurement.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

In-process monitoring and prediction of droplet quality in droplet-on-demand liquid metal jetting additive manufacturing using machine learning

Abstract In droplet-on-demand liquid metal jetting (DoD-LMJ) additive manufacturing, complex physical interactions govern the droplet characteristics, such as size, velocity, and shape. These droplet characteristics, in turn, determine the functional quality of the printed parts. Hence, to ensure repeatable and reliable part quality it is necessary to monitor and control the droplet characteristics. Existing approaches for in-situ monitoring of droplet behavior in DoD-LMJ rely on high-speed imaging sensors. The resulting high volume of droplet images acquired is computationally demanding to analyze and hinders real-time control of the process. To overcome this challenge, the objective of this work is to use time series data acquired from an in-process millimeter-wave sensor for predicting the size, velocity, and shape characteristics of droplets in DoD-LMJ process. As opposed to high-speed imaging, this sensor produces data-efficient time series signatures that allows rapid, real-time process monitoring. We devise machine learning models that use the millimeter-wave sensor data to predict the droplet characteristics. Specifically, we developed multilayer perceptron-based non-linear autoregressive models to predict the size and velocity of droplets. Likewise, a supervised machine learning model was trained to classify the droplet shape using the frequency spectrum information contained in the millimeter-wave sensor signatures. High-speed imaging data served as ground truth for model training and validation. These models captured the droplet characteristics with a statistical fidelity exceeding 90%, and vastly outperformed conventional statistical modeling approaches. Thus, this work achieves a practically viable sensing approach for real-time quality monitoring of the DoD-LMJ process, in lieu of the existing data-intensive image-based techniques.

Gaikwad, Aniruddha (ORCID:0000000285642621)↗

Improving Protein–Ligand Interaction Modeling with cryo-EM Data, Templates, and Deep Learning in 2021 Ligand Model Challenge

Elucidating protein–ligand interaction is crucial for studying the function of proteins and compounds in an organism and critical for drug discovery and design. The problem of protein–ligand interaction is traditionally tackled by molecular docking and simulation, which is based on physical forces and statistical potentials and cannot effectively leverage cryo-EM data and existing protein structural information in the protein–ligand modeling process. In this work, we developed a deep learning bioinformatics pipeline (DeepProLigand) to predict protein–ligand interactions from cryo-EM density maps of proteins and ligands. DeepProLigand first uses a deep learning method to predict the structure of proteins from cryo-EM maps, which is averaged with a reference (template) structure of the proteins to produce a combined structure to add ligands. The ligands are then identified and added into the structure to generate a protein–ligand complex structure, which is further refined. The method based on the deep learning prediction and template-based modeling was blindly tested in the 2021 EMDataResource Ligand Challenge and was ranked first in fitting ligands to cryo-EM density maps. These results demonstrate that the deep learning bioinformatics approach is a promising direction for modeling protein–ligand interactions on cryo-EM data using prior structural information.

59 BASIC BIOLOGICAL SCIENCES↗

Rapid 3D nanoscale coherent imaging via physics-aware deep learning

Phase retrieval, the problem of recovering lost phase information from measured intensity alone, is an inverse problem that is widely faced in various imaging modalities ranging from astronomy to nanoscale imaging. The current process of phase recovery is iterative in nature. As a result, the image formation is time consuming and computationally expensive, precluding real-time imaging. Here, we use 3D nanoscale X-ray imaging as a representative example to develop a deep learning model to address this phase retrieval problem. We introduce 3D-CDI-NN, a deep convolutional neural network and differential programing framework trained to predict 3D structure and strain, solely from input 3D X-ray coherent scattering data. Our networks are designed to be “physics-aware” in multiple aspects; in that the physics of the X-ray scattering process is explicitly enforced in the training of the network, and the training data are drawn from atomistic simulations that are representative of the physics of the material. We further refine the neural network prediction through a physics-based optimization procedure to enable maximum accuracy at lowest computational cost. 3D-CDI-NN can invert a 3D coherent diffraction pattern to real-space structure and strain hundreds of times faster than traditional iterative phase retrieval methods. Our integrated machine learning and differential programing solution to the phase retrieval problem is broadly applicable across inverse problems in other application areas.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗

Dimensionally reduced machine learning model for predicting single component octanol–water partition coefficients

Abstract MF-LOGP, a new method for determining a single component octanol–water partition coefficients ( $$LogP$$ LogP ) is presented which uses molecular formula as the only input. Octanol–water partition coefficients are useful in many applications, ranging from environmental fate and drug delivery. Currently, partition coefficients are either experimentally measured or predicted as a function of structural fragments, topological descriptors, or thermodynamic properties known or calculated from precise molecular structures. The MF-LOGP method presented here differs from classical methods as it does not require any structural information and uses molecular formula as the sole model input. MF-LOGP is therefore useful for situations in which the structure is unknown or where the use of a low dimensional, easily automatable, and computationally inexpensive calculations is required. MF-LOGP is a random forest algorithm that is trained and tested on 15,377 data points, using 10 features derived from the molecular formula to make $$LogP$$ LogP predictions. Using an independent validation set of 2713 data points, MF-LOGP was found to have an average $$RMSE$$ RMSE = 0.77 ± 0.007, $$MAE$$ MAE = 0.52 ± 0.003, and $${R}^{2}$$ R 2 = 0.83 ± 0.003. This performance fell within the spectrum of performances reported in the published literature for conventional higher dimensional models ( $$RMSE$$ RMSE = 0.42–1.54, $$MAE$$ MAE = 0.09–1.07, and $${R}^{2}$$ R 2 = 0.32–0.95). Compared with existing models, MF-LOGP requires a maximum of ten features and no structural information, thereby providing a practical and yet predictive tool. The development of MF-LOGP provides the groundwork for development of more physical prediction models leveraging big data analytical methods or complex multicomponent mixtures. Graphical Abstract

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Directed Energy Deposition Process Modeling, Validation, and Process-Informed Optimization

The directed energy deposition (DED) process, one of the most popular additive manufacturing techniques in use today, involves various complex physical mechanisms that are not yet well understood. In this regard, computational tools show promise for elucidating the manufacturing process and enabling nondestructive performance evaluations of manufactured parts. To better control and optimize the DED process?thereby improving the manufactured product? the present work develops and demonstrates a novel artificial intelligence (AI)-based process control and optimization technique. Specifically, a geometry-free thermo-mechanical model with adaptive subdomain construction is developed to accurately capture the material?s thermo-mechanical response under cyclical reheating and high cooling rates [1]. The model is demonstrated and validated with experimental measurements, in light of various geometries and processing parameters. Moreover, based on this thermo-mechanical model, a physics-informed reduced-order model is developed to enable quick predictions of the temperature field at every time step. Furthermore, an AI-based controller is developed that can adapt to the ever-changing system states by automatically adjusting the manufacturing process parameters. This entire work is based on the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) [2] and its recent integration with Libtorch (the C++ frontend of PyTorch [3]). The combined development of the adaptive material deposition scheme, thermo-mechanical model, associated reduced-order model, and AI-based process controller carries great potential for improving advanced manufacturing processes.

36 MATERIALS SCIENCE↗

Investigating performance and variability of NIF ICF experiments with deep learning

The parameter space involved in designing an inertial confinement fusion shot at the National Ignition Facility (NIF) is massively multi-dimensional and the cost of a single shot makes a comprehensive set of sensitivity studies in the laboratory impractical. The use of machine learning to overcome these challenges has gained popularity and has had several successful applications by the scientific community. We extend on these efforts by training a neural network (NN) on information about the experimental design, engineering elements, and drive asymmetry to predict with uncertainty the neutron yield of an experiment. We find the measured and model predicted values are in good agreement, with an R 2 value of 0.91 for a randomly selected test dataset. Almost all the predicted 95% credible intervals contain the corresponding measured value for both training and test datasets. We identify correlations picked up by the NN between the shot design, yield, and variability and use them to motivate shot sensitivity studies. The first shot to exceed the Lawson-like ignition criteria (N210808) was conducted at the NIF and subsequent shots studied the design’s robustness. In a follow-up shot to N210808, our model predicts capsule quality to be the main performance degradation mechanism that prevented the shot from repeating previous performance levels. Shot N221204 was the first shot to exceed a target energy gain of 1. Our model predicts increased yield with reduced coast time for a N221204 study and greater variability for designs with lower peak powers at constant yield. The model’s fast prediction speed and uncertainty prediction are useful for identifying interesting design paths that could warrant further investigation with conventional simulations to search for robust high yield designs.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Exploration of signal processing methods for superconducting magnet and quench data

Quenching is the phenomenon of a superconducting magnetic material carrying current transitioning into a regular conducting material. This may cause severe and irreparable damage to the superconductor due to Joule heating. The Magnet Department at Fermi National Accelerator Laboratory (FNAL) has acquired experimental data through quench antenna arrays that are recorded when the quench is detected. These data are in terms of voltage signals that are sampled at 100kHz for several minutes. There are multiple channels and each channel provides a data set of more than 20 million observations, while there is one channel, called the trigger channel which shows the time when quench is detected. Despite some advancements that were made including machine learning, data complexity still shadows the progress. In this work, we studied a multi-resolution analysis of the quench antenna data through the Haar wavelet transform. In particular, we applied the maximally overlapped discrete w avelet transform (MODWT) of a suitable level L to the given data and then projected it onto the wavelet basis. This decomposes a given signal (Original data) $x ϵ \mathbb{R}^N$ into $L + 1$ subspaces of $\mathbb{R}^N$. One of the subspaces called the approximation, captures the trend of the signal, and the others, called the details, capture the fluctuations at different frequency bands. This decomposition provides a clear trend of the data at a suitable level and also various activities (spikes) are seen in the details of the decomposition at every level. These spikes might reveal some information about the quench under investigation but in any case, give information about magnet behavior. Also, this decomposition is seen to be very useful in removing noise present in the data due to the source or mechanism of the experiment.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Highly accurate and constrained density functional obtained with differentiable programming

Using an end-to-end differentiable implementation of the Kohn-Sham self-consistent field equations, we obtain a highly accurate neural network–based exchange and correlation (XC) functional of the electronic density. The functional is optimized using information on both energy and density while exact constraints are enforced through an appropriate neural network architecture. Here we evaluate our model against different families of XC approximations and show that at the meta-GGA level our functional exhibits unprecedented accuracy for both energy and density predictions. For nonempirical functionals, there is a strong linear correlation between energy and density errors. We use this correlation to define an XC functional quality metric that includes both energy and density errors, leading to an improved way to rank different approximations.

36 MATERIALS SCIENCE↗

A Framework for Inverse Prediction Using Functional Response Data

Inverse prediction models have commonly been developed to handle scalar data from physical experiments. However, it is not uncommon for data to be collected in functional form. When data are collected in functional form, it must be aggregated to fit the form of traditional methods, which often results in a loss of information. For expensive experiments, this loss of information can be costly. In this study, we introduce the functional inverse prediction (FIP) framework, a general approach which uses the full information in functional response data to provide inverse predictions with probabilistic prediction uncertainties obtained with the bootstrap. The FIP framework is a general methodology that can be modified by practitioners to accommodate many different applications and types of data. We demonstrate the framework, highlighting points of flexibility, with a simulation example and applications to weather data and to nuclear forensics. Results show how functional models can improve the accuracy and precision of predictions.

42 ENGINEERING↗

Active causal learning for decoding chemical complexities with targeted interventions

Abstract Predicting and enhancing inherent properties based on molecular structures is paramount to design tasks in medicine, materials science, and environmental management. Most of the current machine learning and deep learning approaches have become standard for predictions, but they face challenges when applied across different datasets due to reliance on correlations between molecular representation and target properties. These approaches typically depend on large datasets to capture the diversity within the chemical space, facilitating a more accurate approximation, interpolation, or extrapolation of the chemical behavior of molecules. In our research, we introduce an active learning approach that discerns underlying cause-effect relationships through strategic sampling with the use of a graph loss function. This method identifies the smallest subset of the dataset capable of encoding the most information representative of a much larger chemical space. The identified causal relations are then leveraged to conduct systematic interventions, optimizing the design task within a chemical space that the models have not encountered previously. While our implementation focused on the QM9 quantum-chemical dataset for a specific design task—finding molecules with a large dipole moment—our active causal learning approach, driven by intelligent sampling and interventions, holds potential for broader applications in molecular, materials design and discovery.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Development of Physics-Based Transition Models for Unstructured-Mesh CFD Codes Using Deep Learning Models

Predicting transition locations over a vehicle surface is of fundamental importance for many engineering applications. With the transition information, the Reynolds-averaged Navier-Stokes (RANS) computations can turn on the turbulence model at the right locations so that drag, lift and other aerodynamic quantities can be accurately predicted. In contrast to the popularity of RANS-based transition modeling in which transition onset is governed by the turbulence equations, physics-based transition models that account for instability waves within the boundary layer, thus more compliant to flow physics, only gained more attention in recent years. This paper describes the development of a new physics-based transition model based on either the linear stability theory (LST) or parabolized stability equations (PSE). The model is designed to communicate with a structured or unstructured-mesh RANS solver back and forth in order to more accurately compute transition fronts over a three-dimensional body. In the developed model, the Python suite of interface codes in conjunction with the LASTRAC software can be executed autonomously to produce transition onset locations for a given laminar or RANS-computed transitional state. In addition, as a proof of concept, the tool set consists of a deep learning neural network model that has been designed and trained to predict instability wave evolutions inside the boundary layer for various instability wave mechanisms across a selected speed range. A machine-learned intelligent profile interpolation model has also been devised to enable reliable instability-wave spectra predictions with just a few points in the mean flow profiles.

Transition Modeling↗