SEARCH · Engineering Papers

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Differentiable multiphase flow model for physics-informed machine learning in reservoir pressure management

Accurate subsurface reservoir pressure control is extremely challenging due to geological heterogeneity and multiphase fluid-flow dynamics. Predicting behavior in this setting relies on high-fidelity physics-based simulations that are computationally expensive. Yet, the uncertain, heterogeneous properties that control these flows make it necessary to perform many of these expensive simulations, which is often prohibitive. To address these challenges, we introduce a physics-informed machine learning workflow that couples a fully differentiable multiphase flow simulator, which is implemented in the DPFEHM framework with a convolutional neural network (CNN). The CNN learns to predict fluid extraction rates from heterogeneous permeability fields to enforce pressure limits at critical reservoir locations. By incorporating transient multiphase flow physics into the training process, our method enables more practical and accurate predictions for realistic injection-extraction scenarios compared to previous works. To speed up training, we pretrain the model on single-phase, steady-state simulations and then finetune it on full multiphase scenarios, which dramatically reduces the computational cost. We demonstrate that high-accuracy training can be achieved with fewer than three thousand full-physics multiphase flow simulations – compared to previous estimates requiring up to ten million. This drastic reduction in the number of simulations is achieved by leveraging transfer learning from much less expensive single phase simulations.

25 ENERGY STORAGE↗

DOE OSTI2022

A Semi-Automated Approach for Curating a Glossary of Key Terms for Open-Source Data Queries

In FY20, the Savannah River National Laboratory (SRNL) was funded by the National Nuclear Security Administration’s Office of Defense Nuclear Non-Proliferation Research and Development (NA-22) to build a machine learning based modeling pipeline that could extract proliferation events of interest from open text-based data sources. As a test case, the research team targeted the identification/fusion of events and indicators that fissile core fabrication would be executed at the Savannah River Site prior to its official announcement in May of 2018. The demonstration prototype proved successful by applying natural language processing and graph theoretical techniques to identify contextual shifts in key words and phrases that acted as indicators that pit production would be carried out at the Savannah River Site up to two years prior to the official announcement.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2026

Nuclear Data Adjustment for Nonlinear Applications in the OECD/NEA WPNCS SG14 Benchmark—A Bayesian Inverse UQ-Based Approach for Data Assimilation

The Organisation for Economic Co-operation and Development Working Party on Nuclear Criticality Safety has proposed a benchmark exercise to assess the performance of current nuclear data adjustment techniques applied to nonlinear applications and experiments with low correlation to applications. This work introduces Bayesian inverse uncertainty quantification (IUQ) employing scientific machine learning surrogate models as a method for nuclear data adjustments in this benchmark, and compares IUQ to the more traditional methods of generalized linear least squares (GLLS) and Monte Carlo Bayes (MOCABA). Posterior predictions from IUQ showed agreement with GLLS and MOCABA for linear applications. Here, when comparing GLLS, MOCABA, and IUQ posterior predictions to computed model responses using adjusted parameters, we observe that the GLLS predictions failed to replicate the computed response distributions for nonlinear applications, while MOCABA showed near agreement, and IUQ used the computed model responses directly. We also discuss observations on why experiments with low correlation to applications can be informative to nuclear data adjustments and identify some properties useful in selecting experiments for inclusion in nuclear data adjustment. Performance in this benchmark indicates potential for Bayesian IUQ in nuclear data adjustments.

Bayesian calibration↗

DOE OSTI2023

Predicting oxidation damage of ultra high-temperature carbide ceramics in extreme environments using machine learning

Determining the oxidation resistance of UHTC carbides in extreme environments is challenging theoretically and experimentally due to the high dimensional complexity of influencing variables and intricate testing setups. Herein we demonstrate the use of machine learning (ML) models trained with experimental literature data to predict the oxide thickness of UHTC carbides exposed to air based on composition, mean grain size, relative densification, holding time, and temperature. A multi-dimensional database with 76 occurrences is created containing experimental results of Hf, Zr, and Ta carbides plus additives. In this study, the preprocessed database is then used to train ML models to predict their oxidation behavior. The trained model predicts the oxidation damage in the form of an average oxide thickness in UHTC carbides with a Mean Absolute Error (MAE) of ±65.45 μm for samples in the testing set that developed thicknesses up to 1000 μm. The model successfully predicted oxidation damage for a recession rate lower than 60 μm/min. It is noticed that the ensemble method MAE is increased to ±134.34 μm while forecasting the oxidation of samples with a recession rate higher than the threshold. The unprecedented approach is a novel way to predict the damage through the oxidation of carbide compounds before processing for a smarter design with room for improvement.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Localized keyhole pore prediction during laser powder bed fusion via multimodal process monitoring and X-ray radiography

Systematic fault detection and control during laser powder bed fusion (L-PBF) has been a long-standing objective for system manufacturers and researchers in the additive manufacturing (AM) industry. This manuscript investigates a data fusion approach for detection of keyhole porosity formation during laser irradiation of Ti-6Al-4V substrates by concurrent recording of thermally induced optical emission measured using both off-axis and coaxial photodiode sensors, and acoustic emission. Subsurface defect formation was monitored via high-speed synchrotron X-ray imaging at 20,000 frames per second, enabling temporal registration of keyhole pore formation events to the monitoring signals at a resolution of 50 µs. We developed data fusion machine learning (ML) models for localized prediction of keyhole pore formation at various time scales ranging from 0.5 ms to 2 ms. The signal segments were featurized using two independent approaches: (1) power spectral density (PSD) and (2) highly comparative time series analysis (HCTSA) framework. The extracted features from different sensor modalities were fused together to construct a multimodal feature space and sequential feature selection was used to determine the most informative features for training the ML models. The predictive performance was evaluated for three classifying algorithms: Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Gaussian Naive Bayes (GNB). As a result, pore formation events were predicted with up to 0.95 F1-score, 1.0 recall and 0.94 accuracy. The most heavily weighted features indicate that model performance is chiefly governed by the acoustic monitoring signal, with a secondary contribution from the optical emission sensors.

36 MATERIALS SCIENCE↗

DOE OSTI2024

PreMevE‐MEO: Predicting Ultra‐Relativistic Electrons Using Observations From GPS Satellites

Abstract Ultra‐relativistic electrons with energies greater than or equal to two megaelectron‐volt (MeV) pose a major radiation threat to spaceborne electronics, and thus specifying those highly energetic electrons has a significant meaning to space weather communities. Here we report the latest progress in developing our predictive model for MeV electrons in the outer radiation belt. The new version, primarily driven by electron measurements made along medium‐Earth‐orbits (MEO), is called PREdictive MEV Electron (PreMevE)‐MEO model that nowcasts ultra‐relativistic electron flux distributions across the whole outer belt. Model inputs include >2 MeV electron fluxes observed in MEOs by a fleet of GPS satellites as well as electrons measured by one Los Alamos satellite in the geosynchronous orbit. We developed an innovative Sparse Multi‐Inputs Latent Ensemble NETwork (SmileNet) which combines convolutional neural networks with transformers, and we used long‐term in situ electron data from NASA's Van Allen Probes mission to train, validate, optimize, and test the model. It is shown that PreMevE‐MEO can provide hourly nowcasts with high model performance efficiency and high correlation with observations. This prototype PreMevE‐MEO model demonstrates the feasibility of making high‐fidelity predictions driven by observations from longstanding space infrastructure in MEO, thus has great potential of growing into an invaluable space weather operational warning tool.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

DeepAdversaries: examining the robustness of deep learning models for galaxy morphology classification

With increased adoption of supervised deep learning methods for work with cosmological survey data, the assessment of data perturbation effects (that can naturally occur in the data processing and analysis pipelines) and the development of methods that increase model robustness are increasingly important. In the context of morphological classification of galaxies, we study the effects of perturbations in imaging data. In particular, we examine the consequences of using neural networks when training on baseline data and testing on perturbed data. We consider perturbations associated with two primary sources: (a) increased observational noise as represented by higher levels of Poisson noise and (b) data processing noise incurred by steps such as image compression or telescope errors as represented by one-pixel adversarial attacks. We also test the efficacy of domain adaptation techniques in mitigating the perturbation-driven errors. We use classification accuracy, latent space visualizations, and latent space distance to assess model robustness in the face of these perturbations. For deep learning models without domain adaptation, we find that processing pixel-level errors easily flip the classification into an incorrect class and that higher observational noise makes the model trained on low-noise data unable to classify galaxy morphologies. On the other hand, we show that training with domain adaptation improves model robustness and mitigates the effects of these perturbations, improving the classification accuracy up to 23% on data with higher observational noise. Domain adaptation also increases up to a factor of ${\approx}2.3$ the latent space distance between the baseline and the incorrectly classified one-pixel perturbed image, making the model more robust to inadvertent perturbations. Successful development and implementation of methods that increase model robustness in astronomical survey pipelines will help pave the way for many more uses of deep learning for astronomy.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Stanford Thermal Earth Model for the Conterminous United States

Provided here are various forms of the Stanford Thermal Earth Model, as well as the data and methods used for its creation. The predictions produced by this model were visualized in two-dimensional spatial maps across the modeled depths (0-7 km) for the conterminous United States. The thermal earth model is made available as an application programming interface (API) and as feature layers on ArcGIS, which are both provided via links below. A data-driven spatial interpolation algorithm based on physics-informed graph neural networks was used to develop these national temperature-at-depth maps. The model satisfied the three-dimensional heat conduction law by predicting subsurface temperature, surface heat flow, and rock thermal conductivity. Many physical quantities, including bottomhole temperature, depth, geographic coordinates, elevation, sediment thickness, magnetic anomaly, gravity anomaly, gamma-ray flux of radioactive elements, seismicity, and electric conductivity were used as model inputs. Surface heat flow, temperature, and thermal conductivity predictions were constructed for depths of 0-7 km at an interval of 1 km with spatial resolution of 18 km2 per grid cell. The model showed superior temperature, surface heat flow and thermal conductivity mean absolute errors of 4.8C, 8.1 mW/m2 and 0.07 W/(C-m), respectively..

15 GEOTHERMAL ENERGY↗

DOE OSTI2023

Exploring NaCl-PuCl 3 molten salts with machine learning interatomic potentials and graph theory

Actinide molten salts are the basis of the liquid fuels used in molten salt reactors. Due to the inherent difficulties associated with high temperature and hazardous conditions, experimental investigations of fundamental properties of these materials are usually challenging. In this work, we describe the structure and transport of NaCl-PuCl 3 mixtures using computational techniques. Three compositions were considered (16, 25, and 36 mol% PuCl 3 ) over a temperature range (730 – 1257K) using ab initio molecular dynamics, which provided the necessary data sets for training machine learned interatomic potentials. Further, molecular dynamics simulations based on these potentials were then used to determine structure and transport properties. A substantial change was noted in the structure factor when increasing the PuCl 3 content from 25 to 36 mol%. This change is linked to the aggregation of larger Pu 3+ clusters. In addition, the similarity of the atomic environments of metal cations in molten salt systems to their solid states counterparts was investigated using an unsupervised learning technique. Finally, graph theory was employed to explore the structure and size of actinide networks. Consistent with the structure factor, a dense Pu 3+ intermolecular structure is observed within the 36 mol% PuCl 3 mixture. The structure of cation-cation inter-junctions is also discussed. In all cases, the diffusion of Pu 3+ is significantly lower than that of Na + and Cl - .

36 MATERIALS SCIENCE↗

DOE OSTI2025

Learning PDFs through interpretable latent representations in Mellin space

Representing the parton distribution functions (PDFs) of the proton and other hadrons through flexible, high-fidelity parametrizations has been a long-standing goal of particle physics phenomenology. This is particularly true since the chosen parametrization methodology can play an influential role in the ultimate PDF uncertainties as extracted in QCD global analyses; these, in turn, are often determinative of the reach of experiments at the LHC and other facilities to nonstandard physics, including at large 𝑥, where parametrization effects can be significant. In this study, we explore a series of encoder-decoder machine-learning (ML) models with various neural-network topologies as efficient means of reconstructing PDFs from meaningful information stored in an interpretable latent space. Given recent effort to pioneer synergies between QCD analyses and lattice-gauge calculations, we formulate a latent representation based on the behavior of PDFs in Mellin space, i.e., their integrated moments, and test the ability of various models to decode PDFs from this information faithfully. We introduce a numerical package, PDFdecoder, which implements several encoder-decoder models to reconstruct PDFs with high fidelity and use this end-to-end tool to explore how such neural-network-based models might connect PDF parametrizations to underlying properties like their Mellin moments. We additionally dissect patterns of learned correlations between encoded Mellin moments and reconstructed PDFs that suggest opportunities for further improvements to ML-based approaches to PDF parametrizations and uncertainty quantification.

Machine learning↗

DOE OSTI2022

Equation‐Free Surrogate Modeling of Geophysical Flows at the Intersection of Machine Learning and Data Assimilation

Abstract There is a growing interest in developing data‐driven reduced‐order models for atmospheric and oceanic flows that are trained on data obtained either from high‐resolution simulations or satellite observations. The data‐driven models are non‐intrusive in nature and offer significant computational savings compared to large‐scale numerical models. These low‐dimensional models can be utilized to reduce the computational burden of generating forecasts and estimating model uncertainty without losing the key information needed for data assimilation (DA) to produce accurate state estimates. This paper aims at exploring an equation‐free surrogate modeling approach at the intersection of machine learning and DA in Earth system modeling. With this objective, we introduce an end‐to‐end non‐intrusive reduced‐order modeling (NIROM) framework equipped with contributions in modal decomposition, time series prediction, optimal sensor placement, and sequential DA. Specifically, we use proper orthogonal decomposition (POD) to identify the dominant structures of the flow, and a long short‐term memory network to model the dynamics of the POD modes. The NIROM is integrated within the deterministic ensemble Kalman filter (DEnKF) to incorporate sparse and noisy observations at optimal sensor locations obtained through QR pivoting. The feasibility and the benefit of the proposed framework are demonstrated for the NOAA Optimum Interpolation Sea Surface Temperature (SST) V2 data set. Our results indicate that the NIROM is stable for long‐term forecasting and can model dynamics of SST with a reasonable level of accuracy. Furthermore, the prediction accuracy of the NIROM gets improved by almost one order of magnitude by the DEnKF algorithm.

Pawar, Suraj↗

DOE OSTI2020

Automated detection of part quality during two-photon lithography via deep learning

Two-photon lithography (TPL) is an additive manufacturing technique for fabricating three-dimensional objects with nanoscale features. A main challenge of TPL is the routine and labor-intensive task of finding suitable light dosage parameters, i.e. writing speed and laser intensity that induce photo-polymerization within a wide variety of candidate photo-curing polymers. Another challenge is the monitoring required during fabrication. In this work, we apply machine learning (ML) models to accelerate the process of identifying optimal light dosage parameters and automate the detection of part quality. We curate TPL videos of different parts fabricated under a range of light dosage parameters using different resins and train spatial-temporal ML models on this data. Our results show that ML models can detect TPL part quality with a 95.1% accuracy in milliseconds. We also evaluate classification failures and identify two operating modes: parameter optimization and part quality detection. Last but not least, we publicly release this labelled dataset so that it may serve as a useful benchmark to the community. Our approach to process optimization and part quality detection addresses important aspects of TPL industrialization, is applicable beyond TPL and should benefit other additive manufacturing techniques with similar barriers to operating at industrial scale.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Machine learning and deep learning for mineralogy interpretation and CO 2 saturation estimation in geological carbon Storage: A case study in the Illinois Basin

Carbon capture and storage (CCS) is a promising approach to simultaneously maintaining energy security and reducing carbon dioxide (CO 2 ) emissions under the current energy portfolio that is dominated by fossil fuel energy. Pre-injection formation characterization and post-injection CO 2 monitoring are two critical tasks to guarantee storage efficiency in CCS. The CCS projects in the Illinois Basin, the first large-scale CO 2 injection into saline aquifers in the United States, employed conventional and the latest pulsed neutron logging (PNL) tools for mineralogy interpretation and CO 2 saturation estimation, which provide valuable references for future CCS projects. Because of the inherent fuzziness of petrophysical measurements and complex subsurface heterogeneity, interpreting well-logging data is time-consuming, and its accuracy can be user-biased. In recent years, data-driven methods have been widely used to capture the non-linear patterns between input features and interpretation results. This work applied and evaluated four commonly used machine learning (ML) models, including ridge regression (RR), random forest (RF), gradient boosting regression (GBR), support vector regression (SVR), and one deep learning (DL) model, the artificial neural network (ANN). We optimized the hyperparameters of the four ML models and the DL model using the simulated annealing algorithm and the grid search strategy, respectively. The input features of the mineralogy interpretation models were eleven conventional well-logging parameters, and the label data (i.e., ground truth) were the porosity and volumetric fractions of six minerals, including quartz, feldspar, dolomite, calcite, clay, and iron minerals. The results demonstrated that the GBR and RF models were superior in predicting volumetric fractions of minerals and porosity; label data with low coefficient of variation (CV) values tended to yield better performance. For CO 2 saturation estimation, the RF was the best-performing model, followed by SVR, ANN, GBR, and RR. Furthermore, we conducted feature importance ranking using the permutation importance algorithm and found that the formation sigma and well pressure were the most important features in this study. In conclusion, the study of CCS projects in the Illinois Basin bridges the gap between the limited knowledge and understanding of geological carbon storage and the increasing demand for reliable, cost-effective, and sustainable energy solutions.

58 GEOSCIENCES↗

DOE OSTI2024

Machine Learning Inference of Random Medium Properties

Earth materials are heterogeneous across a range of spatial scales, but the resolvability of small structures is limited by sparse data coverage, noise, bandlimitedness, and other difficulties. In practice, heterogeneities below a certain size cannot be recovered from seismic data except through statistical medium descriptions, which even then can be difficult to uniquely determine. To improve the characterization of such heterogeneities, we develop a novel supervised machine learning (ML) model that provides insight about the recoverability of statistical medium properties from elastic waveform data and succeeds despite cycle-skipping and other challenges well known from elastic waveform inversion. We demonstrate the approach using random media generated by superimposing self-affine random variations on homogeneous and layered background structures. After training on sparsely-recorded, high-frequency waveforms from hundreds of different random medium realizations, we show the ability of our ML model to recover correlation lengths and other statistical properties of interest to near-surface and crustal seismology, among other fields. For frequency passbands and spatial offsets encountered in seismology, Gaussian correlation lengths and the amplitude of the random variations relative to the background model are recovered even in challenging scenarios involving unknown medium parameters, complex crustal structures, and low signal-to-noise ratio. In comparison, von Kármán correlation lengths, which are related to larger-wavelength variations of the medium than Gaussian correlation lengths, are not as well recovered. These results provide one of the first and most systematic investigations of the recoverability of statistical properties of heterogeneities below the resolution limit of deterministic seismic tomography, and suggest practical ML strategies for high-frequency waveform seismology.

58 GEOSCIENCES↗

DOE OSTI2020

A machine learning approach to fracture mechanics problems

Analytical and empirical solutions to engineering problems are usually preferred because of their convenience in applications. However, they are not always accessible in complex problems. A new class of solutions, based on machine learning (ML) models such as regression trees and neural networks (NNs), are proposed and their feasibility and value are demonstrated through the analysis of fracture toughness measurements. It is found that both solutions based on regression trees and NNs can provide accurate results for the specific problem, but NN-based solutions outperform regression-tree-based solutions in terms of their simplicity. Here this example demonstrates that ML solutions are a major improvement over analytical and empirical solutions in terms of both reliable functionality and rapid deployment. When analytical solutions are not available, the use of ML solutions can overcome the limitations of empirical solutions and substantially change the way that engineering problems are solved.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Ensemble transfer learning for the prediction of anti-cancer drug response

Abstract Transfer learning, which transfers patterns learned on a source dataset to a related target dataset for constructing prediction models, has been shown effective in many applications. In this paper, we investigate whether transfer learning can be used to improve the performance of anti-cancer drug response prediction models. Previous transfer learning studies for drug response prediction focused on building models to predict the response of tumor cells to a specific drug treatment. We target the more challenging task of building general prediction models that can make predictions for both new tumor cells and new drugs. Uniquely, we investigate the power of transfer learning for three drug response prediction applications including drug repurposing, precision oncology, and new drug development, through different data partition schemes in cross-validation. We extend the classic transfer learning framework through ensemble and demonstrate its general utility with three representative prediction algorithms including a gradient boosting model and two deep neural networks. The ensemble transfer learning framework is tested on benchmark in vitro drug screening datasets. The results demonstrate that our framework broadly improves the prediction performance in all three drug response prediction applications with all three prediction algorithms.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2020

Importance learning estimator for the site-averaged turnover frequency of a disordered solid catalyst

For disordered catalysts such as atomically dispersed “single-atom” metals on amorphous silica, the active sites inherit different properties from their quenched-disordered local environments. The observed kinetics are site-averages, typically dominated by a small fraction of highly active sites. Standard sampling methods require expensive ab initio calculations at an intractable number of sites to converge on the siteaveraged kinetics. We present a new method that efficiently estimates the site-averaged turnover frequency (TOF). The new estimator uses the same importance learning algorithm [Vandervelden et al., React. Chem. Eng. 5, 77 (2020)] that we previously used to compute the siteaveraged activation energy. We demonstrate the method by computing the site-averaged TOF for a simple disordered lattice model of an amorphous catalyst. The results show that with the importance learning algorithm, the site-averaged TOF and activation energy can now be obtained concurrently with orders of magnitude reduction in required ab initio calculations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Learning nuclear cross sections across the chart of nuclides with graph neural networks

We explore the use of deep learning techniques to learn how nuclear cross sections change as we add or remove protons and neutrons. As a proof of principle, we focus on the neutron-induced reactions in the fast energy regime. Our approach follows a two-stage learning framework. First, we apply representation learning to encode cross section data into a latent space using either variational autoencoders (VAEs) or implicit neural representations (INRs). Then, we train graph neural networks (GNNs) on the resulting embeddings to predict missing values across the nuclear chart by leveraging the topological structure of neighboring isotopes. We demonstrate accurate cross section predictions within a 9 × 9 block of missing nuclei. We also find that the optimal GNN training strategy depends on the type of latent representation used, with VAE embeddings performing best under end-to-end optimization in the original space, while INR embeddings achieve better results when the GNN is trained only in the latent space. Furthermore, using clustering algorithms, we map groups of latent vectors into regions of the nuclear chart and show that VAEs and INRs can discover some of the neutron magic numbers. These findings suggest that deep-learning models based on the representation encoding of cross sections combined with graph neural networks hold significant potential in augmenting nuclear theory models, e.g., by providing reliable estimates of covariances of cross sections, including cross-material covariances.

Machine learning↗