Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Surrogate Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Aircraft in-situ measurements from SOCRATES constrain the anthropogenic perturbations of cloud droplet number

Aerosol-cloud interactions (ACI) in warm clouds alter reflected shortwave radiation by influencing cloud microphysical and macrophysical properties. The variable of state controlling ACI is the cloud droplet number concentration (N d ). Here, we examine the perturbations in N d due to anthropogenic aerosols (ΔN d, PD-PI ) using a perturbed parameter ensemble (PPE) hosted in the sixth Community Atmosphere Model (CAM6). Surrogate models are created for the CAM6 PPE outputs and are used to generate 1 million model variants of CAM6 by sampling 45 sources of parameter uncertainty. The range of uncertain physical parameters related to ACI are constrained with observations of aerosol and cloud properties from SOCRATES. The likely range of uncertain parameters and the associated range of ΔN d, PD-PI are more strongly constrained with observations of N d relative to observations of cloud condensation nuclei. We conduct sensitivity tests of how constraints on ΔN d, PD-PI are affected by systematic uncertainties in observations and our limitations in our surrogate models created for CAM6 PPE outputs. Based on this, we provide guidance on the impact of reducing systematic uncertainty in airborne microphysical observations and in surrogate models.

Song, Ci [Univ. of Wyoming, Laramie, WY (United St↗
DOE OSTI2025

A comparison of surrogate constitutive models for viscoplastic creep simulation of HT-9 steel

Mechanistic microstructure-informed constitutive models for the mechanical response of polycrystals are a cornerstone of computational materials science. However, as these models become increasingly more complex – often involving coupled differential equations describing the effect of specific deformation modes – their associated computational costs can become prohibitive, particularly in optimization or uncertainty quantification tasks that require numerous model evaluations. To address this challenge, surrogate constitutive models that balance accuracy and computational efficiency are highly desirable. Data-driven surrogate models, that learn the constitutive relation directly from data, have emerged as a promising solution. In this work, we develop two local surrogate models for the viscoplastic response of a steel: a piecewise response surface method and a mixture of experts model. These surrogates are designed to adapt to complex material behavior, which may vary with material parameters or operating conditions. The surrogate constitutive models are applied to creep simulations of HT-9 steel, an alloy of considerable interest to the nuclear energy sector due to its high tolerance to radiation damage, using training data generated from viscoplastic self-consistent (VPSC) simulations. In conclusion, we define a set of test metrics to numerically assess the accuracy of our surrogate models for predicting viscoplastic material behavior, and show that the mixture of experts model outperforms the piecewise response surface method in terms of accuracy.

36 MATERIALS SCIENCE↗
DOE OSTI2024

On the transferability of residence time distributions in two 10-km long river sections with similar hydromorphic units

Quantifying hydrologic exchange fluxes (HEFs) at the stream-groundwater interface and their residence time distributions (RTDs) in the subsurface are important for managing the water quality and ecosystem health in dynamic river corridors. However, direct simulating high-spatial resolution HEFs and RTDs can be time-consuming, especially for watershed-scale modeling. Efficient surrogate models linking RTDs to hydromorphic units (HUs) can be alternatives for simulating RTDs in large-scale models. A common concern of these surrogate models, though, is the transferability of the relationship between the RTDs and HUs from one river corridor to another. To address this issue, this work evaluates the HEFs and resulting RTD-HU relationships for two 10-km long river corridors along the Columbia River leveraging a one-way coupled three-dimensional transient surface-subsurface water transport modeling framework we previously developed. Applying such a framework at the two river corridors with similar HUs allows for quantitative comparisons of HEFs and RTDs using both statistical tests and machine learning classification models. Finally, our comparison shows that the similarity and transferability of the RTD-HU relationship is very low for the two investigated river sections, which suggests that devising a general algorithm to estimate RTDs based solely on surface water hydrodynamics and short-distance river channel topography data, as well as HU classification, might be nearly impossible.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Data and scripts associated with a manuscript on residence time distribution simulation in two 10-kilometer long river sections

This data package is associated with the publication “On the Transferability of Residence Time Distributions in Two 10-km Long River Sections with Similar Hydromorphic Units” submitted to the Journal of Hydrology (Bao et al. 2024).Quantifying hydrologic exchange fluxes (HEFs) at the stream-groundwater interface, along with their residence time distributions (RTDs) in the subsurface, is crucial for managing water quality and ecosystem health in dynamic river corridors. However, directly simulating high-spatial resolution HEFs and RTDs can be a time-consuming process, particularly for watershed-scale modeling. Efficient surrogate models that link RTDs to hydromorphic units (HUs) may serve as alternatives for simulating RTDs in large-scale models. One common concern with these surrogate models, however, is the transferability of the relationship between the RTDs and HUs from one river corridor to another. To address this, we evaluated the HEFs and the resulting RTD-HU relationships for two 10-kilometer-long river corridors along the Columbia River, using a one-way coupled three-dimensional transient surface-subsurface water transport modeling framework that we previously developed. Applying this framework to the two river corridors with similar HUs allows for quantitative comparisons of HEFs and RTDs using both statistical tests and machine learning classification models. This data package includes the model inputs files and the simulation results data. This data package contains 10 folders. The modeling simulation results data are in the folders 100H_pt_data and 300area_pt_data, for the study domain Hanford 100H and 300 area respectively. The remaining eight folders contain the scripts and data to generate the manuscript figures. The file-level metadata file (Bao_2024_Residence_Time_Distribution _flmd.csv) includes a list of all files contained in this data package and descriptions for each. The data dictionary file (Bao_2024_Residence_Time_Distribution _dd.csv) includes column header definitions and units of all tabular files.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Accelerating Scientific Simulations with Bi-Fidelity Weighted Transfer Learning

High-fidelity modeling is an essential design tool for many engineering applications. However, for complex systems, computational cost can be a limiting factor. Analyzing parameter sensitivity, uncertainty quantification, and design optimization require many model evaluations. Surrogate models are often used to develop the relationship between model parameters and quantities of interest. However, in the case of complex systems, surrogate models require several degrees of freedom and, thus, a large number of data points to determine the correct dependencies. For many applications, this may be prohibitively expensive. The reduction of computational requirements can be achieved by leveraging low-fidelity models. Low-fidelity models represent the system at a coarser resolution with the advantage of computational efficiency. Therefore, a bi-fidelity modeling paradigm, which augments the accuracy of a low-fidelity model in a computationally efficient manner by invoking limited runs of a high-fidelity model, can be leveraged to sufficiently balance the accuracy and computational requirements. In this work, a bi-fidelity weighted transfer learning method using neural networks was applied to a computational fluid dynamics heat transfer modeling problem. The transfer learning advantage was investigated as a function of hyperparameters. Our main finding is that the use of a bi-fidelity modeling paradigm achieves accuracy close to that of a high-fidelity Gaussian process model while significantly reducing computational cost. The bi-fidelity model achieves comparable performance with 90 high-fidelity samples-that is, 60% less than the samples needed to achieve similar accuracy without the use of bi-fidelity modeling,

Borowiec, Katarzyna↗
DOE OSTI2024

Rapid Inverse Parameter Inference Using Physics-Informed Neural Network

As Li-ion batteries become more essential in today's economy, tools need to be developed to accurately and rapidly diagnose a battery's internal state-of-health. Using a Li-ion battery's (high-rate) voltage response, it is proposed to determine a battery's internal state through Bayesian calibration. However, Bayesian calibration is notoriously slow and requires thousands of model runs. To accelerate parameter inference using Bayesian calibration, a surrogate model is developed to replace the underlying physics-based Li-ion model. Developing a surrogate model for rapid Bayesian calibration analysis is discussed for both the single particle model (SPM) and the pseudo two-dimensional (P2D) model. Surrogate models are constructed using physics-informed neural networks (PINNs) that encode the influence of internal properties on observed voltage responses. In practice, a neural network can be trained by: 1) using simulation results of the physics-based model (i.e., a data-loss approach); 2) using the residuals of the governing equations themselves (i.e., a physics-loss approach); or 3) using a combination of simulation results and governing equation residuals. In the present work, PINNs are developed using a variety of training losses and neural network architectures. In this analysis, it is shown that a PINN surrogate model can be reliably trained with only physics-informed loss. However, using a coupled data-informed and physics-loss approach produced the most accurate PINNs.

Bayesian calibration↗
DOE OSTI2025

Deep Learning–Assisted Multiobjective Optimization of Geological CO 2 Storage Performance under Geomechanical Risks

In geological CO 2 storage, designing the optimal well control strategy for CO 2 injection to maximize CO 2 storage while minimizing the associated geomechanical risks is not trivial. This challenge arises due to pressure buildup, CO 2 plume migration, the highly nonlinear nature of geomechanical responses to rock-fluid interaction, and the high computational cost associated with coupled flow and geomechanics simulations. In this paper, we introduce a novel optimization framework to address these challenges. The optimization problem is formulated as follows: maximize total CO 2 storage while minimizing geomechanical risks by adjusting the injection schedules within bounded constraints. The geomechanical risks are primarily driven by injection-induced pressure build-up, which is characterized by ground displacement and the induced microseismicity. We used the Fourier neural operator (FNO)-based deep learning model to construct surrogate models, replacing the time-consuming coupled flow and geomechanics simulations for evaluating the aforementioned objective functions. The developed surrogate models have been incorporated into a multiobjective optimization framework through a genetic algorithm to reduce the computational burden. The proposed optimization framework reduces the computational cost from approximately 2,400 hours, when using objective function evaluations based on physics-based simulations, to around 20 minutes. A set of Pareto-optimal solutions of the proposed workflow yields nontrivial optimal decisions, reducing the microseismicity potential and the vertical displacement. This Pareto front highlights the optimal trade-offs between CO 2 storage amount, safety, and ground displacement, emphasizing the need for careful optimization and management of injection strategies to achieve a balanced outcome. The novelty of this work is twofold. First, we demonstrate the importance of incorporating the minimization of the geomechanical risks as objective functions into the CO 2 storage optimization workflow to mitigate the potential risk of induced microseismicity and ground displacement. Second, we leverage the FNO-based surrogate models to optimize a real-field CO 2 storage operation.

42 ENGINEERING↗
DOE OSTI2022

Calibrating hypersonic turbulence flow models with the HIFiRE-1 experiment using data-driven machine-learned models.

In this paper we study the efficacy of combining machine-learning methods with projection-based model reduction techniques for creating data-driven surrogate models of computationally expensive, high-fidelity physics models. Such surrogate models are essential for many-query applications e.g., engineering design optimization and parameter estimation, where it is necessary to invoke the high-fidelity model sequentially, many times. Surrogate models are usually constructed for individual scalar quantities. However there are scenarios where a spatially varying field needs to be modeled as a function of the model’s input parameters. Here we develop a method to do so, using projections to represent spatial variability while a machine-learned model captures the dependence of the model’s response on the inputs. The method is demonstrated on modeling the heat flux and pressure on the surface of the HIFiRE-1 geometry in a Mach 7.16 turbulent flow. The surrogate model is then used to perform Bayesian estimation of freestream conditions and parameters of the SST (Shear Stress Transport) turbulence model embedded in the high-fidelity (Reynolds-Averaged Navier–Stokes) flow simulator, using shock-tunnel data. The paper provides the first-ever Bayesian calibration of a turbulence model for complex hypersonic turbulent flows. We find that the primary issues in estimating the SST model parameters are the limited information content of the heat flux and pressure measurements and the large model-form error encountered in a certain part of the flow.

42 ENGINEERING↗
DOE OSTI2023

Conditional Karhunen–Loève regression model with Basis Adaptation for high-dimensional problems: Uncertainty quantification and inverse modeling

Here, we propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems as a function of the systems’ spatially heterogeneous parameter fields, with applications to uncertainty quantification and parameter estimation in high-dimensional problems. Practitioners often formulate finite-dimensional representations of spatially heterogeneous parameter fields using truncated unconditional Karhunen–Loève expansions (KLEs) for a certain choice of unconditional covariance kernel and construct surrogate models of the observable response with respect to the KLE coefficients. When direct measurements of the parameter fields are available, we propose improving the accuracy of these surrogate models by representing the parameter fields via conditional Karhunen-Loève expansions (CKLEs). CKLEs are constructed by conditioning the covariance kernel of the unconditional expansion on the direct measurements of the parameter field via Gaussian process regression, and then truncating the corresponding KLE. We apply the proposed methodology to constructing surrogate models via the Basis Adaptation (BA) method of the stationary hydraulic head response, measured at spatially discrete observation locations, of a groundwater flow model of the Hanford Site, as a function of the 1000-dimensional representation of the model’s log-transmissivity field. We find that BA surrogate models of the hydraulic head based on CKLEs are more accurate than BA surrogate models based on unconditional expansions for forward uncertainty quantification tasks. Furthermore, we find that inverse estimates of the hydraulic transmissivity field computed using CKLE-based BA surrogate models are more accurate than those computed using unconditional BA surrogate models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Machine learning for surrogate process models of bioproduction pathways

Technoeconomic analysis and life-cycle assessment are critical to guiding and prioritizing bench-scale experiments and to evaluating economic and environmental performance of biofuel or biochemical production processes at scale. Traditionally, commercial process simulation tools have been used to develop detailed models for these purposes. However, developing and running such models can be costly and computationally intensive, which limits the degree to which they can be shared and reproduced in the broader research community. This study evaluates the potential of an automated machine learning approach to develop surrogate models based on conventional process simulation models. The analysis focuses on several high-value biofuels and bioproducts for which pathways of production from biomass feedstocks have been well-established. The results demonstrate that surrogate models can be an accurate and effective tool for approximating the cost, mass and energy balance outputs of more complex process simulations at a fraction of the computational expense.

09 BIOMASS FUELS↗
DOE OSTI2022

Application of machine learning and artificial intelligence to extend EFIT equilibrium reconstruction

Recent progress in the application of machine learning (ML)/artificial intelligence (AI) algorithms to improve the Equilibrium Fitting (EFIT) code equilibrium reconstruction for fusion data analysis applications is presented. A device-independent portable core equilibrium solver capable of computing or reconstructing equilibrium for different tokamaks has been created to facilitate adaptation of ML/AI algorithms. A large EFIT database comprising of DIII-D magnetic, motional Stark effect, and kinetic reconstruction data has been generated for developments of EFIT model-order-reduction (MOR) surrogate models to reconstruct approximate equilibrium solutions. Furthermore, a neural-network MOR surrogate model has been successfully trained and tested using the magnetically reconstructed datasets with encouraging results. Other progress includes developments of a Gaussian process Bayesian framework that can adapt its many hyperparameters to improve processing of experimental input data and a 3D perturbed equilibrium database from toroidal full magnetohydrodynamic linear response modeling using the Magnetohydrodynamic Resistive Spectrum - Feedback (MARS-F) code for developments of 3D-MOR surrogate models.

Gaussian process↗
DOE OSTI2026

JetGP: A derivative enhanced Gaussian process library

Derivative enhanced Gaussian Processes (DEGPs) can significantly improve surrogate model accuracy over standard Gaussian Process (GP) formulations by incorporating derivative information. However, standard implementations scale poorly with dimension, limiting their use in high dimensional engineering problems. JetGP is a Python framework that unifies existing derivative enhanced GP methodologies into a single library and extends them to support arbitrary order derivative information. The library implements four complementary formulations: standard derivative enhanced Gaussian Processes (DEGP), directional DEGP (DDEGP), generalized directional DEGP (GDDEGP), and weighted DEGP (WDEGP). By unifying these approaches in a consistent interface with robust numerical implementations, JetGP enables practitioners to balance predictive accuracy and computational efficiency for high dimensional optimization, uncertainty quantification, and sensitivity analysis in engineering design.

Derivative enhanced Gaussian process↗
DOE OSTI2024

Non-smooth Bayesian optimization in tuning scientific applications

Tuning algorithmic parameters to optimize the performance of large, complicated computational codes is an important problem involving finding the optima and identifying regimes defined by non-smooth boundaries in black-box functions. Within the Bayesian optimization framework, the Gaussian process surrogate model produces smooth mean functions, but functions in the tuning problem are often non-smooth, which is exacerbated by the fact that we usually have limited sequential samples from the black-box function. Here, motivated by these issues encountered in tuning, we propose a novel Gaussian process model called a clustered Gaussian process (cGP), where the components are dynamically updated by clustering. In our studies, the performance of cGP can be better than stationary GPs in nearly 90% of the experiments and better than non-stationary GPs in nearly 70% of the repeated experiments while requiring less computational cost. cGP provides a novel approach for dynamic GP, computes more efficiently than recursive partitioning, and discovers non-smoothness regimes. We provide extensive experiments including high-performance computing (HPC) and industrial simulation functions to show the effectiveness of our methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Machine Learning for First Principles Calculations of Material Properties for Ferromagnetic Materials

The investigation of finite temperature properties using Monte-Carlo (MC) methods requires a large number of evaluations of the system’s Hamiltonian to sample the phase space needed to obtain physical observables as function of temperature. DFT calculations can provide accurate evaluations of the energies, but they are too computationally expensive for routine simulations. To circumvent this problem, machine-learning (ML) based surrogate models have been developed and implemented on high-performance computing (HPC) architectures. In this paper, we describe two ML methods (linear mixing model and HydraGNN) as surrogates for first principles density functional theory (DFT) calculations with classical MC simulations. These two surrogate models are used to learn the dependence of target physical properties from complex compositions and interactions of their constituents. We present the predictive performance of these two surrogate models with respect to their complexity while avoiding the danger of overfitting the model. An important aspect of our approach is the periodic retraining with newly generated first principles data based on the progressive exploration of the system’s phase space by the MC simulation. The numerical results show that HydraGNN model attains superior predictive performance compared to the linear mixing model for magnetic alloy materials.

Eisenbach, Markus↗
DOE OSTI2025

Deep Learning for Subsurface Flow: A Comparative Study of U‐Net, Fourier Neural Operators, and Transformers in Underground Hydrogen Storage

Subsurface flow research is essential for the sustainable management of natural resources and the environment. Deep learning (DL) has significantly advanced this field by developing efficient and accurate surrogate models to replace computationally expensive physics‐based simulations. These surrogate models are commonly used to predict the spatiotemporal evolution of state variables, such as gas saturation and reservoir pressure, in heterogeneous geological formations. Despite the various DL models applied to this task, there is a lack of studies systematically comparing their performance. This absence of comparative analysis leads to somewhat arbitrary DL model selection in subsurface flow research, resulting in suboptimal performance and potentially inaccurate predictions. To bridge this gap, we conduct a systematic comparison study of three popular DL architectures—U‐Net, Fourier Neural Operators (FNO), and Segmentation Transformer (SETR)—in surrogate modeling of underground hydrogen storage (UHS). We focus on UHS due to its promise of enhancing clean energy resilience and its cyclic operational conditions that represent common scenarios in various subsurface applications. We evaluate the models based on accuracy, training cost, and inference speed. The comparison shows that U‐Net achieves the highest accuracy, followed by SETR and FNO. Despite its lower accuracy, FNO has the highest inference speed. SETR offers competitive accuracy with the least training memory usage, demonstrating the potential of transformers in learning subsurface flow. Our results provide guidance for selecting DL models for surrogate modeling in a wide range of subsurface flow problems.

42 ENGINEERING↗
DOE OSTI2020

UQpy: A general purpose Python package and development environment for uncertainty quantification

In this paper, we present the UQpy software toolbox, an open-source Python package for general uncertainty quantification (UQ) in mathematical and physical systems. The software serves as both a user-ready toolbox that includes many of the latest methods for UQ in computational modeling and a convenient development environment for Python programmers advancing the field of UQ. The paper presents an introduction to the software's architecture and existing capabilities, divided in the code in a set of modules centered around different UQ tasks such as sampling methods, generation of random processes and random fields, probabilistic inverse modeling, reliability analysis, surrogate modeling, and active learning. The paper also highlights the importance of the RunModel module, which is used to drive simulations in the uncertainty analyses performed in UQpy. This module conveniently allows the user to define computational models directly in Python, or to run simulations from a third-party software in serial or in parallel. To illustrate the various capabilities, two examples are tracked throughout the paper and analyzed repeatedly for various UQ tasks. The first is a Python model solving a nonlinear structural dynamics problem, used to illustrate UQpy's capabilities in sampling and forward propagation of high dimensional random vectors (stochastic processes), and probabilistic inference. The second model is a third-party Abaqus finite element model solving the thermomechanical response of a beam structure. This example is used to illustrate UQpy's capabilities in variance reduction sampling techniques, reliability analysis, surrogate modeling and active learning techniques.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Physics-Informed Machine Learning Model for Ceramic Matrix Composite Creep

A physics-informed recurrent neural network (RNN) based surrogate model is developed to emulate the nonlinear, time-dependent constitutive behavior of ceramic matrix composites (CMCs) driven by matrix damage and constituent creep at the microscale. Physics-informed constraints are introduced into the surrogate model through regularization to ground the prediction in physics and improve its predictive capabilities. Training data is generated using the high-fidelity generalized method of cells (HFGMC) approach which calls appropriate creep and damage models for each of the constituents. This coupling permits simulating the nonlinear behavior of CMCs based on constituent response at the microscale along with microstructural features such as fiber and porosity volume fraction and fiber radius. The microscale repeating unit cell is loaded under creep fatigue conditions to replicate the material loading experienced in a turbine engine. Therefore, the RNN-based surrogate model is tasked with predicting, as a function of variable input stress sequence, temperature, and microstructural features, the resulting strain history response while satisfying physical constraints related to creep rate, isochoric inelastic deformation, and strain energy density. The trained surrogate model is shown to effectively match the strain history over quantified distributions of microstructural features and relevant loading regimes and temperatures. Neural network based surrogate models can offer efficient alternatives to running computationally intensive multiscale material models to simulate the nonlinear response of large structural models. Therefore, the presented work provides evidence towards the feasibility of developing, training, and running such models for CMCs with complex microstructures, nonlinear time-dependent material response, and under non-monotonic loading conditions.

ceramic matrix composites↗
DOE OSTI2020

ROM-Based Surrogate Systems Modeling of EBR-II

We report the System Analysis Module (SAM), developed and maintained by Argonne National Laboratory, is designed to provide whole-plant transient safety analysis capabilities for a number of advanced non-light water reactors, including sodium-cooled fast reactor (SFR), lead-cooled fast reactor (LFR), and molten salt reactor (MSR)/fluoride-salt-cooled high-temperature reactor (FHR) designs. SAM is primarily constructed as a systems-level analysis tool, with the potential to incorporate reduced order models from three-dimensional computational fluid dynamics (CFD) simulations to improve characterization of complex, multidimensional physics. It is recognized that the computational expense associated with CFD can be intractable for various engineering analyses, such as uncertainty quantification, inference, and design optimization. This paper explores the reducibility of a SAM model using recent advances in randomized linear algebra techniques, which attempt to find recurring patterns in the various realizations generated by a model after randomly perturbing all its input parameters. The reduction is described in terms of fewer degrees of freedom (DOFs), referred to as the active DOFs, for the model variables such as input model parameters and model responses. The results indicate that there is significant room for additional reduction that may be leveraged for additional computational gains when employing SAM for engineering-intensive analyses that require repeated model executions. Different from physics-based reduction approaches, the proposed approach allows one to estimate upper bounds on the reduction errors, which are rigorously developed in this work. Finally, different methods for surrogate model construction, such as regression and neural network-based training, are employed to correlate the input and output active DOFs, which are related back to the original variables using matrix-based linear transformations.

42 ENGINEERING↗