Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

“Which Projections Do I Use?” Strategies for Climate Model Ensemble Subset Selection Based on Regional Stakeholder Needs

Climate model (or earth system model) projections are increasingly used for climate adaptation planning and impact assessments. As part of this process, many end‐users evaluate a subset of downscaled climate projections without being aware of the implications of downscaling methodology for statistics or event outcomes. Approaches for determining a subset of global climate models to use often focus on values from the raw models, rather than from their downscaled counterparts, in other words assuming that the statistical distribution of the multi‐model ensemble does not change post downscaling. This study demonstrates that a downscaled ensemble will typically retain the change distribution as a raw ensemble, but individual models can differ dramatically post‐downscaling. We recommend that subset‐selection methods account for this possibility and that decision‐relevant downscaled climate projections provide proper descriptions of fitness‐for‐purpose and essential caveats, so that non‐specialists can interpret the results with an appropriate level of confidence.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Inferring safety critical events from vehicle kinematics in naturalistic driving environment: Application of deep learning Algorithms

Advances in sensing technology has enabled the collection of countless terabytes of second-by-second kinematics data. Such data provides opportunities for real-time monitoring of driving behavior and identification of safety critical events (SCEs) including crashes and near crashes. The concept of volatility is relevant in this context, which identifies instability and erratic variations in driving behavior prior to involvement in SCEs. This study utilized vehicle kinematics from a large-scale naturalistic driving data to develop a deep learning approach based on 1D convolutional neural networks (CNN) for inferring SCEs. The data are unique in the sense that such accurate pre-crash data at high fidelity are not available in traditional crash repositories. This study contributes to the literature by providing a first attempt at predicting responses to SCEs by developing deep learning-based CNN architectures using novel driving volatility based kinematic thresholds for a sample of 9553 events. The key contribution lies in developing a volatility-based CNN input layout that is acceptable to CNN schemes and represents the motion kinematics such as speed, acceleration and volatility measures. Several 1D-CNN architectures were developed using layers, numbers of convolutions, layer patterns, and kernels. Shallow and deep architectures were tested, revealing higher accuracy of shallow architectures in detecting SCEs. The optimal number of epochs were identified using an early stopping method while the CNN performance was improved by increasing the number of epochs. The ensemble CNN had the highest predictive accuracy of 95.6% for detection of crashes and near crashes, which was 2.5% higher than the optimal CNN using 20% hold out test data. The ensemble CNN also outperformed classical machine learning models and model performance reported in past studies on detection of SCEs. Finally, these results have implications for identification of safety hotspots and providing real-time alerts and warnings in connected and highly automated vehicle environment including society of automotive engineers levels 3–5.

42 ENGINEERING↗
DOE OSTI2022

Out of Distribution Detection with Neural Network Anchoring

This is code to reproduce and build on OOD detection from the paper "Out of Distribution Detection with Neural Network Anchoring". Our goal here is to exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection. Heteroscedasticity here refers to the fact that the optimal temperature parameter for each sample can be different, as opposed to conventional approaches that use the same value for the entire distribution. To enable this, we propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample, leading to state-of-the-art OOD detection performance across several benchmarks. Using NTK theory, we show that this temperature function estimate is closely linked to the epistemic uncertainty of the classifier, which explains its behavior. In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets, custom calibration objectives, or model ensembling. Through empirical studies with different OOD detection settings - far OOD, near OOD, and semantically coherent OOD - we establish a highly effective OOD detection approach.

Thiagarajan, Jayaraman↗
DOE OSTI2025

Ensemble Simulation Techniques and Fast Randomized Algorithms

The major goals of the project were to develop and analyze new ensemble simulation techniques, including trajectory stratification and preconditioned MCMC techniques, as well as develop fast numerical linear algebra techniques closely related to ensemble simulation ideas. The trajectory stratification techniques involve simulating in parallel short trajectory fragments of a Markov process confined to a specific region of space‐time and then patching together the statistics gathered to assemble estimates of very general dynamical properties. We have also developed this approach for rare event simulation and extended the techniques to applications requiring a more general framework (such as electronic structure calculations). The preconditioned MCMC techniques involve simulating multiple Markov chains in parallel and then using information from the ensemble to speed the mixing of each individual chain. The fast randomized linear algebra methods are motivated by the diffusion Monte Carlo technique, but are applicable to finding the dominant eigenvalue of (almost) general matrices. For most non‐negative matrices, the schemes result in an error (compared to the power method) that is constant in the dimension of the problem. For more general matrices, we see a very clear sublinear cost trend in computational tests.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

A stochastic covariance shrinkage approach to particle rejuvenation in the ensemble transform particle filter

Rejuvenation in particle filters is necessary to prevent the collapse of the weights when the number of particles is insufficient to properly sample the high-probability regions of the state space. Rejuvenation is often implemented in a heuristic manner by the addition of random noise that widens the support of the ensemble. This work aims at improving canonical rejuvenation methodology by the introduction of additional prior information obtained from climatological samples; the dynamical particles used for importance sampling are augmented with samples obtained from stochastic covariance shrinkage. A localized variant of the proposed method is developed. Numerical experiments with the Lorenz '63 model show that modified filters significantly improve the analyses for low dynamical ensemble sizes. Furthermore, localization experiments with the Lorenz '96 model show that the proposed methodology is extendable to larger systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

A new spatially distributed added value index for regional climate models: the EURO-CORDEX and the CORDEX-CORE highest resolution ensembles

The added value of using regional climate models (RCMs) to downscale data from general circulation models (GCMs) has often been questioned and researched. Although several studies have used different methods to identify (and in some cases quantify) the added value, there is still a need to find a general metric that quantifies the added value of any variable. This paper builds on past studies to propose a new metric of added value in the simulation of present-day climate which measures the difference in the probability density functions (PDFs) at each grid-cell between a model and an observation source, and then compares the results of the RCM and GCM in order to spatially compute the added value index. The same method is also adapted to quantify the climate change downscaling signal in a way that is consistent with the present-day metric. These new metrics are tested on the daily precipitation output from the EURO-CORDEX and CORDEX-CORE projection ensembles and reveal an overall positive added value of RCMs, especially at the tail-end of the distribution. Higher added value is obtained in areas of complex topography and coast-lines, as well as in tropical regions. Areas with large added value in present-day climate are consistent with areas of significant climate change downscaling signal in the RCP 8.5 far future simulations, and when the analysis is repeated at a low-resolution. The use of different resolution observations shows that the added value tends to decrease when models are compared to low-resolution observation datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Vanilloid-dependent TRPV1 opening trajectory from cryo EM ensemble analysis

Single particle cryo-EM often yields multiple protein conformations within a single dataset, but experimentally deducing the temporal relationship of these conformers within a conformational trajectory is not trivial. Here, we use thermal titration methods and cryo-EM in an attempt to obtain temporal resolution of the conformational trajectory of the vanilloid receptor TRPV1 with resiniferatoxin (RTx) bound. Based on our cryo-EM ensemble analysis, RTx binding to TRPV1 appears to induce intracellular gate opening first, followed by selectivity filter dilation, then pore loop rearrangement to reach the final open state. This apparent conformational wave likely arises from the concerted, stepwise, additive structural changes of TRPV1 over many subdomains. Greater understanding of the RTx-mediated long-range allostery of TRPV1 could help further the therapeutic potential of RTx, which is a promising drug candidate for pain relief associated with advanced cancer or knee arthritis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

GenAI4UQ: A software for forward and inverse uncertainty quantification using conditional generative AI

We introduce GenAI4UQ, a software package for forward and inverse uncertainty quantification in model calibration, parameter estimation, and ensemble forecasting. GenAI4UQ leverages a generative AI-based conditional modeling framework to address limitations of traditional inverse modeling techniques, such as Markov Chain Monte Carlo (MCMC) methods. By replacing computationally intensive iterative processes with a direct, learned mapping, GenAI4UQ enables efficient calibration of input parameters and generation of predictions directly from observations. The software supports rapid ensemble forecasting with robust uncertainty quantification while maintaining computational and storage efficiency. Built-in auto-tuning of hyperparameters simplifies model training, ensuring accessibility for users with varying expertise. Its versatile conditional generative framework is applicable across diverse scientific domains. While GenAI4UQ offers significant advantages in flexibility and efficiency, users should interpret its uncertainty estimates with caution in data-sparse scenarios, as the model may overestimate uncertainty—an effect common to all surrogate-based approaches including MCMC with surrogate models. Despite this, GenAI4UQ transforms inverse modeling by providing a fast, reliable, and user-friendly solution. It empowers researchers and practitioners to quickly estimate parameter distributions and generate model predictions for new observations, facilitating efficient decision-making and advancing the state of uncertainty quantification in computational modeling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Physiological Controls on Carbon Fluxes and Biomass Production in Miscanthus: Insights From a Process‐Based Agroecosystem Model

Biomass crops serve as essential feedstocks for renewable energy and bioproducts and play a critical role in achieving lower emissions in the transportation sector. However, dedicated perennial biomass crops such as Miscanthus × giganteus (Miscanthus) remain underrepresented in process-based agroecosystem models, limiting robust evaluation of their economic and environmental performance. In this study, we developed a data-constrained representation of the sterile triploid Miscanthus (IL clone) within the process-based model ecosys, integrating global sensitivity analysis, ensemble simulation, and parameter calibration. Planting, harvesting, and fertilization practices consistent with field management were incorporated, and phenology was constrained using PhenoCam-derived Green Chromatic Coordinate (GCC) data. Using the Morris global sensitivity analysis method, we identified 11 key physiological parameters governing plant carbon, water, and nutrient relations, particularly processes associated with CO 2 assimilation. We then conducted ensemble simulations by perturbing these parameters and calibrated the model against eddy covariance fluxes and field-measured biomass. Building on the calibrated operating state, parameter-response analyses show that different photosynthetic processes influence productivity in different ways. Protein allocation determines whether productivity increases toward a higher level, whereas electron transport capacity controls additional gains once protein allocation approaches saturation. These findings demonstrate that parameter importance depends on physiological context and on which photosynthetic processes remain limiting. Calibration and validation against observations show that ecosys can reliably reproduce carbon and water fluxes, as well as both above- and belowground biomass, with post-calibration GPP R 2 improving from 0.67 to 0.95 during the calibration period and remaining high during validation (R 2 = 0.95). These results provide a mechanistic foundation for regional simulations and sustainable bioenergy assessments.

ecosys↗
DOE OSTI2020

Improving Enzyme Optimum Temperature Prediction with Resampling Strategies and Ensemble Learning

Accurate prediction of the optimal catalytic temperature ( T opt ) of enzymes is vital in biotechnology, as enzymes with high T opt values are desired for enhanced reaction rates. Recently, a machine learning method (temperature optima for microorganisms and enzymes, TOME) for predicting T opt was developed. TOME was trained on a normally distributed data set with a median T opt of 37 °C and less than 5% of T opt values above 85 °C, limiting the method’s predictive capabilities for thermostable enzymes. Due to the distribution of the training data, the mean squared error on T opt values greater than 85 °C is nearly an order of magnitude higher than the error on values between 30 and 50 °C. Here, we apply ensemble learning and resampling strategies that tackle the data imbalance to significantly decrease the error on high T opt values (>85 °C) by 60% and increase the overall R 2 value from 0.527 to 0.632. The revised method, temperature optima for enzymes with resampling (TOMER), and the resampling strategies applied in this work are freely available to other researchers as Python packages on GitHub.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Assessing the impact of substrate-level enzyme regulations limiting ethanol titer in Clostridium thermocellum using a core kinetic model

Clostridium thermocellum is a promising candidate for consolidated bioprocessing because it can directly ferment cellulose to ethanol. Despite significant efforts, achieved yields and titers fall below industrially relevant targets. This implies that there still exist unknown enzymatic, regulatory, and/or possibly thermodynamic bottlenecks that can throttle back metabolic flow. By (i) elucidating internal metabolic fluxes in wild-type C. thermocellum grown on cellobiose via 13 C-metabolic flux analysis ( 13 C-MFA), (ii) parameterizing a core kinetic model, and (iii) subsequently deploying an ensemble-docking workflow for discovering substrate-level regulations, this paper aims to reveal some of these factors and expand our knowledgebase governing C. thermocellum metabolism. Generated 13 C labeling data were used with 13 C-MFA to generate a wild-type flux distribution for the metabolic network. Notably, flux elucidation through MFA alluded to serine generation via the mercaptopyruvate pathway. Using the elucidated flux distributions in conjunction with batch fermentation process yield data for various mutant strains, we constructed a kinetic model of C. thermocellum core metabolism (i.e. k-ctherm138). Subsequently, we used the parameterized kinetic model to explore the effect of removing substrate-level regulations on ethanol yield and titer. Upon exploring all possible simultaneous (up to four) regulation removals we identified combinations that lead to many-fold model predicted improvement in ethanol titer. In addition, by coupling a systematic method for identifying putative competitive inhibitory mechanisms using K-FIT kinetic parameterization with the ensemble-docking workflow, we flagged 67 putative substrate-level inhibition mechanisms across central carbon metabolism supported by both kinetic formalism and docking analysis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Evaluating the Performance of Random Forest and Iterative Random Forest Based Methods when Applied to Gene Expression Data

Gene-to-gene networks, such as Gene Regulatory Networks (GRN) and Predictive Expression Networks (PEN) capture relationships between genes and are beneficial for use in downstream biological analyses. There exists multiple network inference tools to produce these gene-to-gene networks from matrices of gene expression data. Random Forest-Leave One Out Prediction (RF-LOOP) is a method that has been shown to be efficient at producing these gene-to-gene networks, frequently known as GEne Network Inference with Ensemble of trees (GENIE3). Here we validate that iterative Random Forest-Leave One Out Prediction (iRF-LOOP) produces higher quality networks than GENIE3. We use both synthetic and empirical networks from the Dialogue for Reverse Engineering Assessment and Methods (DREAM) Challenges by Sage Bionetworks, as well as two additional empirical networks created from Arabidopsis thaliana and Populus trichocarpa expression data.

iRF-Loop, expression network, Populus Trichocarpa↗
DOE OSTI2026

Scalable Quantum Monte Carlo Method for Polariton Chemistry via Mixed Block Sparsity and Tensor Hypercontraction Method

We present a reduced-scaling auxiliary-field quantum Monte Carlo (AFQMC) framework designed for large molecular systems and ensembles, with or without coupling to optical cavities. Our approach leverages the natural block sparsity of the Cholesky decomposition (CD) of electron repulsion integrals in molecular ensembles and employs tensor hypercontraction (THC) to efficiently compress low-rank Cholesky blocks. By representing the Cholesky vectors in a mixed format, keeping high-rank blocks in block-sparse form and compressing low-rank blocks with THC, we reduce the scaling of exchange-energy evaluation from quartic to robust cubic in the number of molecular orbitals N, while lowering memory from cubic toward quadratic. Benchmark analyses on one-, two-, and three-dimensional molecular ensembles (up to ∼1,200 orbitals) show that (a) the number of nonzeros in Cholesky tensors grows linearly with system size across dimensions; (b) the average numerical rank increases sublinearly and does not saturate at these sizes; and (c) rank heterogeneity─some blocks nearly full rank and many low rank, naturally motivates the proposed mixed block sparsity and THC scheme for efficient calculation of exchange energy. In conclusion, we demonstrate that the mixed scheme yields cubic wall-time scaling with favorable prefactors and preserves AFQMC accuracy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Benchmarking the performance of uncertainty quantification methods for neural network-based interatomic potentials

Machine-learned interatomic potentials (ML-IAPs) continue to gain popularity as accurate, computationally efficient replacements for traditional, physics-based interatomic potentials and expensive ab initio methods. Uncertainty quantification (UQ) of ML-IAPs is a growing area of research as UQ is critical in many applications of IAPs, such as developing curated datasets, active learning-based data augmentation, self-improving models, and estimating the uncertainty of molecular dynamics simulations. In this paper, we construct and benchmark a series of different neural network potentials (NNPs) with varying network architectures to determine the performance of these models with respect to both the mean and uncertainty calibration error. Each NNP method is specifically designed to predict either epistemic or aleatoric uncertainty with particular focus on the differences in behavior between the epistemic and aleatoric uncertainty estimates. We benchmark these methods using multiple datasets common in the ML-IAP literature. The results show that the aleatoric uncertainty from single-shot model architectures is a competitive alternative to ensemble-based epistemic uncertainty predictions in regions of sufficient data-density. However, in regions where the representative data is sparse, aleatoric uncertainty models tend to overpredict and epistemic methods tend to underpredict the actual model error. We conclude that the type of UQ is crucial when discussing performance of probabilistic model results as different methods have different performance characteristics depending on the regime in which they are evaluated. Therefore, the type of UQ method should be carefully evaluated against both the data characteristics and requirements for the intended application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Theory-Guided Autoencoders

Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models in order to increase the computational efficiency of data assimilation. The standard MFEnKF uses linear couplings between models, and is statistically optimal in case of Gaussian probability densities. This work extends the MFEnKF into to make use of a broader class of surrogate model such as those based on machine learning methods such as autoencoders non-linear couplings in between the model hierarchies. We identify the right-invertibility property for autoencoders as being a key predictor of success in the forecasting power of autoencoder-based reduced order models. We propose a methodology that allows us to construct reduced order surrogate models that are more accurate than the ones obtained via conventional linear methods. Numerical experiments with the canonical Lorenz'96 model illustrate that nonlinear surrogates perform better than linear projection-based ones in the context of multifidelity ensemble Kalman filtering. We additionality show a large-scale proof-of-concept result with the quasi-geostrophic equations, showing the competitiveness of the method with a traditional reduced order model-based MFEnKF.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Interactions of πK, ππK and KKπ systems at maximal isospin from lattice QCD

We study the interactions of systems of two and three nondegenerate mesons composed of pions and kaons at maximal isospin using lattice QCD, specifically $π^+K^+, π^+π^+K^+$ and $K^+K^+π^+$. Utilizing the stochastic LapH method, we determine the spectrum of these systems on two CLS $N_f$ = 2 + 1 ensembles with pion masses of 200 MeV and 340 MeV, and include many levels in different momentum frames. We constrain the K matrices describing two- and three-particle interactions by fitting the spectrum to the results predicted by the finite-volume formalism, including up to p waves. This requires also results for the $π^+π^+$ and $K^+K^+$ spectrum, which have been obtained previously on the same configurations. We explore different fitting strategies, comparing fits to energy shifts with fits to energies boosted to the rest frame, and also comparing simultaneous global fits to all relevant two- and three-particle channels to those where we first fit twoparticle channels and then add in the three-particle information. We provide the first determination of the three-particle K matrix in $π^+π^+K^+$ and $K^+K^+π^+$ systems, finding statistically significant nonzero results in most cases. We include s and $\textit{p}$ waves in the K matrix for $π^+K^+$ scattering, finding evidence for an attractive p-wave scattering length. We compare our results to Chiral Perturbation Theory, including an investigation of the impact of discretization errors, for which we provide the leading order predictions obtained using Wilson Chiral Perturbation Theory.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

A multi-dimensional parametric study of variability in multi-phase flow dynamics during geologic CO 2 sequestration accelerated with machine learning

Successful geologic CO 2 storage projects depend on numerical simulations to predict reservoir performance during site selection, injection verification, and post-injection monitoring phases of the project. These numerical simulations solve non-linear sets of coupled partial differential equations, while accounting for multi-phase fluid dynamics on the basis of constitutive equations that are embedded into the solution scheme. As a consequence, individual simulations often require tens to hundreds of hours to complete on high-performance computing clusters. Moreover, laboratory experiments reveal that parametric functions for capillary pressure and relative permeability exhibit substantial variability, even within the same rock type. This combination of computational expense and wide-ranging parametric variability means that there remains substantial uncertainty in the behavior of multi-phase CO 2 -water systems, particularly in the context of feedbacks between relative permeability and capillary pressure. To bridge this knowledge gap, here we develop a novel workflow that utilizes physics-based numerical simulation to train an artificial neural network (ANN) emulator for interrogating the multivariate parameter space that governs both capillary pressure and relative permeability. With this approach, the ANN is trained to emulate both fluid pressure distribution and CO 2 saturation, which are then interrogated quantitatively to generate parametric response surface mappings with high-fidelity resolution. Results from this study initially show that capillary entry pressure is the dominant control on both CO 2 plume geometry and fluid pressure propagation when considering the combined effects of capillary pressure and relative permeability, particularly when phase interference is low and residual CO 2 saturation is high. Moreover, the ANN emulator provides tremendous computational speed-up by computing 2691 individual simulations in several minutes; whereas, the same simulation ensemble would have required ~3 years of simulation time using only physics-based simulation methods (25,000 times speed up).

58 GEOSCIENCES↗
DOE OSTI2020

Grain boundary structure search by using an evolutionary algorithm with effective mutation methods

Grain boundaries (GBs) accommodate the misorientation between adjacent grains in a polycrystalline material. GBs are geometrically described by the macroscopic and microscopic degrees of freedom. Besides, at the atomistic level, GBs exhibit complicated behaviors under varying thermodynamic conditions. The complexity of atomistic GB structures demands stochastic searching for possible states. The effectiveness of stochastic search methods relies on techniques to recreate and select atomistic structures. In this work, we developed a new mutation operator that can induce direct and collective atomistic structure changes to boost the search efficiency of exploring GB structures with evolutionary algorithms (EA). We implemented the mutation methods along with innovative selection, crossover, boundary condition preprocessing methods to form an EA-based package to explore GB structures in grand canonical ensembles with atomistic simulations. We used this package to study the [001] symmetric tilt grain boundaries (STGBs) in FCC copper (Cu), the [110] STGBs in BCC tungsten (W), and the $[1\bar{2}10]$ STGBs in HCP magnesium (Mg). The results show that our design and implementation based on new mutation procedures, selection, and boundary conditions provide a high-quality search of atomistic GB structures in the grand canonical ensemble for different crystal lattices.

36 MATERIALS SCIENCE↗