Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Theory-Guided Autoencoders

Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models in order to increase the computational efficiency of data assimilation. The standard MFEnKF uses linear couplings between models, and is statistically optimal in case of Gaussian probability densities. This work extends the MFEnKF into to make use of a broader class of surrogate model such as those based on machine learning methods such as autoencoders non-linear couplings in between the model hierarchies. We identify the right-invertibility property for autoencoders as being a key predictor of success in the forecasting power of autoencoder-based reduced order models. We propose a methodology that allows us to construct reduced order surrogate models that are more accurate than the ones obtained via conventional linear methods. Numerical experiments with the canonical Lorenz'96 model illustrate that nonlinear surrogates perform better than linear projection-based ones in the context of multifidelity ensemble Kalman filtering. We additionality show a large-scale proof-of-concept result with the quasi-geostrophic equations, showing the competitiveness of the method with a traditional reduced order model-based MFEnKF.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While traditional ensemble classifiers yield improved task performance, such methods come with a steep cost in model complexity. Through knowledge distillation, it is possible to retain some performance improvements from an ensemble, while minimizing costs to model complexity. In this work, we implement a weakly supervised ensemble using clustering-constrained-attention multiple-instance learners (CLAM), which uses attention and instance-level clustering to identify task salient regions and feature extraction in whole slides. By applying logit-based and attention-based knowledge distillation, we show it is possible to retain some performance improvements resulting from the ensemble at zero cost to model complexity.

Alamudun, Folami↗
DOE OSTI2026

Conformalized-KANs: Uncertainty Quantification with Coverage Guarantees for Kolmogorov-Arnold Networks (KANs) in Scientific Machine Learning

This paper explores uncertainty quantification (UQ) methods in the context of Kolmogorov–Arnold Networks (KANs). We apply an ensemble approach to KANs to obtain a heuristic measure of UQ, enhancing interpretability and robustness in modeling complex functions. Building on this, we introduce Conformalized-KANs, which integrate conformal prediction, a distribution-free UQ technique, with KAN ensembles to generate calibrated prediction intervals with guaranteed coverage.} Extensive numerical experiments are conducted to evaluate the effectiveness of these methods, focusing particularly on the robustness and accuracy of the prediction intervals under various hyperparameter settings. We show that the conformal KAN predictions can be applied to recent extensions of KANs, including Finite Basis KANs (FBKANs) and multifideilty KANs (MFKANs). The results demonstrate the potential of our approaches to significantly improve the reliability and applicability of KANs in scientific machine learning.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2020

Deeply uncertain: comparing methods of uncertainty quantification in deep learning algorithms

We present a comparison of methods for uncertainty quantification (UQ) in deep learning algorithms in the context of a simple physical system. Three of the most common uncertainty quantification methods - Bayesian Neural Networks (BNN), Concrete Dropout (CD), and Deep Ensembles (DE) - are compared to the standard analytic error propagation. We discuss this comparison in terms endemic to both machine learning ("epistemic" and "aleatoric") and the physical sciences ("statistical" and "systematic"). The comparisons are presented in terms of simulated experimental measurements of a single pendulum - a prototypical physical system for studying measurement and analysis techniques. Our results highlight some pitfalls that may occur when using these UQ methods. For example, when the variation of noise in the training set is small, all methods predicted the same relative uncertainty independently of the inputs. This issue is particularly hard to avoid in BNN. On the other hand, when the test set contains samples far from the training distribution, we found that no methods sufficiently increased the uncertainties associated to their predictions. This problem was particularly clear for CD. In light of these results, we make some recommendations for usage and interpretation of UQ methods.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Enhanced analysis of experimental x-ray spectra through deep learning

X-ray spectroscopic data from high-energy-density laser-produced plasmas has long required thorough, time-consuming analysis to extract meaningful source conditions. There are often confounding factors due to rapidly evolving states and finite spatial gradients (e.g., the existence of multi-temperature, multi-density, multi-ionization states, etc.) that make spectral measurements and analysis difficult. Here, in this paper, we demonstrate how deep learning can be applied to enhance x-ray spectral data analysis in both speed and intricacy. Neural networks (NNs) are trained on ensemble atomic physics simulations so that they can subsequently construct a model capable of extracting plasma parameters directly from experimental spectra. Through deep learning, the models can extract temperature distributions as opposed to single or dual temperature/density fits from standard trial-and-error atomic modeling at a significantly reduced computational cost compared to traditional trial-and-error methods. These NNs are envisioned to be deployed with high repetition rate x-ray spectrometers in order to provide detailed real-time analysis of experimental spectra.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Graph reinforcement learning for exploring model spaces beyond the standard model

We present a methodology for performing scans of beyond the standard model (BSM) parameter spaces with reinforcement learning. We identify a novel procedure using graph neural networks that is capable of exploring spaces of models without the user specifying a fixed particle content, allowing broad classes of BSM models to be explored—in theory, the technique is applicable to nearly any model space with a prespecified gauge group. We provide a generic procedure by which a suitable graph grammar can be developed for any BSM model that features user-specified symmetry groups and a finite number of different possible particle species, the use of which is applicable to a variety of machine learning tasks over the actions of BSM theories beyond our particular reinforcement learning use case. As a proof of concept, we construct the graph grammar for theories with vectorlike leptons that may or may not be charged under a dark U ( 1 ) group, inspired by portal matter extensions of the sub-GeV vector portal/kinetic mixing simplified dark matter models. We then use this graph grammar to create a reinforcement learning environment tasked with creating models with these vectorlike leptons that are consistent with a list of a variety of precision observables. The reinforcement learning agent succeeds in developing models that can address the observed muon anomalous magnetic moment discrepancy while remaining consistent with flavor violation and electroweak precision observables, including both constructions that have previously been studied as well as new models that have not, to our knowledge, previously been identified. By inspecting the resulting ensembles of models that the agent produces and experimenting with different configurations for our reinforcement learning environment and graph grammar, we also infer various lessons about the development of these environments that can be transferable to reinforcement learning scans of more complicated model spaces and comment on future directions for the development of this technique into a more mature tool. Published by the American Physical Society 2025

Wojcik, George N.↗
DOE OSTI2022

Learning emergent partial differential equations in a learned emergent space

We propose an approach to learn effective evolution equations for large systems of interacting agents. This is demonstrated on two examples, a well-studied system of coupled normal form oscillators and a biologically motivated example of coupled Hodgkin-Huxley-like neurons. For such types of systems there is no obvious space coordinate in which to learn effective evolution laws in the form of partial differential equations. In our approach, we accomplish this by learning embedding coordinates from the time series data of the system using manifold learning as a first step. In these emergent coordinates, we then show how one can learn effective partial differential equations, using neural networks, that do not only reproduce the dynamics of the oscillator ensemble, but also capture the collective bifurcations when system parameters vary. The proposed approach thus integrates the automatic, data-driven extraction of emergent space coordinates parametrizing the agent dynamics, with machine-learning assisted identification of an emergent PDE description of the dynamics in this parametrization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Learning from many collider events at once

There have been a number of recent proposals to enhance the performance of machine learning strategies for collider physics by combining many distinct events into a single ensemble feature. To evaluate the efficacy of these proposals, we study the connection between single-event classifiers and multievent classifiers under the assumption that collider events are independent and identically distributed. We show how one can build optimal multievent classifiers from single-event classifiers, and we also show how to construct multievent classifiers such that they produce optimal single-event classifiers. This is illustrated for a Gaussian example as well as for classification tasks relevant for searches and measurements at the Large Hadron Collider. We extend our discussion to regression tasks by showing how they can be phrased in terms of parametrized classifiers. Empirically, we find that training a single-event (per-instance) classifier is more effective than training a multievent (per-ensemble) classifier, as least for the cases we studied, and we relate this fact to properties of the loss function gradient in the two cases. While we did not identify a clear benefit from using multievent classifiers in the collider context, we speculate on the potential value of these methods in cases involving only approximate independence, as relevant for jet substructure studies.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Continual learning in the presence of repetition

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.

Class-incremental learning↗
DOE OSTI2025

Transfer learning of neural surrogates on multifidelity groundwater simulations

Multifidelity data used in the paper published in Advances in Water Resources 206 (2025) 105140, https://doi.org/10.1016/j.advwatres.2025.105140 The code used to process the data is openly available on GitHub at https://github.com/Model-Reduction-and-UQ-Group/Transfer_Learning_K_reconstruction Computationally inexpensive surrogates of process-based models, such as deep neural networks, enable ensemble-based computations used in risk assessment, data assimilation, etc. However, generation of large datasets required to train a neural network can be as expensive as the ensemble simulations themselves. We ameliorate this challenge by using data from multifidelity (MF) groundwater simulations and transfer learning (TL) to reduce data generation costs while maintaining model accuracy. As a computational example, we train a deep convolutional neural network (CNN) to reconstruct permeability fields from saturation maps derived from a multiphase flow model. Starting with very low- and low-fidelity data generated on increasingly coarse meshes, we pretrain the CNN, followed by output-layer training and fine-tuning using only a limited number of high-fidelity samples. We demonstrate the surrogate’s robustness when interpreting low-quality inputs—such as interpolated maps or data affected by noise—which has strong implications for the applicability in practical hydrogeological scenarios. This multilevel MF-TL strategy achieves a favorable trade-off between computational efficiency and predictive accuracy, significantly outperforming high-fidelity-only approaches under the same computational budget.

Chiofalo, Alessia [University of Bologna] (ORCID:0↗
DOE OSTI2026

KRAS4a and KRAS4b show distinct lipid-dependent regulation of RAS-RAF membrane dynamics

KRAS4a and KRAS4b are important regulators of signaling, and their interactions with the plasma membrane are dynamic and influenced by lipid composition. KRAS 4a and 4b have nearly identical globular domains but differ in their membrane-associated hyper variable region (HVR). The functional distinctions between these isoforms remain unclear, particularly with regards to their dependence on specific lipids and the membrane environment. Previous work showed that the membrane orientation of KRAS4b affects its ability to bind to RAF kinase RBDCRD and that the KRAS–RBDCRD complex adopts different poses on the membrane as well as influences the size and composition of the lipid environment. To model differences between KRAS 4a and 4b protein–lipid interactions, we extended the Multiscale Machine-Learned Modeling Infrastructure (MuMMI) to incorporate continuum simulations in the grand canonical ensemble, enabling sampling across macroscopic, coarse-grained, and all-atom resolutions. Using this framework, we systematically altered PIP2 concentrations, KRAS 4a versus 4b, and RAF RBDCRD complexation to assess impacts on membrane–protein interactions and dynamics. Our results reveal that reducing PIP2 shifts and broadens the membrane orientational preference of both KRAS 4b and 4a, with stronger effects on 4b HVR localization versus 4a. We demonstrate that with depletion of the strong negatively charged PIP2 lipid, the less charged phosphatidylserine replaces PIP2. Our findings highlight similarities and distinctions in the dynamics and lipid dependency of KRAS isoforms and suggest that ordering of the local lipid composition by HVRs is a shared property and key modulator of RAS-mediated signaling at the plasma membrane.

Biological and medical sciences↗
DOE OSTI2021

Artificial intelligence techniques for integrative structural biology of intrinsically disordered proteins

Here, we outline recent developments in artificial intelligence (AI) and machine learning (ML) techniques for integrative structural biology of intrinsically disordered proteins (IDP) ensembles. IDPs challenge the traditional protein structure-function paradigm by adapting their conformations in response to specific binding partners leading them to mediate diverse, and often complex cellular functions such as biological signaling, self-organization and compartmentalization. Obtaining mechanistic insights into their function can therefore be challenging for traditional structural determination techniques. Often, scientists have to rely on piecemeal evidence drawn from diverse experimental techniques to characterize their functional mechanisms. Multiscale simulations can help bridge critical knowledge gaps about IDP structure-function relationships - however, these techniques also face challenges in resolving emergent phenomena within IDP conformational ensembles. We posit that scalable statistical inference techniques can effectively integrate information gleaned from multiple experimental techniques as well as from simulations, thus providing access to atomistic details of these emergent phenomena.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

A comparison of model validation approaches for echo state networks using climate model replicates

As global temperatures continue to rise, climate mitigation strategies such as stratospheric aerosol injections (SAI) are increasingly discussed, but the downstream effects of these strategies are not well understood. As such, there is interest in developing statistical methods to quantify the evolution of climate variable relationships during the time period surrounding an SAI. Feature importance applied to echo state network (ESN) models has been proposed as a way to understand the effects of SAI using a data-driven model. This approach depends on the ESN fitting the data well. If not, the feature importance may place importance on features that are not representative of the underlying relationships. Typically, time series prediction models such as ESNs are assessed using out-of-sample performance metrics that divide the times series into separate training and testing sets. However, this model assessment approach is geared towards forecasting applications and not scenarios such as the motivating SAI example where the objective is using a data driven model to capture variable relationships. Here, in this paper, we demonstrate a novel use of climate model replicates to investigate the applicability of the commonly used repeated hold-out model assessment approach for the SAI application. Simulations of an SAI are generated using a simplified climate model, and different initialization conditions are used to provide independent training and testing sets containing the same SAI event. The climate model replicates enable out-of-sample measures of model performance, which are compared to the single time series hold-out validation approach. For our case study, it is found that the repeated hold-out sample performance is comparable, but conservative, to the replicate out-of-sample performance when the training set contains enough time after the aerosol injection.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Developing a complete AI-accelerated workflow for superconductor discovery

The quest to identify new superconducting materials with enhanced properties is hindered by the prohibitive cost of computing electron-phonon spectral functions, severely limiting the materials space that can be explored. Here, we introduce a Bootstrapped Ensemble of Equivariant Graph Neural Networks (BEE-NET), a machine-learning model trained to predict the Eliashberg spectral function and superconducting critical temperature with a mean-absolute-error of 0.87 K relative to DFT-based Allen-Dynes calculations. Intriguingly, BEE-NET achieves a true-negative-rate of 99.4%, enabling highly efficient screening for the rare property of superconductivity. Integrated into a multi-stage, AI-accelerated discovery pipeline that incorporates elemental-substitution strategies and machine-learned interatomic potentials, our workflow reduced over 1.3 million candidate structures to 741 dynamically and thermodynamically stable compounds with DFT-confirmed T c > 5 K. We report the successful synthesis and experimental confirmation of superconductivity in two of these previously unreported compounds. This study establishes a data-driven framework that integrates machine learning, quantum calculations, and experiments to systematically accelerate superconductor discovery.

Gibson, Jason B. [Quantum Formatics, Cambridge, MA↗
DOE OSTI2025

Micromechanical Surrogate Machine Learning Model for Creep Deformation Modeling

Process variability during the manufacture of gas turbine engine hot section components can significantly affect the material’s resulting microstructure. In casting, for instance, geometric variation within a component (thin sections versus thick sections, radial location) influences cooling rates and the resulting grain size. The high temperature creep response is known to be sensitive to grain size owing to a diffusional creep mechanism which occurs more readily along grain boundaries. Microstructural variation correspondingly drives mechanical behavior which propagates into component scale performance uncertainty. These factors are essential when planning inspection, maintenance, and repair strategies within a reliability framework. These benefits provide opportunities to increase overall energy efficiency through refined margins. Critically, there is an opportunity to bolster existing data-driven reliability models using physics-driven process-structure-property relations. Here we present recent work establishing a framework for evaluating the probabilistic creep performance of high-temperature materials. A novel microstructure-sensitive crystal plasticity finite element model is established that captures both grain boundary and crystallographic deformation effects. The computationally expensive physics model is calibrated using a statistical approach and this high-fidelity model is subsequently used to train a computationally efficient machine learning surrogate model. The surrogate model is essential for sampling a large ensemble of simulated structure-property pair results. The ensemble data are then mined to extract salient trends to be incorporated into a microstructure-sensitive reliability model. The proposed approach represents a novel way to capture microstructure-sensitive trends from physics-based models within a modern reliability framework.

Fernandez-Zelaia, Patxi [ORNL]↗
DOE OSTI2022

#COVIDisAirborne: AI-enabled multiscale computational microscopy of delta SARS-CoV-2 in a respiratory aerosol

We seek to completely revise current models of airborne transmission of respiratory viruses by providing never-before-seen atomic-level views of the SARS-CoV-2 virus within a respiratory aerosol. Our work dramatically extends the capabilities of multiscale computational microscopy to address the significant gaps that exist in current experimental methods, which are limited in their ability to interrogate aerosols at the atomic/molecular level and thus obscure our understanding of airborne transmission. We demonstrate how our integrated data-driven platform provides a new way of exploring the composition, structure, and dynamics of aerosols and aerosolized viruses, while driving simulation method development along several important axes. We present a series of initial scientific discoveries for the SARS-CoV-2 Delta variant, noting that the full scientific impact of this work has yet to be realized.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

OSW Consortium 2 - Validated National Offshore Wind Resource Dataset with Uncertainty Quantification (CRADA Report)

This research has led to the development of the 2023 National Offshore Wind data set (NOW-23), which offers the latest wind resource information for offshore regions in the United States. NOW-23 supersedes, for its offshore component, the Wind Integration National Dataset (WIND) Toolkit, which was published a decade ago and is currently a primary resource for wind resource assessments and grid integration studies in the contiguous United States. By incorporating advancements in the Weather Research and Forecasting (WRF) model, NOW-23 delivers an updated and cutting-edge product to stakeholders. As part of this project, we also developed a summary of the uncertainty quantification in NOW-23, along with NOW-WAKES, a 1-year post-construction data set that quantifies expected offshore wake effects in the US Mid-Atlantic lease areas. Stakeholders can access the NOW-23 data set at https://doi.org/10.25984/1821404.

17 WIND ENERGY↗
DOE OSTI2026

High-Efficiency Solar-To-Fuel Photoelectrochemistry in Disordered Photonic Glass Electrodes (Final Technical Report)

This project investigated how photonic glass (PG) photoelectrodes—disordered arrangements of dielectric scatterers—can serve as scalable, tunable platforms for light trapping in photoelectrochemical (PEC) solar-to-fuel systems. By leveraging disorder-driven optical phenomena such as multiple scattering resonances and light localization, PG structures offer an alternative to conventional photonic crystals and inverse opals that require high structural precision. The scientific goals were to twofold: (1) develop approaches to predictive models for high performance PG electrodes based on light absorption simulations, and (2) fabricate, characterize, and optimize PG-based photoelectrodes for solar-to-hydrogen and solar-to-fuel photoelectrochemical applications. To overcome the complexity of ensemble optical simulations for disordered materials, the researchers developed a machine-learning-accelerated emulation of all configurations in the design space. With this approach, PG photoelectrodes based on a TiO2 semiconductor were designed to enhance PEC currents of up to one hundred times higher than the equivalent ultra-thin film photoanodes and several times higher than the equivalent photonic crystal. The research also explored integrated systems for electrochemical hydrogen production based on replacing water oxidation with the specific glycerol oxidation electrocatalysis. Overall, the project outlined an approach to a simple-to-fabricate photoelectrode system to drive photoelectrochemical reactions relevant to solar photochemical energy conversion.

14 SOLAR ENERGY↗