Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Arctic Tipping Points Triggering Global Change (LDRD Final Report)

The Arctic is warming and feedbacks in the coupled Earth system may be driving the Arctic to tipping events that could have critical downstream impacts for the rest of the globe. In this project we have focused on analyzing sea ice variability and loss in the coupled Earth system Summer sea ice loss is happening rapidly and although the loss may be smooth and reversible, it has significant consequences for other Arctic systems as well as geopolitical and economic implications. Accurate seasonal predictions of sea ice minimum extent and long-term estimates of timing for a seasonally ice-free Arctic depend on a better understanding of the factors influencing sea ice dynamics and variation in this strongly coupled system. Under this project we have investigated the most influential factors in accurate predictions of September Arctic sea ice extent using machine learning models trained separately on observational data and on simulation data from five E3SM historical ensembles. Monthly averaged data from June, July, and August for a selection of ice, ocean, and atmosphere variables were used to train a random forest regression model. Gini importance measures were computed for each input feature with the testing data. We found that sea ice volume is most important earlier in the season (June) and sea ice extent became a more important predictor closer to September. Results from this study provide insight into how feature importance changes with forecast length and illustrates differences between observational data and simulated Earth system data. We have additionally performed a global sensitivity analysis (GSA) using a fully coupled ultra- low resolution configuration E3SM. To our knowledge, this is the first global sensitivity analysis involving the fully-coupled E3SM Earth system model. We have found that parameter variations show significant impact on the Arctic climate state and atmospheric parameters related to cloud parameterizations are the most significant. We also find significant interactions between parameters from different components of E3SM. The results of this study provide invaluable insight into the relative importance of various parameters from the sea ice, atmosphere and ocean components of the E3SM (including cross-component parameter interactions) on various Arctic-focused quantities of interest (QOIs).

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Comparison of Multivariate Time Series Prediction Techniques for Emulating Noah-LSM Soil Moisture Outputs

Land surface models are crucial tools for many earth science applications including numerical weather prediction, water resource and crop monitoring, and climatological analysis. Given a set of atmospheric forcings, seasonal data, and static parameters, models like Noah-LSM solve for land surface quantities including skin temperature, sensible heat flux, and soil moisture. While these calculations are theoretically robust, they are often computationally expensive. Since artificial neural networks (ANNs) are universal function approximators, they can learn to emulate the output of a deterministic numerical model given a time series of input forcings, with the learned ANN having substantially shorter execution time. The ANN could efficiently parameterize other models, generate ensembles, and provide first-guess inputs for retrievals. As such, with the goal of developing a model that efficiently mimics the output of Noah-LSM given NLDAS2 forcings on a region covering much of the central US, we examine and compare several neural network architectures for the multi-horizon multivariate time series forecasting problem. Recent literature includes a diverse set of approaches including autoregressive architectures like LSTM and GRU, parametric and non-parametric statistical predictors (ForecastNet and MQRNN), self-attention (LSTM-attention-LSTM), and temporal convovlution (DeepTCN). We implement several of these models for the Noah-LSM prediction task, highlighting the features and challenges for each and providing practical insight on the training process.

Mitchell Dodson↗
DOE OSTI2022

Prediction of grain structure after thermomechanical processing of U-10Mo alloy using sensitivity analysis and machine learning surrogate model

Abstract Hot rolling and annealing are critical intermediate steps for controlling microstructures and thickness variations when fabricating uranium alloyed with 10% molybdenum (U-10Mo), which is highly relevant to worldwide nuclear non-proliferation efforts. This work proposes a machine-learning surrogate model combined with sensitivity analysis to identify and predict U-10Mo microstructure development during thermomechanical processing. Over 200 simulations were collected using physics-based microstructure models covering a wide range of thermomechanical processing routes and initial alloy grain features. Based on the sensitivity analysis, we determined that an increase in rolling reduction percentage at each processing pass has the strongest effect in reducing the grain size. Multi-pass rolling and annealing can significantly improve recrystallization regardless of the reduction percentage. With a volume fraction below 2%, uranium carbide particles were found to have marginal effects on the average grain size and distribution. The proposed stratified stacking ensemble surrogate predicts the U-10Mo grain size with a mean square error four times smaller than a standard single deep neural network. At the same time, with a significant speedup (1000×) compared to the physics-based model, the machine learning surrogate shows good potential for U-10Mo fabrication process optimization.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions

The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.

Durden, David J.↗
NASA NTRS2024

An Extensible Perturbed Parameter Ensemble for the Community Atmosphere Model Version 6

This paper documents the methodology and preliminary results from a Perturbed Parameter Ensemble (PPE) technique, where multiple parameters are varied simultaneously and the parameter values are determined with Latin hypercube sampling. This is done with the Community Atmosphere Model version 6 (CAM6), the atmospheric component of the Community Earth System Model version 2 (CESM2). We apply the PPE method to CESM2-CAM6 to understand climate sensitivity to atmospheric physics parameters. The initial simulations vary 45 parameters in the microphysics, convection, turbulence and aerosol schemes with 263 ensemble members. These atmospheric parameters are typically the most uncertain in many climate models. Control simulations are analyzed and targeted simulations to understand climate forcing due to aerosols and fast climate feedbacks. The use of various emulators is explored in the multi- dimensional space mapping input parameters to output metrics. Parameter impacts on various model outputs, such as radiation, cloud and aerosol properties are evaluated. Machine learning is also used to probe optimal parameter values against observations. Our findings show that using PPE is a valuable tool for climate uncertainty analysis. Furthermore, by varying many parameters simultaneously, we find that many different combinations of parameter values can produce results consistent with observations, and thus careful analysis of tuning is important. The CESM2-CAM6 PPE is publicly available, and extensible to other configurations to address questions of other model processes in the atmosphere and other model components (e.g. coupling to the land surface).

Machine learning↗
DOE OSTI2021

PreMevE Update: Forecasting Ultra-Relativistic Electrons Inside Earth's Outer Radiation Belt

Energetic electrons inside Earth's Van Allen belts pose a major radiation threat to spaceborne electronics that often play vital roles in modern society. Ultra-relativistic electrons with energies greater than or equal to two megaelectron-volt (MeV) are of particular interest, and thus forecasting these ≥2 MeV electrons has a significant meaning to all space sectors. Here, we update the latest development of the predictive model for MeV electrons in the outer radiation belt. The new version, called PREdictive MEV Electron (PreMevE)-2E, forecasts ultra-relativistic electron flux distributions across the outer belt, with no need for in situ measurements of the trapped MeV electron population except at the geosynchronous orbit (GEO). Model inputs include precipitating electrons observed in low-Earthorbits by NOAA satellites, upstream solar wind speeds and densities from solar wind monitors, as well as ultra-relativistic electrons measured by one Los Alamos GEO satellite. We evaluated 32 supervised machine learning models that fall into four different classes of linear and neural network architectures, and successfully tested ensemble forecasting by using groups of top-performing models. All models are individually trained, validated, and tested by in situ electron data from NASA's Van Allen Probes mission. It is shown that the final ensemble model outperforms individual models at most L-shells, and this PreMevE-2E model can provide 25-h (~1-day) and 50-h (~2-day) forecasts with high mean performance efficiency and correlation values. Our results also suggest that this new model is dominated by nonlinear components at L-shells <~4 for ultra-relativistic electrons, different from the dominance of linear components for 1 MeV electrons as previously discovered.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Energy transfer in random-matrix ensembles of Floquet Hamiltonians

Here, we explore the statistical properties of energy transfer in ensembles of doubly driven random-matrix Floquet Hamiltonians based on universal symmetry arguments. The energy-pumping efficiency distribution P⁡($\overline{E}$) is associated with the Hamiltonian parameter ensemble and the eigenvalue statistics of the Floquet operator. For specific Hamiltonian ensembles, P⁡($\overline{E}$) undergoes a transition which cannot be associated with a symmetry breaking of the instantaneous Hamiltonian. The Floquet eigenvalue spacing distribution indicates the considered ensembles constitute generic nonintegrable Hamiltonian families. As a step towards Hamiltonian engineering, we develop a machine-learning classifier to understand the relative parameter importance in resulting high-conversion efficiency. We propose random Floquet Hamiltonians as a general framework to investigate frequency conversion effects in a class of generic dynamical processes beyond adiabatic pumps.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

Residue Specific Characterization of Electrostatics, Conformational Heterogeneity, and Dynamics of Electron Transfer Proteins with Linear and Nonlinear Infrared Spectroscopy. Final Report

The goal of this research project was to further our understanding of biological electron transfer through the development and application of new methods for studying proteins and their complexes with high spatial and temporal detail. The approach was based on incorporation of reporter bonds with distinct vibrational frequencies at specific locations in proteins and their characterization through the interaction of the bonds with pulses of infrared light. The project focused on the proteins plastocyanin (Pc) and cytochrome (cyt) f, two electron transfer partners in photosynthesis. The first thrust of the project was the incorporation of carbon deuterium bonds as non-perturbative probes at a ligand to the copper ion of Pc and their use to detect changes in the bonding due to the number of electrons at the metal and how the bonding changes upon association with cyt f. Toward developing more advanced approaches to characterize proteins, we incorporated a stronger reporter, the cyano group of cyanophenylalanine, at several locations in Pc and measured the frequency dynamics in the absence and presence of cyt f. The acquired data improved the current model for the protein complex. Notably, the data indicated that the majority of the population is involved in a loosely bound complex, providing insight into the nature of the ensembles formed by electron transfer protein complexes that is not available via other approaches. In addition to learning about the protein complex, we made strides toward developing and improving the new experimental approach to study highly dynamic behaviors of proteins.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

CERES: CRISPR Engineering for the Rapid Enhancement of Strains

Previous strain development efforts for cyanobacteria have failed to achieve the necessary productivities needed to support economic biofuel production. We proposed to develop CRISPR Engineering for Rapid Enhancement of Strains (CERES). We developed genetic and computational tools to enable future high-throughput screening of CRISPR interference (CRISPRi) libraries in the cyanobacterium Synechococcus sp. PCC 7002, including: (1) Operon- SEQer: an ensemble of algorithms for predicting operon pairs using RNA-seq data, (2) experimental characterization and machine learning prediction of gRNA design rules for CRISPRi, and (3) a shuttle vector for gene expression. These tools lay the foundation for CRISPR library screening to develop cyanobacterial strains that are optimized for growth or metabolite production under a wide range of environmental conditions. The optimization of cyanobacterial strains will directly advance U.S. energy and climate security by enabling domestic biofuel production while simultaneously mitigating atmospheric greenhouse gases through photoautotrophic fixation of carbon dioxide.

09 BIOMASS FUELS↗
DOE OSTI2022

Deep learning-accelerated 3D carbon storage reservoir pressure forecasting based on data assimilation using surface displacement from InSAR

Fast forecasting of the reservoir pressure distribution during geologic carbon storage (GCS) by assimilating monitoring data is a challenging problem. Due to high drilling cost, GCS projects usually have spatially sparse measurements from few wells, leading to high uncertainties in reservoir pressure prediction. To address this challenge, we use low-cost Interferometric Synthetic-Aperture Radar (InSAR) data as monitoring data to infer reservoir pressure build up. We develop a deep learning-accelerated workflow to assimilate surface displacement maps interpreted from InSAR and to forecast dynamic reservoir pressure. Employing an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates three-dimensional (3D) geologic properties and predicts reservoir pressure with quantified uncertainties. We use a synthetic commercial-scale GCS model with bimodally distributed permeability and porosity to demonstrate the efficacy of the workflow. A two-step CNN-PCA approach is employed to parameterize the bimodal fields. The computational efficiency of the workflow is boosted by two residual U-Net based surrogate models for surface displacement and reservoir pressure predictions, respectively. The workflow can complete data assimilation and reservoir pressure forecasting in half an hour on a personal computer.

25 ENERGY STORAGE↗
DOE OSTI2022

Development of advanced machine learning models for analysis of plutonium surrogate optical emission spectra

This work investigates and applies machine learning paradigms seldom seen in analytical spectroscopy for quantification of gallium in cerium matrices via processing of laser-plasma spectra. Ensemble regressions, support vector machine regressions, Gaussian kernel regressions, and artificial neural network techniques are trained and tested on cerium-gallium pellet spectra. A thorough hyperparameter optimization experiment is conducted initially to determine the best design features for each model. The optimized models are evaluated for sensitivity and precision using the limit of detection (LoD) and root mean-squared error of prediction (RMSEP) metrics, respectively. Gaussian kernel regression yields the superlative predictive model with an RMSEP of 0.33% and an LoD of 0.015% for quantification of Ga in a Ce matrix. This study concludes that these machine learning methods could yield robust prediction models for rapid quality control analysis of plutonium alloys.

Rao, Ashwin P. (ORCID:0000000319312568)↗
DOE OSTI2023

AI ensemble for signal detection of higher order gravitational wave modes of quasi-circular, spinning, non-precessing binary black hole mergers

We introduce spatiotemporal-graph models that concurrently process data from the twin advanced LIGO detectors and the advanced Virgo detector. We trained these AI classifiers with 2.4 million IMRPhenomXPHM waveforms that describe quasi-circular, spinning, non-precessing binary black hole mergers with component masses m{1,2}∈[3M⊙,50M⊙], and individual spins sz{1,2}∈[−0.9,0.9]; and which include the (ℓ,|m|)={(2,2),(2,1),(3,3),(3,2),(4,4)} modes, and mode mixing effects in the ℓ=3,|m|=2 harmonics. We trained these AI classifiers within 22 hours using distributed training over 96 NVIDIA V100 GPUs in the Summit supercomputer. We then used transfer learning to create AI predictors that estimate the total mass of potential binary black holes identified by all AI classifiers in the ensemble. We used this ensemble, 3 classifiers for signal detection and 2 total mass predictors, to process a year-long test set in which we injected 300,000 signals. This year-long test set was processed within 5.19 minutes using 1024 NVIDIA A100 GPUs in the Polaris supercomputer (for AI inference) and 128 CPU nodes in the ThetaKNL supercomputer (for post-processing of noise triggers), housed at the Argonne Leadership Computing Facility. These studies indicate that our AI ensemble provides state-of-the-art signal detection accuracy, and reports 2 misclassifications for every year of searched data. This is the first AI ensemble designed to search for and find higher order gravitational wave mode signals.

Tian, Minyang↗
DOE OSTI2026

Enhanced accuracy through ensembling of randomly initialized auto-regressive models for dynamical systems

Computational mechanics simulations using traditional finite element methods (FEM) require prohibitively expensive computational resources for real-time engineering applications, design optimization, and digital twin implementations. While machine learning (ML) surrogate models offer significant computational speedups, autoregressive ML models for time-dependent mechanical systems suffer from error accumulation that compromises long-term prediction reliability - a critical concern for engineering applications where accuracy over extended time horizons is essential for safety and performance assessments. Here, we propose a deep ensemble framework specifically designed to address this challenge in computational mechanics applications, where multiple ML surrogate models with random weight initializations are trained in parallel and their predictions aggregated during inference. This approach leverages statistical diversity to maximize information gain from a fixed set of training data and to mitigate error propagation, while maintaining the computational efficiency that makes ML surrogates attractive for engineering practice. We validate the framework on three representative problems spanning critical areas of computational mechanics: stress field evolution in heterogeneous microstructures under complex loading (relevant to advanced materials design and composite analysis), planetary-scale shallow water dynamics (applicable to environmental and geotechnical engineering), and Gray-Scott reaction-diffusion systems (relevant to mass transport and chemical process engineering). Across all test cases, the ensemble approach demonstrates consistent error reduction of 15-33% compared to individual models. The codes for this work are available on GitHub (https://github.com/Graham-Brady-Research-Group/AutoregressiveEnsemble_SpatioTemporal_Evolution).

autoregressive prediction↗
DOE OSTI2026

Leveraging machine learning to enhance aerosol classification using Single-Particle Mass Spectrometry

Advancing automated classification of atmospheric aerosols from Single-Particle Mass Spectrometry (SPMS) data remains challenging due to overlapping ion signatures, compositional diversity, and limited labeled data. This study evaluates supervised and semi-supervised learning frameworks to enhance aerosol identification by jointly leveraging labeled and unlabeled spectra. Four models were compared: a supervised Support Vector Machine (SVM), a self-training SVM, a stacked autoencoder classifier, and a stacked autoencoder trained using a temporal-ensembling Mean Teacher approach. All models achieved high and stable accuracies (90.0 %–91.1 %), surpassing previous results on the same dataset (87 %) and matching the performance of state-of-the-art deep learning methods. Despite small global metric differences (≤ 1 %), semi-supervised variants yielded up to 5 %–10 % improvements for compositionally rare particle types – such as soot (0.77 % of spectra, F1-score: 0.93–0.97) and hazelnut pollen (0.98 % of spectra, F1-score: 0.97–1.00) – equating to roughly ∼ 187 additional correctly classified spectra. These gains are scientifically significant, as such rare particles exert disproportionate influence on radiative absorption and ice nucleation processes; their improved detection reduces modeled uncertainties in aerosol absorption optical depth and mixed-phase cloud ice nucleation rates. The models' residual misclassifications (≈ 9 %) largely arise from true spectral overlap among chemically adjacent species (e.g., Na- vs. K-feldspar, coated vs. uncoated feldspars), reflecting physical compositional continuity rather than algorithmic error. Collectively, these findings demonstrate that leveraging unlabeled data to learn robust spectral representations and refine classification enhances both fidelity and interpretability, bridging data-driven analysis with aerosol–climate process understanding.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Physics-Based Machine Learning Methods for U-235 Forensics Signatures

Signatures of low-intensity U-235 sources have been recently studied by utilizing a variety of machine learning (ML) classifiers using features derived from gamma spectral measurements collectedunder structured campaigns. Several ML classifiers, such as ensemble of tress and classification trees, revealed misleadingly-optimistic training error due to over-fitting, and furthermore,their performance is not directly relatable to the physical properties due to their data-driven, opaque designs. We present a regression-based ML method that first estimates the inverse distanceto the source and then utilizes a threshold to infer its presence, by representing the background as a source located at an infinite distance. For the inverse distance estimation, we study the ensembleof trees and Gaussian process regression methods, and a hyper parameter auto-tuning and selection method that employs five regression estimators. These methods avoid the over-fittingobserved in several ML classifiers, while providing the classification error nearly comparable to them based on independent test data. Their error is directly related to estimates of the inversephysical distance to source, and the precision of error determines the seperability property that determines the false alarm and missed detection rates. The property of monotonic decrease of thesource strength with increasing detector distance combined with Poisson distribution of measurements is utilized to analytically validate these methods by deriving the generalization equations ofunderlying regression methods.

Rao, Nageswara↗
DOE OSTI2021

COVID-19 dynamics across the US: A deep learning study of human mobility and social behavior

This paper presents a deep learning framework for epidemiology system identification from noisy and sparse observations with quantified uncertainty. The proposed approach employs an ensemble of deep neural networks to infer the time-dependent reproduction number of an infectious disease by formulating a tensor-based multi-step loss function that allows us to efficiently calibrate the model on multiple observed trajectories. The method is applied to a mobility and social behavior-based SEIR model of COVID-19 spread. The model is trained on Google and Unacast mobility data spanning a period of 66 days, and is able to yield accurate future forecasts of COVID-19 spread in 203 US counties within a time-window of 15 days. Interestingly, a sensitivity analysis that assesses the importance of different mobility and social behavior parameters reveals that attendance of close places, including workplaces, residential, and retail and recreational locations, has the largest impact on the effective reproduction number. Furthermore, the model enables us to rapidly probe and quantify the effects of government interventions, such as lock-down and re-opening strategies. Taken together, the proposed framework provides a robust workflow for data-driven epidemiology model discovery under uncertainty and produces probabilistic forecasts for the evolution of a pandemic that can judiciously provide information for policy and decision making. All codes and data accompanying this manuscript are available at https://github.com/PredictiveIntelligenceLab/DeepCOVID19.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Supercomputer-Based Ensemble Docking Drug Discovery Pipeline with Application to Covid-19

In this work, we present a supercomputer-driven pipeline for in silico drug discovery using enhanced sampling molecular dynamics (MD) and ensemble docking. Ensemble docking makes use of MD results by docking compound databases into representative protein binding-site conformations, thus taking into account the dynamic properties of the binding sites. We also describe preliminary results obtained for 24 systems involving eight proteins of the proteome of SARS-CoV-2. The MD involves temperature replica exchange enhanced sampling, making use of massively parallel supercomputing to quickly sample the configurational space of protein drug targets. Using the Summit supercomputer at the Oak Ridge National Laboratory, more than 1 ms of enhanced sampling MD can be generated per day. We have ensemble docked repurposing databases to 10 configurations of each of the 24 SARS-CoV-2 systems using AutoDock Vina. Comparison to experiment demonstrates remarkably high hit rates for the top scoring tranches of compounds identified by our ensemble approach. We also demonstrate that, using Autodock-GPU on Summit, it is possible to perform exhaustive docking of one billion compounds in under 24 h. Finally, we discuss preliminary results and planned improvements to the pipeline, including the use of quantum mechanical (QM), machine learning, and artificial intelligence (AI) methods to cluster MD trajectories and rescore docking poses.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Pyrocumulonimbus Events Over British Columbia in 2017: An Ensemble Model Study of Parameter Sensitivities and Climate Impacts of Wildfire Smoke in the Stratosphere

Abstract The Pyrocumulonimbus (pyroCb) events over British Columbia in 2017 were observed in the lower stratosphere for about 8–10 months after the smoke injections. Several previous studies used global climate models to investigate the physical parameters for the 2017 pyroCb events, but the conclusions show strong model dependency. In this study, we use the Energy Exascale Earth System Model (E3SM) and complete an ensemble of runs exploring three injection parameters: smoke aerosol mass, the percentage of black carbon within the smoke aerosols, and plume injection height. Additionally, we consider the heterogeneous reaction of ozone and primary organic matter. According to the satellite daily observed aerosol optical depth (AOD), we find that the best ensemble member is the simulation with 0.4 Tg of smoke, 3% of which is black carbon, a 13.5 km smoke injection height, and a 10 −5 probability factor of the heterogeneous reaction. Besides AOD, we examine the ensemble score based on the metrics used in previous studies: the metrics of the extinction coefficient at 18 km altitude and the maximum plume height. The conclusion of the best estimate of the injection parameters for 2017 pyroCb events shows strong not only model but also evaluation metric dependency. We use the Random Forest machine learning technique to quantify the relative importance of each parameter in accurately simulating the 2017 pyroCb events and find that the injection height is the most critical feature, no matter which metric is used to score the ensemble members.

54 ENVIRONMENTAL SCIENCES↗