Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Ensemble learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A Remote Sensing Technique to Upscale Methane Emission Flux in a Subtropical Peatland

Abstract Quantification of methane (CH 4 ) gas emission from peat is critical to understand CH 4 budget from natural wetlands under a climate warming scenario. Previous studies have focused on prediction and mapping of CH 4 emission flux using process‐based models, while application of statistical‐empirical models for upscaling spatially sparse in situ measurements is scarce. In this study, we developed an empirical remote sensing upscaling approach to estimate CH 4 emission flux in the Everglades using limited in situ point‐based CH 4 emission flux measurements and Landsat data during 2013–2018. We spatially and temporally linked in situ data with Landsat surface reflectance based on temporally composite data sets and developed an object‐based machine learning framework to model and map CH 4 emission flux. An ensemble analysis of two machine learning models, k ‐Nearest Neighbor ( k ‐NN) and Support Vector Machine (SVM), shows that the upscaling approach is promising for predicting CH 4 emission flux with a R 2 of 0.65 and 0.87 based on a fivefold cross‐validation for a dry season and wet season estimation, respectively. We generated emission flux map products that successfully revealed the spatial and temporal heterogeneity of CH 4 emission within the dominant freshwater marsh ecosystem in the Everglades. We conclude that Landsat is promising for upscaling and monitoring CH 4 emission flux and reducing the uncertainty in emission estimates from wetlands.

Zhang, Caiyun↗
DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗
DOE OSTI2025

Uncertainty based Online Ensemble on Non-Stationary Data for Fusion Science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior due to drifts in the data. The drifts can arise from both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with non-stationary data streams.Online learning can be used to continuously adapt the models with new data as it is acquired. However, traditional online learning can suffer from short-term performance degradation, as ground truth are not available before making the prediction. To address this challenge, we propose uncertainty aware ensemble approach for online learning. We use Deep Gaussian Process Approximation (DGPA) technique for calibrated uncertainty estimation and use the uncertainty values to guide a meta-algorithm that produces predictions based on ensemble of learners. Moreover, DGPA also provides uncertainty estimation along with the predictions for decision makers. This paper demonstrates that the proposed method outperforms traditional online learning approach, and a naive ensemble without uncertainty guidance by about 7% and 6%, respectively, on B-coil deflection prediction at DIII-D Fusion Facility.

Rajput, Kishansingh [Thomas Jefferson National Acc↗
DOE OSTI2025

Uncertainty based Online Ensemble on Non-Stationary Data for Fusion Science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior due to drifts in the data. The drifts can arise from both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with non-stationary data streams.Online learning can be used to continuously adapt the models with new data as it is acquired. However, traditional online learning can suffer from short-term performance degradation, as ground truth are not available before making the prediction. To address this challenge, we propose uncertainty aware ensemble approach for online learning. We use Deep Gaussian Process Approximation (DGPA) technique for calibrated uncertainty estimation and use the uncertainty values to guide a meta-algorithm that produces predictions based on ensemble of learners. Moreover, DGPA also provides uncertainty estimation along with the predictions for decision makers. This paper demonstrates that the proposed method outperforms traditional online learning approach, and a naive ensemble without uncertainty guidance by about 7% and 6%, respectively, on B-coil deflection prediction at DIII-D Fusion Facility.

Rajput, Kishansingh [Thomas Jefferson National Acc↗
DOE OSTI2021

Scalable statistical inference of photometric redshift via data subsampling

Handling big data has largely been a major bottleneck in traditional statistical models. Consequently, when accurate point prediction is the primary target, machine learning models are often preferred over their statistical counterparts for bigger problems. But full probabilistic statistical models often outperform other models in quantifying uncertainties associated with model predictions. We develop a data-driven statistical modeling framework that combines the uncertainties from an ensemble of statistical models learned on smaller subsets of data carefully chosen to account for imbalances in the input space. We demonstrate this method on a photometric redshift estimation problem in cosmology, which seeks to infer a distribution of the redshift—the stretching effect in observing the light of far-away galaxies—given multivariate color information observed for an object in the sky. Our proposed method performs balanced partitioning, graph-based data subsampling across the partitions, and training of an ensemble of Gaussian process models.

data subsampling↗
DOE OSTI2020

Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms [Slides]

Deep learning is used in many applications in the physical sciences. In those sciences we are used to having uncertainty on every measurement or prediction. In the last years, many methods have been put forth for uncertainty quantification: Bayesian Neural Networks, Concrete Dropout, and Deep Ensembles. Uncertainty in deep learning is often divided into aleatoric or irreducible and epistemic or reducible. The problem remains: Which uncertainty quantification method should be chosen? How are those results interpreted?

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

The ASHRAE Great Energy Predictor III competition: Overview and results

In late 2019, ASHRAE hosted the Great Energy Predictor III (GEPIII) machine learning competition on the Kaggle platform. This launch marked the third energy prediction competition from ASHRAE and the first since the mid-1990s. In this updated version, the competitors were provided with over 20 million points of training data from 2,380 energy meters collected for 1,448 buildings from 16 sources. This competition’s overall objective was to find the most accurate modeling solutions for the prediction of over 41 million private and public test data points. Furthermore, the competition had 4,370 participants, split across 3,614 teams from 94 countries who submitted 39,403 predictions. In addition to the top five winning workflows, the competitors publicly shared 415 reproducible online machine learning workflow examples (notebooks), including over 40 additional, full solutions. This paper gives a high-level overview of the competition preparation and dataset, competitors and their discussions, machine learning workflows and models generated, winners and their submissions, discussion of lessons learned, and competition outputs and next steps. The most popular and accurate machine learning workflows used large ensembles of mostly gradient boosting tree models, such as LightGBM. Similar to the first predictor competition, preprocessing of the data sets emerged as a key differentiator.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

A benchmark dataset for Hydrogen Combustion

The generation of reference data for deep learning models is challenging for reactive systems, and more so for combustion reactions due to the extreme conditions that create radical species and alternative spin states during the combustion process. Here, we extend intrinsic reaction coordinate (IRC) calculations with ab initio MD simulations and normal mode displacement calculations to more extensively cover the potential energy surface for 19 reaction channels for hydrogen combustion. A total of ~290,000 potential energies and ~1,270,000 nuclear force vectors are evaluated with a high quality range-separated hybrid density functional, ωB97X-V, to construct the reference data set, including transition state ensembles, for the deep learning models to study hydrogen combustion reaction.

08 HYDROGEN↗
DOE OSTI2026

Evaluating the Limits of QAOA Parameter Transfer at High-Rounds on Sparse Ising Models With Geometrically Local Cubic Terms

The emergent practical applicability of the Quantum Approximate Optimization Algorithm (QAOA) for approximate combinatorial optimization is a subject of considerable interest. One of the primary limitations of QAOA is the task of finding a set of good parameters, which is usually done using a variational optimization loop. Parameter transfer, or parameter concentration, is a phenomenon where QAOA angles trained on problem instances that are self-similar tend to perform well for other problem instances from that similar class. This suggests a potentially highly efficient and scalable non-variational learning method for QAOA angle finding. In this work, we systematically study QAOA parameter transferability from small problem sizes (16 and 27 decision variables) onto large problem instances (up to 156 qubits) for heavy-hex graph Ising models with geometrically local higher order terms using the Julia based QAOA simulation tool \texttt{JuliQAOA} to perform classical angle finding for up to $49$ QAOA layers ($p$). Parameter transfer of the fixed angles is validated using a combination of full statevector, Projected Entangled Pair States (PEPS), Matrix Product State (MPS), and LOWESA numerical simulations. We find that the QAOA parameter transfer from single instances applied to other (unseen) problem instances does not in general provide monotonically improving performance as a function of $p$ - there are many cases where the performance temporarily decreases as a function of $p$ - but despite this the transferred angles have a general trend of improved expectation value as the QAOA depth increases, in many cases converging close to the true ground-state energy of the $100+$ qubit instances. We also sample the hardware-compatible Ising models using the ensemble of transfer-learned QAOA parameters on several superconducting qubit IBM Quantum processors with 127, 133, and 156 qubits. We find continuous solution quality improvement of the hardware-compatible QAOA circuits run on the IBM NISQ processors up to $p=5$ on \texttt{ibm\_fez}, up to $p=9$ on \texttt{ibm\_torino}, and up to $p=10$ on \texttt{ibm\_pittsburgh}.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Characterization and prediction of the electromechanical wear of contact tips during wire arc additive manufacturing of 316L stainless steel

Here, this study seeks to better understand the degradation of the contact tip with respect to WAAM for a 316L wire electrode as well as explore methods of monitoring the contact tip state from process data. The contact tip, a consumable component, positions the wire and serves as the electrical contact surface between the wire electrode and the welding power supply. The wear of the contact tip was characterized in terms of material loss and material contamination for a set of tips worn to discrete levels as measured by the amount of wire fed or arc time. Geometrical characterization found a 49% increase in the bore exit area at 180 meters of wire fed. Machine learning models were developed to predict the relative bore exit area of the contact tip from arc-based process data and a random forest classifier exhibited favorable performance with a cross-validated f1-score of 0.84. The regression architecture implemented a multi-layer perceptron with the ability to predict the relative exit area with an $R^2$ score of 0.75. Key features used in the prediction include the standard deviation of the voltage and the time between shorts.

Contact tip wear↗
DOE OSTI2026

Rational design of heterogeneous single-site catalysts via surface organometallic chemistry

Single-site heterogeneous catalysts offer an attractive route to unite the molecular precision of homogeneous catalysis with the durability and practical advantages of solids. Surface organometallic chemistry (SOMC) provides a particularly powerful strategy for this purpose by grafting molecular precursors onto tailored surfaces and converting support functionalities into ligand environments for isolated metal centers. As a result, SOMC brings the language and logic of coordination chemistry to heterogeneous catalysis, where the support becomes an integral part of the active site coordination sphere. This Review surveys recent progress in the rational design of SOMC-derived single-site catalysts, with emphasis on synthetic routes, post synthetic transformations, and the deliberate tuning of catalytic behavior through metal-support interactions. Discussions are made on how support identity, hydroxyl topology, acidity, and redox activity shape the geometry, electronic structure, and oxidation state of supported metal sites, as well as how these factors determine activity, selectivity, and stability. We also examine a central limitation of these systems: despite their molecularly informed design, supported single sites often exist as structurally distributed ensembles rather than uniform species, particularly on amorphous supports. This site heterogeneity, along with catalyst dynamics under operating conditions, remains a major barrier to definitive structure-activity relationships. Therefore, emerging approaches that combine advanced characterization, first-principles modeling, ensemble kinetics, and machine learning to resolve active-site structure and guide catalyst development are highlighted. Together, these advances position SOMC as a versatile coordination chemistry framework for the predictive design of heterogeneous catalysts with well-defined molecularly tailored active sites.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Chemical-Free Lithium Separation from High-Salinity Brines Using Model-Informed and Machine Learning-Optimized Multi-Column Zwitterionic Chromatography

Direct Lithium Extraction (DLE) technologies often struggle to produce high-purity lithium salts from high-salinity brines, as current approaches require chemical-based elution, regeneration, and precipitation steps, resulting in significant environmental footprints. A novel salt fractionation approach using carboxybetaine resin, known as zwitterionic chromatography (ZIC), has demonstrated that lithium ions can be separated from divalent cations under high-salinity conditions using only water as eluent, with no regeneration required. To enable continuous and scalable deployment of this approach, we developed a chemical-free Multi-column Zwitterionic Chromatography (MZC) process and its theoretical and process models. To predict and optimize this nontraditional separation system, we introduced a novel anti-Langmuir isotherm, and the isotherm parameters were estimated through a machine learning-driven optimization based on artificial neural network ensembles with numerical feasibility assessment. Using machine learning-driven optimization, the MZC process achieved 98.0% lithium recovery, 99.5 % Li/(Li + Mg + Ca) purity, a 31.3% productivity increase, and a 33% reduction in water use compared to batch operation. The proposed MZC process enables lithium separation at $0.6-1.2 kg-1 Li, with costs dominated by resin manufacturing, while offering lower separation costs and carbon footprint compared with conventional carbonation. Overall, these findings position the MZC process as an effective polishing step within scalable and sustainable lithium production pipelines.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Using Convolutional Neural Network to Emulate Seasonal Tropical Cyclone Activity

Abstract It has been widely recognized that tropical cyclone (TC) genesis requires favorable large‐scale environmental conditions. Based on these linkages, numerous efforts have been made to establish an empirical relationship between seasonal TC activities and large‐scale environmental favorability in a quantitative way, which lead to conceptual functions such as the TC genesis index. However, due to the limited amount of reliable TC observations and complexity of the climate system, a simple analytic function may not be an accurate portrait of the empirical relationship between TCs and their ambiences. In this research, we use convolution neural networks (CNNs) to disentangle this complex relationship. To circumvent the limited amount of seasonal TC observation records, we implement transfer‐learning technique to train ensemble of CNNs first on suites of high‐resolution climate model simulations with realistic seasonal TC activities and large‐scale environmental conditions, and then on a state‐of‐the‐art reanalysis from 1950 to 2019. The trained CNNs can well reproduce the historical TC records and yields significant seasonal prediction skills when the large‐scale environmental inputs are provided by operational climate forecasts. Furthermore, by inputting the ensemble CNNs with 20th century reanalysis products and Phase 6 of the Coupled Model Intercomparison Project (CMIP6) simulations, we investigated TC variability and its changes in the past and future climates. Specifically, our ensemble CNNs project a decreasing trend of global mean TC activity in the future warming scenario, which is consistent with our future projections using high‐resolution climate model.

Meteorology & Atmospheric Sciences↗
DOE OSTI2020

Mesoscale Modeling to Characterize Eagle Soaring Habitat

Uncovering drivers of risk is crucial to understanding interactions between wildlife and wind turbines, and identifying options for impact minimization. These drivers tie to co-variates linked to behavior and movement patterns that allow us to estimate locations and periods of risk. For volant species, atmospheric flow can have significant influence on flight patterns. For obligate soaring birds, like golden eagles, updraft velocities can inform where eagles are likely to travel, at what altitude, and where conditions are not likely sufficient to sustain soaring flight. This has been an active area of study in recent years, using relatively coarse atmospheric data generally at the 20km x 20km scale or larger. Leveraging a 20-year dataset from the Weather Research and Forecasting Model (WRF) (https://www.mmm.ucar.edu/weather-research-and-forecasting-model), we are quantifying vertical velocities across the continental United States at a 2km x 2km resolution. Wind resource data sets originally were static maps showing the mean annual wind speed over an area. However, for these data sets to be optimally used for various applications they must be high-resolution time series, seamlessly span large geographic contexts, and account for uncertainty in wind speed. The National Renewable Energy Laboratory is producing public available datasets that meet these criteria and will be bias corrected to yield the most accurate wind resource data. This effort is an augment to the current WIND Toolkit which houses a high resolution data set. In the new iteration of the WIND Toolkit, a 20-year dataset will be used to improve the accuracy and estimate uncertainty using ensemble and machine-learning techniques. The resulting product will be the most accurate dataset of its size and at a 2km x 2km spatial and 5-minute temporal resolution. Through this work, a mesoscale vertical velocity layer will be produced by calculating the likelihood of orographic updraft and thermal updraft conditions across the continental United States. Specifically, we will use WRF model output combined with digital elevation maps to predict updrafts and then determine if vertical velocities are sufficient to support Golden Eagle soaring and gliding. Ultimately this data layer will be made available as a GIS layer in the Wind Prospector (maps.nrel.gov/wind-prospector/) tool or a similar framework. Data that will be incorporated include wind speed, direction temperature, relative humidity, barometric pressure, air density, precipitation rate, solar radiation, atmospheric stability, skin temperature, and upward heat flux. These products will advance research on interactions between volant species and wind energy by providing open access to highly resolved data with uncertainty quantification not previously available at this scale.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Semi-supervised Bayesian Low-shot Learning

Deep neural networks (NNs) typically outperform traditional machine learning (ML) approaches for complicated, non-linear tasks. It is expected that deep learning (DL) should offer superior performance for the important non-proliferation task of predicting explosive device configuration based upon observed optical signature, a task which human experts struggle with. However, supervised machine learning is difficult to apply in this mission space because most recorded signatures are not associated with the corresponding device description, or “truth labels.” This is challenging for NNs, which traditionally require many samples for strong performance. Semi-supervised learning (SSL), low-shot learning (LSL), and uncertainty quantification (UQ) for NNs are emerging approaches that could bridge the mission gaps of few labels and rare samples of importance. NN explainability techniques are important in gaining insight into the inferential feature importance of such a complex model. In this work, SSL, LSL, and UQ are merged into a single framework, a significant technical hurdle not previously demonstrated. Exponential Average Adversarial Training (EAAT) and Pairwise Neural Networks (PNNs) are chosen as the SSL and LSL methods of choice. Permutation feature importance (PFI) for functional data is used to provide explainability via the Variable importance Explainable Elastic Shape Analysis (VEESA) pipeline. A variety of uncertainty quantification approaches are explored: Bayesian Neural Networks (BNNs), ensemble methods, concrete dropout, and evidential deep learning. Two final approaches, one utilizing ensemble methods and one utilizing evidential learning, are constructed and compared using a well-quantified synthetic 2D dataset along with the DIRSIG Megascene.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Uncertainty guided online ensemble for non-stationary data streams in fusion science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior with distribution drifts, resulted by both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with such non-stationary data streams. Online learning techniques have been leveraged in other domains, however it has been largely unexplored for fusion applications. In this paper, we investigate online learning for continuous adaptation to drifting data streams in the prediction of Toroidal Field (TF) coils deflection at the DIII-D fusion facility. We further address the short-term performance degradation inherent to standard online learning, which arises because ground truth is unavailable at prediction time. To mitigate this issue, we propose an uncertainty-guided online ensemble framework. The method leverages the Deep Gaussian Process Approximation (DGPA) for calibrated uncertainty estimation and uses these uncertainty measures to guide a meta-algorithm that aggregates predictions from learners trained over different historical horizons. Our results show that online learning reduces prediction error by 80% compared to a static model. The online ensemble and the proposed uncertainty-guided ensemble further reduce error by approximately 6%, and 10% respectively, relative to standard single-model online learning, while also providing calibrated uncertainty estimates to support operational decision-making.

AI↗
DOE OSTI2021

Parameter inference from event ensembles and the top-quark mass

One of the key tasks of any particle collider is measurement. In practice, this is often done by fitting data to a simulation, which depends on many parameters. Sometimes, when the effects of varying different parameters are highly correlated, a large ensemble of data may be needed to resolve parameter-space degeneracies. An important example is measuring the top-quark mass, where other physical and unphysical parameters in the simulation must be profiled when fitting the top-quark mass parameter. We compare four different methodologies for top-quark mass measurement: a classical histogram fit similar to one commonly used in experiment augmented by soft-drop jet grooming; a 2D profile likelihood fit with a nuisance parameter; a machine-learning method called DCTR; and a linear regression approach, either using a least-squares fit or with a dense linearly-activated neural network. Despite the fact that individual events are totally uncorrelated, we find that the linear regression methods work most effectively when we input an ensemble of events sorted by mass, rather than training them on individual events. Although all methods provide robust extraction of the top-quark mass parameter, the linear network does marginally best and is remarkably simple. For the top study, we conclude that the Monte-Carlo-based uncertainty on current extractions of the top-quark mass from LHC data can be reduced significantly (by perhaps a factor of 2) using networks trained on sorted event ensembles. More generally, machine learning from ensembles for parameter estimation has broad potential for collider physics measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Contrasting Carbon–Water–Energy Dynamics in Perennial and Annual Bioenergy Agroecosystems Using Eddy Covariance and Interpretable Machine Learning

Understanding how agroecosystems respond to environmental variability is fundamental to predicting productivity and sustainability under a changing climate. We analyzed 55 site-years of high-frequency eddy covariance observations from five agroecosystems—two perennial grasses (miscanthus and switchgrass), two annual rotation systems (maize–soybean and sorghum–soybean), and a restored native prairie—to examine ecosystem-scale carbon, water, and energy fluxes. Using an interpretable machine-learning framework with regression tree ensembles, Shapley Additive Explanations, and Accumulated Local Effects, we quantified how environmental and temporal factors regulate gross primary productivity (GPP), evapotranspiration (ET), water-use efficiency, and the Bowen ratio. Perennials exhibited stronger physiological buffering and maintained fluxes across a broader range of temperature and moisture conditions, reflecting deeper rooting and persistent canopy cover. Annuals, in contrast, showed greater short-term variability and stronger coupling to atmospheric demand, with GPP and ET declining rapidly under low humidity or soil moisture. Differences in temperature sensitivity of Bowen ratio further revealed that perennials sustained proportionally greater sensible heat flux under cool conditions, whereas annuals exhibited constrained energy exchange when evaporative demand was low. Together, these results demonstrate that crop life cycle and canopy structure are fundamental determinants of ecosystem-scale carbon–water–energy coupling. By integrating long-term flux observations with interpretable machine learning, this study identifies the environmental drivers that shape agroecosystem function and highlights how conversion from annual to perennial feedstocks can enhance climatic resilience and alter land–atmosphere energy feedbacks. These findings provide a data-driven basis for improving crop and Earth-system models and for guiding bioenergy landscape design under future climate scenarios.

Accumulated Local Effects↗