Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

FY21 Progress Report: SRNL Analysis of ICCWR LCM and WAMS data for Corrosion and Cracking

The development of algorithms for machine learning and data analysis for the 3013 Surveillance Program is a collaborative effort by the Savannah River National Laboratory (SRNL) and the University of South Carolina (USC). For corrosion detection, Laser Confocal Microscope (LCM) or Wide Area 3D Measurement System (WAMS) data is extracted from large binary files, with software written to convert the data to physical attributes (e.g., height, color and grayscale values; all as functions of a location in a plane projection). A user-friendly Matlab Graphical User Interface (GUI) that reads data from either LCM or WAMS files was developed to integrate input data with software developed for processing and evaluation. The GUI can selectively download binary data, interrogate data attributes, label data, flag significant features, execute Machine Learning (ML) algorithms, output parameters for trained ML algorithms, report ML model accuracy with respect to labeled data, and generate graphical representations for various analyses. Features can be called out by user-specified thresholds, manual labeling or machine learning algorithms when they have been completed. The ability to rapidly label data is important because of the volume of data required for training machine learning algorithms. The GUI has the flexibility to allow addition of improved ML algorithms, methods for data visualization, and statistical computations. Statistical analyses via the GUI include areas of pits within a defined range of pit depths, correlations between Red-Green-Blue (RGB) or grayscale intensity and relative surface height, covariances between values associated with features, and feature histograms. The development of supervised machine learning algorithms, however, has been hindered by a lack of training data. The machine learning algorithms for crack identification are being refined but require improvements to the true positive rate for crack detection. This shortcoming is an artifact of the limited training data currently available, perhaps more so than the structure of the neural networks. At present, the best results are had from a consensus over an ensemble of randomly generated Deep Neural Network (DNN) or Convolutional Neural Network (CNN) algorithms. Although the consensus accuracy method has yielded optimum true positive and true negative rates in excess of 80%, additional validation testing is necessary. In addition to the suite of LCM data that was initially used, and which represents the majority of the work presented in this report, WAMS image data was also reviewed at a preliminary level. The review included a comparison between image resolution and dynamic range for each method. WAMS (ZON file) image data was found to have a pixel pitch of 3.69μm compared to 1 μm for the LCM (vk4 file) data, which implies a lower resolution for the WAMS images. Conversely, the ratio of dynamic range of the WAMS data to the LCM data was approximately 41:20 for height data, suggesting that information from WAMS should more accurately determine the depth of pits. At present, the significance of the greater dynamic range of the WAMS data relative to the LCM data has not yet been evaluated.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Neural network representation of electronic structure from ab initio molecular dynamics

Despite their rich information content, electronic structure data amassed at high volumes in ab initio molecular dynamics simulations are generally under-utilized. Here we introduce a transferable high-fidelity neural network representation of such data in the form of tight-binding Hamiltonians for crystalline materials. This predictive representation of ab initio electronic structure, combined with machine-learning boosted molecular dynamics, enables efficient and accurate electronic evolution and sampling. When it is applied to a one-dimension charge-density wave material, carbyne, we are able to compute the spectral function and optical conductivity in the canonical ensemble. The spectral functions evaluated during soliton-antisoliton pair annihilation process reveal significant renormalization of low-energy edge modes due to retarded electron-lattice coupling beyond the Born–Oppenheimer limit. The availability of an efficient and reusable surrogate model for the electronic structure dynamical system will enable calculating many interesting physical properties, paving the way to previously inaccessible or challenging avenues in materials modeling.

74 ATOMIC AND MOLECULAR PHYSICS↗
NASA NTRS2024

Landslide Hazard is Projected to Increase Across High Mountain Asia

High Mountain Asia has long been known as a hotspot for landslide risk, and studies have suggested that landslide hazard is likely to increase in this region over the coming decades. Extreme precipitation may become more frequent, with a nonlinear response relative to increasing global temperatures. However, these changes are geographically varied. This article maps probable changes to landslide hazard, as shown by a landslide hazard indicator (LHI) derived from downscaled precipitation and temperature. In order to capture the nonlinear response of slopes to extreme precipitation, a simple machine-learning model was trained on a database of landslides across High Mountain Asia to develop a regional LHI. This model was applied to statistically downscaled data from the 30 members of the Seamless System for Prediction and Earth System Research large ensembles to produce a range of possible outcomes under the Shared Socioeconomic Pathways 2-4.5 and 5-8.5. The LHI reveals that landslide hazard will increase in most parts of High Mountain Asia. Absolute increases will be highest in already hazardous areas such as the Central Himalaya, but relative change is greatest on the Tibetan Plateau. Even in regions where landslide hazard declines by year 2100, it will increase prior to the mid-century mark. However, the seasonal cycle of landslide occurrence will not change greatly across High Mountain Asia. Although substantial uncertainty remains in these projections, the overall direction of change seems reliable. These findings highlight the importance of continued analysis to inform disaster risk reduction strategies for stakeholders across High Mountain Asia.

Thomas A Stanley↗
DOE OSTI2021

A multi-dimensional parametric study of variability in multi-phase flow dynamics during geologic CO 2 sequestration accelerated with machine learning

Successful geologic CO 2 storage projects depend on numerical simulations to predict reservoir performance during site selection, injection verification, and post-injection monitoring phases of the project. These numerical simulations solve non-linear sets of coupled partial differential equations, while accounting for multi-phase fluid dynamics on the basis of constitutive equations that are embedded into the solution scheme. As a consequence, individual simulations often require tens to hundreds of hours to complete on high-performance computing clusters. Moreover, laboratory experiments reveal that parametric functions for capillary pressure and relative permeability exhibit substantial variability, even within the same rock type. This combination of computational expense and wide-ranging parametric variability means that there remains substantial uncertainty in the behavior of multi-phase CO 2 -water systems, particularly in the context of feedbacks between relative permeability and capillary pressure. To bridge this knowledge gap, here we develop a novel workflow that utilizes physics-based numerical simulation to train an artificial neural network (ANN) emulator for interrogating the multivariate parameter space that governs both capillary pressure and relative permeability. With this approach, the ANN is trained to emulate both fluid pressure distribution and CO 2 saturation, which are then interrogated quantitatively to generate parametric response surface mappings with high-fidelity resolution. Results from this study initially show that capillary entry pressure is the dominant control on both CO 2 plume geometry and fluid pressure propagation when considering the combined effects of capillary pressure and relative permeability, particularly when phase interference is low and residual CO 2 saturation is high. Moreover, the ANN emulator provides tremendous computational speed-up by computing 2691 individual simulations in several minutes; whereas, the same simulation ensemble would have required ~3 years of simulation time using only physics-based simulation methods (25,000 times speed up).

58 GEOSCIENCES↗
DOE OSTI2024

Spectral Data Fusion From Handheld Laser-Induced Breakdown Spectroscopy (LIBS) and X-ray Fluorescence (XRF) Analyzers for Improved Detection of Cerium in a Simulated Dispersal Accident

Here, this work implements a mid-level data fusion methodology on spectral data from handheld X-ray fluorescence and laser-induced breakdown spectroscopy analyzers to quantify plutonium surrogate (CeO 2 ) contamination in soil samples for the first time. Spectral data from each analyzer were used independently to train supervised machine learning regressions to predict Ce concentration. Fused features from both data sets were then used to train the same models, comparing prediction performance by evaluating model precision and sensitivity. Fusing principal component scores from the two sensors yielded an order of magnitude improvement in precision and sensitivity of predictions made with an artificial neural network, compared to predictions made by models trained on independent sensor data. As a result, a boosted ensemble trained on the fused spectral features yielded an ideal predictor with root-mean-squared error on the order of 10 –6 and calculated limit of detection order 10 –5 wt %.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Revealing the evolution of order in materials microstructures using multi-modal computer vision

The development of high-performance materials for microelectronics, energy storage, and extreme environments depends on our ability to describe and direct property-defining microstructural order. Our present understanding is typically derived from laborious manual analysis of imaging and spectroscopy data, which is difficult to scale, challenging to reproduce, and lacks the ability to reveal latent associations needed for mechanistic models. Here, we demonstrate a multi-modal machine learning (ML) approach to describe order from electron microscopy analysis of the complex oxide La 1−x Sr x FeO 3 . We construct a hybrid pipeline based on fully and semi-supervised classification, allowing us to evaluate both the characteristics of each data modality and the value each modality adds to the ensemble. We observe distinct differences in the performance of uni- and multi-modal models, from which we draw general lessons in describing crystal order using computer vision.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Prediction of Distributed River Sediment Respiration Rates Using Community-Generated Data and Machine Learning

River sediment microbial respiration is a key indicator of ecosystem functioning and the biogeochemical fluxes across this critical zone link surface and subsurface waters. As such, there is tremendous interest in measuring and mapping these respiration rates. Respiration observations are expensive and labor intensive; there is limited data available to the community. An open science, collaborative initiative is collecting samples for respiration rate analysis and multi-scale metadata; this evolving data set is being used for making machine learning (ML) predictions at unsampled sites to help inform continued community engagement. However, it is a challenge to find an optimum configuration for ML models to work with this feature-rich (i.e., 100+ possible input variables) data set. Here, we present results from a two-tiered approach to managing the analysis of this complex data set: (a) a stacked ensemble of models that automatically optimizes hyperparameters and manages the training of many models and (b) feature permutation importance to detect the most important features in the models. The major elements of this workflow are modular, portable, open, and cloud-based thus making this implementation a potential template for other applications. The models developed here predict that sediment organic matter chemistry is one of the most important features for predicting sediment respiration rate. Other larger-scale, important features fall into the categories of climatic, ecological, geological, and fluvial settings. Leveraging these larger-scale features to generate data-driven estimates of river sediment respiration rates reveals spatially consistent but heterogeneous patterns across the river network of the Columbia River Basin.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Detection of Diversion in a Realistic Heat Pipe Microreactor Using Supervised Machine Learning

Microreactors (MRs) pose new challenges for international safeguards. Here, their small size and mass reproducibility make them ideal for deployment in greater numbers and in remote locations, making the job of safeguards inspectors more challenging. Machine learning (ML) is currently being applied to many fields to augment human performance and increase automation; in particular, ML could be used to provide insight for international inspectors to help detect the diversion of nuclear fuel from MR cores. Four ML model types (k-nearest neighbors, decision tree, random forest, and histogram-based gradient boosted ensemble) were trained on integrated flux and critical control drum angle data generated with Serpent 2 for a realistic heat pipe MR design, achieving nearly 100% binary classification accuracy of nominal and diversion core configurations by the end of 1 full power year for three of the four model types. Regression model variants were also trained, using the same input data, for predicting the number of fuel pins diverted. Root-mean-square errors below 5% of the total number of fuel pins were achieved by the 1 full power year mark for all models.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Loosely Conditioned Emulation of Global Climate Models With Generative Adversarial Networks

Climate models encapsulate our best understanding of the Earth system, allowing research to be conducted on its future under alternative assumptions of how human-driven climate forces are going to evolve. An important application of climate models is to provide metrics of mean and extreme climate changes, particularly under these alternative future scenarios, as these quantities drive the impacts of climate on society and natural systems. Because of the need to explore a wide range of alternative scenarios and other sources of uncertainties in a computationally efficient manner, climate models can only take us so far, as they require significant computational resources, especially when attempting to characterize extreme events, which are rare and thus demand long and numerous simulations in order to accurately represent their changing statistics. Here we use deep learning in a proof of concept that lays the foundation for emulating global climate model output for different scenarios. We train two "loosely conditioned" Generative Adversarial Networks (GANs) that emulate daily precipitation output from a fully coupled Earth system model: one GAN modeling Fall-Winter behavior and the other Spring-Summer. Our GANs are trained to produce spatiotemporal samples: 32 days of precipitation over a 64x128 regular grid discretizing the globe. We evaluate the generator with a set of related performance metrics based upon KL divergence, and find the generated samples to be nearly as well matched to the test data as the validation data is to test. We also find the generated samples to accurately estimate the mean number of dry days and mean longest dry spell in the 32 day samples. Our trained GANs can rapidly generate numerous realizations at a vastly reduced computational expense, compared to large ensembles of climate models, which greatly aids in estimating the statistics of extreme events.

climate emulation, extreme climate, impacts, machi↗
DOE OSTI2022

Application of Dimensionality Reduction in Machine Learning Modeling of CO2 Storage

In this study, we developed deep learning models that are capable of predicting spatio-temporal outputs of CO2 saturation, pressure, and brine production in a 3D saline storage reservoir over 30 years of continuous CO2 injection and a 50-year post-injection timeframe. To improve computational efficiency and maintain performance accuracy, the model framework involves ensembling multi-layer autoencoder networks that provide dimensionality reduction of geologic inputs with fully connected long short-term memory (LSTM) neural networks that generate time-series prediction. This study was presented as poster at the 2022 Carbon Management Project Review Meeting held in Pittsburgh, PA (August 15 – 19, 2022).

Bello, Kolawole↗
DOE OSTI2021

Scalable statistical inference of photometric redshift via data subsampling

Handling big data has largely been a major bottleneck in traditional statistical models. Consequently, when accurate point prediction is the primary target, machine learning models are often preferred over their statistical counterparts for bigger problems. But full probabilistic statistical models often outperform other models in quantifying uncertainties associated with model predictions. We develop a data-driven statistical modeling framework that combines the uncertainties from an ensemble of statistical models learned on smaller subsets of data carefully chosen to account for imbalances in the input space. We demonstrate this method on a photometric redshift estimation problem in cosmology, which seeks to infer a distribution of the redshift—the stretching effect in observing the light of far-away galaxies—given multivariate color information observed for an object in the sky. Our proposed method performs balanced partitioning, graph-based data subsampling across the partitions, and training of an ensemble of Gaussian process models.

data subsampling↗
DOE OSTI2020

Spatial Mapping of Riverbed Grain-Size Distribution Using Machine Learning

Recent alluvial sediments in riverbeds play a significant role in controlling hydrologic exchange flows (HEFs) in river systems. The alluvial layer is usually associated with strong heterogeneity in physical properties (e.g., permeability and hydraulic conductivity), which affects local HEFs and therefore biogeochemical processes. The spatial distribution of these physical properties needs to be determined to inform the numerical models used to reveal the realistic hydro-biogeochemical behaviors. Such information can be obtained based on the intrinsic link between sediment grain-size distribution and hydraulic properties where sediment texture information is available. However, grain-size measurements are usually spatially sparse and do not have adequate coverage and resolution, particularly for a relatively large domain such as the Hanford Reach of the Columbia River. In this paper, we adopted machine learning (ML) approaches for categorizing and mapping the spatial distributions of riverbed substrate grain size and filling in missing areas of substrate data using the ML models along the reach. Such ML models for substrate size mapping were trained at 13,372 locations using measured substrate sizes along with observed and simulated attributes, including bathymetric attributes (e.g., elevation, slope, and aspect ratio) from LIDAR and bathymetric surveys, and hydrodynamic properties (e.g., water depth, velocity, shear stress, and their statistical moments). An ensemble bagging-based ML technique, Random Forest, was adopted to identify the most influential factors as predictors to develop the predictive models with over-fitting issues addressed. The models were evaluated with respect to each individual substrate size class and the lumped group, and then used to generate the final substrate size maps covering all the grid cells in the numerical modeling domain.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Machine learning-based observation-constrained projections reveal elevated global socioeconomic risks from wildfire

Reliable projections of wildfire and associated socioeconomic risks are crucial for the development of efficient and effective adaptation and mitigation strategies. The lack of or limited observational constraints for modeling outputs impairs the credibility of wildfire projections. Here, we present a machine learning framework to constrain the future fire carbon emissions simulated by 13 Earth system models from the Coupled Model Intercomparison Project phase 6 (CMIP6), using historical, observed joint states of fire-relevant variables. During the twenty-first century, the observation-constrained ensemble indicates a weaker increase in global fire carbon emissions but higher increase in global wildfire exposure in population, gross domestic production, and agricultural area, compared with the default ensemble. Such elevated socioeconomic risks are primarily caused by the compound regional enhancement of future wildfire activity and socioeconomic development in the western and central African countries, necessitating an emergent strategic preparedness to wildfires in these countries.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Machine learning-driven descriptions of protein dynamics at solid-liquid interfaces

This chapter has described how ML has enabled quantitative analysis of HS-AFM data to discover the physical phenomena governing protein dynamics and ordering at solid-liquid interfaces. The research detailed in this chapter modeled the rotation models of protein nanorods, the discovery of which would otherwise not be possible. By tracking the trajectories of individual protein rods from frame to frame, it was possible to model Brownian type motion and behaviors and Levy-flight dynamics that had not previously been shown. We also described the application of the Python package AtomAI, which has been developed specifically to analyze and extract physical phenomena, providing exemplar code for training an ensemble of deep neural networks to produce the semantic segmentation of AFM data and functions for encoding and decoding local environments. We last described a combinatorial approach to analyze very noisy data with a densely covered substrate where the emergence of order for the protein liquid crystals could be elucidated. By combining the methods from Case 1 and 2, it was possible to obtain the center of mass and angle for each rod in the images and track the assembly of the rods over time into a 2D liquid crystal array on the surface of mica.

protein dynamics, solid-liquid interfaces, atomic ↗
DOE OSTI2023

Fusion of Multiple Models for Improving Gross Primary Production Estimation With Eddy Covariance Data Based on Machine Learning

Abstract Terrestrial gross primary production (GPP) represents the magnitude of CO 2 uptake through vegetation photosynthesis, and is a key variable for carbon cycles between the biosphere and atmosphere. Light use efficiency (LUE) models have been widely used to estimate GPP for its physiological mechanisms and availability of data acquisition and implementation, yet each individual GPP model has exhibited large uncertainties due to input errors and model structure, and further studies of systematic validation, comparison, and fusion of those models with eddy covariance (EC) site data across diverse ecosystem types are still needed in order to further improve GPP estimation. We here compared and fused five GPP models (VPM, EC‐LUE, GOL‐PEM, CHJ, and C‐Fix) across eight ecosystems based on FLUXNET2015 data set using the ensemble methods of Bayesian Model Averaging (BMA), Support Vector Machine (SVM), and Random Forest (RF) separately. Our results showed that for individual models, EC‐LUE gave a better performance to capture interannual variability of GPP than other models, followed by VPM and GLO‐PEM, while CHJ and C‐Fix were more limited in their estimation performance. We found RF and SVM were superior to BMA on merging individual models at various plant functional types (PFTs) and at the scale of individual sites. On the basis of individual models, the fusion methods of BMA, SVM, and RF were examined by a five‐fold cross validation for each ecosystem type, and each method successfully improved the average accuracy of estimation by 8%, 18%, and 19%, respectively.

Environmental Sciences & Ecology↗
DOE OSTI2024

Addressing bias in bagging and boosting regression models

As artificial intelligence (AI) becomes widespread, there is increasing attention on investigating bias in machine learning (ML) models. Previous research concentrated on classification problems, with little emphasis on regression models. This paper presents an easy-to-apply and effective methodology for mitigating bias in bagging and boosting regression models, that is also applicable to any model trained through minimizing a differentiable loss function. Our methodology measures bias rigorously and extends the ML model's loss function with a regularization term to penalize high correlations between model errors and protected attributes. We applied our approach to three popular tree-based ensemble models: a random forest model (RF), a gradient-boosted model (GBT), and an extreme gradient boosting model (XGBoost). We implemented our methodology on a case study for predicting road-level traffic volume, where RF, GBT, and XGBoost models were shown to have high accuracy. Despite high accuracy, the ML models were shown to perform poorly on roads in minority-populated areas. Our bias mitigation approach reduced minority-related bias by over 50%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Gearbox bearing crack growth prognostics and uncertainty quantification with physics-informed machine learning

This paper introduces the extreme theory of functional connections (X-TFC), a physics-informed machine learning algorithm, and tailors it to estimate the remaining useful life (RUL) of wind turbine gearbox bearings experiencing fatigue crack growth. Unlike purely data-driven methods, X-TFC embeds a physics model, based on Head's theory in this work, into its training objective. The core of X-TFC is a random-projection single-layer neural network trained via an extreme learning machine, which requires only limited damage progression data and solves for output weights with a least-squares optimization algorithm. A composite loss function balances the network's fit to observed degradation data against the residuals of the governing crack growth differential equation, ensuring the learned damage trajectory remains physically plausible. When applied to a vibration-based health-index (HI) dataset measured during the growth of a crack on the inner ring of a high-speed bearing in a wind turbine gearbox (Bechhoefer and Dubé, 2020), X-TFC achieves near-zero prediction bias. Even when trained on only the first 10 %–20 % of the damage progression data, with sufficient physics weighting its predictions remain monotonic and smooth, delivering high prognosability and trendability. To quantify the epistemic uncertainty, we employ a Monte Carlo ensemble of independently initialized X-TFC models trained on noise-perturbed data, which yields confidence intervals around each RUL estimate and captures both model-parameter and epistemic uncertainty. In addition to a vibration-based HI, we demonstrate that the proposed framework can be directly applied to a supervisory control and data acquisition (SCADA) data-based HI (Eftekhari Milani et al., 2026) measured during similar wind turbine gearbox bearing crack faults, preserving its accuracy and interpretability. This extension shows the versatility of our approach, which is applicable to bearings of multiple gearbox manufacturers, models, and ratings using only SCADA data. By integrating domain knowledge with machine learning, X-TFC offers a rapid, reliable tool for crack prognostics. Its adaptability to other bearing failure modes, such as pitch bearing ring cracks, positions X-TFC as a powerful enabler of data-driven, physics-informed asset management in the wind energy sector and beyond.

17 WIND ENERGY↗