Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gradient boosting”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Learning electric vehicle driver range anxiety with an initial state of charge-oriented gradient boosting approach

This manuscript focuses on the modeling of electric vehicle (EV) driver’s range anxiety, a fear that a vehicle does not have sufficient range, or state of charge (SOC) of the battery pack, to reach its destination and would strand its occupants. Despite numerous research studies on the modeling of charging behaviors, modeling efforts to understand at what battery percentages do EV drivers charge their vehicles, and what are the associated contributing factors, are rather limited. To this end, an ensemble learning model based on gradient boosting is developed. The model sequentially fits new predictors to new residuals of the previous prediction and, then, minimizes the loss when adding the latest prediction. A total of 18 features are defined and extracted from the multisource data, which cover information on driver, vehicles, stations, traffic conditions, as well as spatial-temporal context information of the charging events. The analyzed dataset includes 4.5-year’s charging event log data from 3,096 users and 468 public charging stations in Kansas City Missouri, and the macroscopic travel demand model maintained by the metropolitan planning organization. Here, the result shows the proposed model achieved a satisfactory result with a R square value of 0.54 and root mean square error of 0.14, both better than multiple linear regression model and random forest model. To reduce range anxiety, it is suggested that the priorities of deploying new charging facilities should be given to the areas with higher daily traffic prediction, with more conservative EV users or that are further from residential areas.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRSDate not supplied

Differential Equation Approximation Using Gradient-Boosted Quantile Regression

The operation of cyber-physical-human (CPH) systems is subject to various epistemic and aleatory uncertainties. Overall trustworthiness of CPH systems relies on the trustworthiness of its components and their interactions. It is important that computational models comprising the cyber component of CPH provide predictions accompanied by a measure of confidence in model outcomes. Uncertainty quantification (UQ) and propagation are especially important in safety critical CPH systems. Gradient-boosted trees is a modeling approach capable both of learning the dynamics of a system and performing UQ. In this paper, we devise a method for using gradient boosting to learn the dynamics of a second order differential equation and estimate uncertainty at the same time. We do this by creating a custom loss function that trains the model to approximate the second derivative of a noisy time series, and to penalize based on a parameter that corresponds to the desired quantile. The resulting gradient boosting model can simulate stochastic trajectories of the system given a single starting point, that is, it can estimate both the expected trajectory and its uncertainty. We show that the uncertainty estimation is well calibrated and that the model can learn the dynamics even in the presence of noise. We demonstrate the approach on a simple cartpole system.

Autonomous systems↗
DOE OSTI2023

An Iterative Bidirectional Gradient Boosting Algorithm for CVR Baseline Estimation

This paper presents a novel iterative, bidirectional, gradient boosting (bidirectional-GB) algorithm for estimating the baseline of the Conservation Voltage Reduction (CVR) program. We define the CVR baseline as the load profile during the CVR period if the substation voltage is not lowered. The proposed algorithm consists of two key steps: selection of similar days and iterative bidirectional-GB training. In the first step, preand post-event temperature profiles of the targeted CVR day are used to select similar days from historical non-CVR days. In the second step, the pre-event and post-event similar days are used to train two GBMs iteratively: a forward-GBM and a backwardGBM. After each iteration, the two generated CVR baselines are reconciled and only the first and the last points on the reconciled baseline are kept. The iteration repeats until all CVR baseline points are generated. We tested two gradient boosting methods (i.e., GBM and LighGBM) with two data resolutions (i.e., 15and 30-minute). The results demonstrate that both the accuracy and performance of the algorithm are satisfactory.

Lee, Han Pyo↗
DOE OSTI2024

An iterative bidirectional gradient boosting approach for CVR baseline estimation

Here this paper presents a novel Iterative Bidirectional Gradient Boosting Model (IBi-GBM) for estimating the baseline of Conservation Voltage Reduction (CVR) programs. In contrast to many existing methods, we treat CVR baseline estimation as a missing data retrieval problem. The approach involves dividing the load and its corresponding temperature profiles into three periods: pre-CVR, CVR, and post-CVR. To restore the missing load profile during the CVR period, the method employs a three-step process. First, a forward-pass GBM is executed using data from the pre-CVR period as inputs. Subsequently, a backward-pass GBM is applied using data from the post-CVR period. The two restored load profiles are reconciled, considering pre-calculated weights derived from forecasting accuracy, and only the leftmost and rightmost points are retained. The newly restored points are then included as inputs for the subsequent iteration. This iterative procedure continues until the original load data in the CVR period is fully restored. We develop IBi-GBM using actual smart meter and Supervisory Control and Data Acquisition (SCADA) data. Our results demonstrate that IBi-GBM exhibits robust performance across various data resolutions and in different seasons and outperforms existing methods by achieving a 1-2% reduction in normalized Root Mean Square Error (nRMSE).

42 ENGINEERING↗
DOE OSTI2024

Offshore application of landslide susceptibility mapping using gradient-boosted decision trees: a Gulf of Mexico case study

Abstract Among natural hazards occurring offshore, submarine landslides pose a significant risk to offshore infrastructure installations attached to the seafloor. With the offshore being important for current and future energy production, there is a need to anticipate where future landslide events are likely to occur to support planning and development projects. Using the northern Gulf of Mexico (GoM) as a case study, this paper performs Landslide Susceptibility Mapping (LSM) using a gradient-boosted decision tree (GBDT) model to characterize the spatial patterns of submarine landslide probability over the United States Exclusive Economic Zone (EEZ) where water depths are greater than 120 m. With known spatial extents of historic submarine landslides and a Geographic Information System (GIS) database of known topographical, geomorphological, geological, and geochemical factors, the resulting model was capable of accurately forecasting potential locations of sediment instability. Results of a permutation modelling approach indicated that LSM accuracy is sensitive to the number of unique training locations with model accuracy becoming more stable as the number of training regions was increased. The influence that each input feature had on predicting landslide susceptibility was evaluated using the SHapely Additive exPlanations (SHAP) feature attribution method. Areas of high and very high susceptibility were associated with steep terrain including salt basins and escarpments. This case study serves as an initial assessment of the machine learning (ML) capabilities for producing accurate submarine landslide susceptibility maps given the current state of available natural hazard-related datasets and conveys both successes and limitations.

Dyer, Alec S. (ORCID:0000000219813904)↗
DOE OSTI2024

Photon classification with Gradient Boosted Trees at CLAS12

Dihadron semi-inclusive deep inelastic scattering (SIDIS) of 10.6 GeV longitudinally polarized electrons off the proton has been measured using the CLAS12 detector at Jefferson Lab. Two separate channels, π + π 0 and π - π 0 , were analyzed, requiring the reconstruction of diphoton pairs. Here, in this analysis, we addressed the problem of false neutral particles being reconstructed by CLAS12's event builder, polluting the otherwise physical combinatorial background underneath the π 0 peak. A photon classifier using a Gradient Boosted Trees (GBTs) architecture was trained with Monte Carlo simulations to reduce the amount of background π 0 's. We show that the nearest-neighbor features learned by the model lead to a substantial increase in signal vs. background discrimination compared to previous CLAS12 π^0 analyses. The machine learning approach recovers several times more dihadron statistics for the dataset.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

A Predictive Prescription Framework for Stochastic Unit Commitment Using Boosting Ensemble Learning Algorithms

To take unit commitment (UC) decisions under uncertain load, most existing stochastic optimization (SO) frameworks adopt a generic representation of uncertainty. While load levels that materialize on a particular day are influenced by various covariates (such as the day of the week or temperature), SO frameworks typically disregard such side observations, wasting actionable information that could significantly enhance decision quality. Here, this article proposes a contextual SO (CSO) framework for UC under uncertain load, which can effectively exploit covariate observations in conjunction with a class of machine learning (ML) algorithms to improve the out-of-sample performance of UC decisions. It shows how three ML algorithms, adaptive boosting, gradient boosted trees, and extreme gradient boosting, can be used to this end, constituting the first application of these algorithms in any CSO framework. Using real-world data harvested from the New York ISO grid, we measure the out-of-sample performance of the framework in terms of total operation cost, shed load values, locational marginal prices, and total payments by the loads, against several benchmark methods proposed in the literature. The article has an online companion (Yurdakul et al.), wherein we present additional results and lay out further mathematical formulations used in this work.

42 ENGINEERING↗
NASA NTRS2023

Large-Scale Groundwater Monitoring in Brazil Assisted With Satellite-Based Artificial Intelligence Techniques

Here, we develop and test an artificial intelligence (AI)-based approach to monitor major Brazilian aquifers. The approach combines Gravity Recovery and Climate Experiment (GRACE) data and ground-based hydrogeological measurements from Brazil’s Integrated Groundwater Monitoring Network at hundreds of wells distributed in twelve aquifers across the country. We tested model ensembles based on three AI approaches: Extreme Gradient Boost, Light Gradient Boosting Model and CatBoost, followed by a Linear Regression (LR) step. The approach is further boosted with wavelet and seasonal decomposition processes applied to GRACE data. To determine the AI-based model’s sensitivity to data availability, we propose four experiments combining hydrogeological measurements from different aquifers. Groundwater storage estimates from the Global Land Data Assimilation System (GLDAS) are used as benchmark. A sensitivity analysis shows that the LR-based model ensemble is the best suited and to reproduce groundwater storage change in all studied Brazilian aquifers. Results show that the proposed approach outperforms GLDAS in all experiments, with an RMSE value of 2.68cm for the experiment that covers all monitored wells in Brazil. GLDAS resulted in RMSE=6.76cm. Using our AI model outputs, we quantified the groundwater storage change of two major aquifers, Urucuia and Bauru-Caiuá, over the past two decades: -31km 3 and -6km 3 , respectively. Water loss is driven by a prolonged drought across most of the country and intensification of groundwater pumping for irrigation. This study demonstrates that combining satellite data and AI can be a cost-effective alternative to monitor poorly equipped aquifers at the continental scale, with possible global replicability.

GRACE↗
DOE OSTI2023

Machine Learning Downscaling of SoilMERGE in the United States Southern Great Plains

SoilMERGE (SMERGE) is a root-zone soil moisture (RZSM) product that covers the entire continental United States and spans 1978 to 2019. Machine learning techniques, Random Forest (RF), eXtreme Gradient Boosting (XGBoost), and Gradient Boost (GBoost) downscaled SMERGE to spatial resolutions straddling the field scale domain (100 to 3000 m). Study area was northern Oklahoma and southern Kansas. The coarse resolution of SMERGE (0.125 degree) limits this product’s utility. To validate downscaled results in situ data from four sources were used that included: United States Department of Energy Atmospheric Radiation Measurement (ARM) observatory, United States Climate Reference Network (USCRN), Soil Climate Analysis Network (SCAN), and Soil moisture Sensing Controller and oPtimal Estimator (SoilSCAPE). In addition, RZSM retrievals from NASA’s Airborne Microwave Observatory of Subcanopy and Surface (AirMOSS) campaign provided a nearly spatially continuous comparison. Three periods were examined: era 1 (2016 to 2019), era 2 (2012 to 2015), and era 3 (2003 to 2007). During eras 1 and 2, RF outperformed XGBoost and GBoost, whereas during era 3 no model dominated. Performance was better during eras 1 and 2 as opposed to the pre-L band era 3. Improvements across all eras, regions, and models realized from downscaling included an increase in correlation from 0.03 to 0.42 and a decrease in ub RMSE from -0.0005 to -0.0118 m 3 /m 3 . This study demonstrates the feasibility of SMERGE downscaling opening the prospect for the development of a long-term RZSM dataset at a more desirable field-scale resolution with the potential to support diverse hydrometeorological and agricultural applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Using Machine Learning to Predict Cloud Turbulent Entrainment–Mixing Processes

Different turbulent entrainment–mixing mechanisms between clouds and environment are essential to cloud–related processes; however, accurate representation of entrainment–mixing in weather/climate models still poses a challenge. This study exploits the use of machine learning (ML) to address this challenge. Four ML (Light Gradient Boosting Machine [LGB], eXtreme Gradient Boosting, Random Forest, and Support Vector Regression) are examined and compared. It is found that LGB performs best, and thus is selected to understand the impact of entrainment–mixing on microphysics using simulation data from Explicit Mixing Parcel Model. Compared with traditional parameterizations, the trained LGB provides more accurate microphysical properties (number concentration and cloud droplet spectral dispersion). The partial dependences of predicted microphysics on features exhibit a strong alignment with physical mechanisms and expectations, as determined by the interpreting method, thus overcoming the limitations of the “black box” scheme. The underlying mechanisms are that the smaller number concentration and larger spectral dispersion correspond to more inhomogeneous entrainment–mixing. Specifically, number concentration after entrainment–mixing is positively correlated with adiabatic number concentration and liquid water content affected by entrainment–mixing, and inversely correlated with adiabatic volume mean radius. Spectral dispersion after entrainment–mixing is negatively correlated with liquid water content affected by entrainment–mixing, turbulent dissipation rate and relative humidity of entrained air. Sensitivity analysis further suggests that number concentration is mainly determined by cloud microphysical properties whereas spectral dispersion is influenced by both cloud microphysical properties and environmental variables. The results indicate that the LGB scheme has the potential to enhance the representation of entrainment–mixing in weather/climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Predictive understanding of the surface tension and velocity of sound in ionic liquids using machine learning

Knowledge of the physical properties of ionic liquids (ILs), such as the surface tension and speed of sound, is important for both industrial and research applications. Unfortunately, technical challenges and costs limit exhaustive experimental screening efforts of ILs for these critical properties. Previous work has demonstrated that the use of quantum-mechanics-based thermochemical property prediction tools, such as the conductor-like screening model for real solvents, when combined with machine learning (ML) approaches, may provide an alternative pathway to guide the rapid screening and design of ILs for desired physiochemical properties. However, the question of which machine-learning approaches are most appropriate remains. In the present study, we examine how different ML architectures, ranging from tree-based approaches to feed-forward artificial neural networks, perform in generating nonlinear multivariate quantitative structure–property relationship models for the prediction of the temperature- and pressure-dependent surface tension of and speed of sound in ILs over a wide range of surface tensions (16.9–76.2 mN/m) and speeds of sound (1009.7–1992 m/s). The ML models are further interrogated using the powerful interpretation method, shapley additive explanations. We find that several different ML models provide high accuracy, according to traditional statistical metrics. The decision tree-based approaches appear to be the most accurate and precise, with extreme gradient-boosting trees and gradient-boosting trees being the best performers. However, our results also indicate that the promise of using machine-learning to gain deep insights into the underlying physics driving structure–property relationships in ILs may still be somewhat premature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Machine learning for fundamental spectroscopic and thermodynamic data of actinides and lanthanides

Accurately modeling optical spectra with absolute radiometric intensities is vital for nuclear forensics applications that depend on characterizing optical emissions from energetic nuclear phenomena. This requires precise knowledge of the individual atomic transition probabilities, known as Einstein A-coefficients, for each emission line. Obtaining these values theoretically or experimentally is often impractical due to the complex electronic structures and the number of transitions involved in atoms relevant to nuclear applications. In this study, we explore the use of machine learning to predict the Einstein A coefficients for atomic transitions. Seven models were evaluated that ranged from deep learning to decision tree algorithms, and found that gradient boosting performed best, specifically the Extreme Gradient Boosting (XGB) architecture, achieving a precision of 86% across transitions of 36 elements. Furthermore, the model was cross-validated using published transition probabilities reported in the literature and applied to estimate Pu plasma temperatures from a previous experiment conducted at Savannah River National Laboratory.

Atomic spectroscopy↗
DOE OSTI2021

Automatic Search of Cataclysmic Variables Based on LightGBM in LAMOST-DR7

The search for special and rare celestial objects has always played an important role in astronomy. Cataclysmic Variables (CVs) are special and rare binary systems with accretion disks. Most CVs are in the quiescent period, and their spectra have the emission lines of Balmer series, HeI, and HeII. A few CVs in the outburst period have the absorption lines of Balmer series. Owing to the scarcity of numbers, expanding the spectral data of CVs is of positive significance for studying the formation of accretion disks and the evolution of binary star system models. At present, the research for astronomical spectra has entered the era of Big Data. The Large Sky Area Multi-Object Fiber Spectroscopy Telescope (LAMOST) has produced more than tens of millions of spectral data. the latest released LAMOST-DR7 includes 10.6 million low-resolution spectral data in 4926 sky regions, providing ideal data support for searching CV candidates. To process and analyze the massive amounts of spectral data, this study employed the Light Gradient Boosting Machine (LightGBM) algorithm, which is based on the ensemble tree model to automatically conduct the search in LAMOST-DR7. Finally, 225 CV candidates were found and four new CV candidates were verified by SIMBAD and published catalogs. This study also built the Gradient Boosting Decision Tree (GBDT), Adaptive Boosting (AdaBoost), and eXtreme Gradient Boosting (XGBoost) models and used Accuracy, Precision, Recall, the F1-score, and the ROC curve to compare the four models comprehensively. Experimental results showed that LightGBM is more efficient. The search for CVs based on LightGBM not only enriches the existing CV spectral library, but also provides a reference for the data mining of other rare celestial objects in massive spectral data.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRSDate not supplied

Uncertainty Quantification using Deep Ensembles for Decision Making in Cyber-Physical-Human Systems

In this paper and its companion, Differential Equation Approximation Using Gradient-Boosted Quantile Regression, Robison et al., we examine an approach to quantifying model uncertainty with the aim of increasing the trustworthiness of computational models in human-machine interactions. In Differential Equation Approximation Using Gradient-Boosted Quantile Regression, we focus on gradient-boosted decision trees, while in this one, we give more details about deep ensembles. Uncertainty quantification is crucial for building trustworthy autonomous decision-making agents in human-machine teams. There are two types of uncertainties: aleatoric and epistemic. The former is related to the inherent stochasticity (noise) of the process, whereas the latter is associated with the lack of knowledge or representation capability of models, such as neural networks. By lack of knowledge, we mean the model’s inability to accurately predict outputs for all possible inputs. The aleatory uncertainty can be estimated fairly easily with, for example, filters, whereas epistemic uncertainty is challenging to compute. This paper uses deep ensembles to quantify both aleatory and epistemic uncertainty. It can act as an uncertainty-aware surrogate transition model for decision-making frameworks. "Uncertainty-aware" means that the surrogate transition model should make predictions along with confidence in those predictions. In the context of decision-making, the transition models are ordinary differential equations (ODEs). Since ODEs can be simulated to make one-step or multi-step predictions, a good surrogate model for them should perform reasonably well in both modes. In a multi-step approach, the trajectory sampling method TS∞ was used to propagate uncertainty over multiple steps. The cartpole dynamical system was selected to demonstrate the ability of deep ensembles as good surrogate transition models for decision-making frameworks. The deep ensembles modeled the dynamics of cartpole ODEs and made uncertainty-aware predictions in single-step and multi-step transition modes.

CPH systems↗
DOE OSTI2025

Mapping tree height in complex terrain of northern China using ultra-high-resolution images

Tree height is a key parameter for estimating forest biomass and carbon sequestration. In recent years, notable progress has been made in mapping tree height using satellite imagery. However, existing tree height products show low accuracy in mountainous and complex terrains, and few studies typically addressed tree height estimations in mountain areas. This study examines the Mentougou district of Beijing, China, characterized by complex terrain and mountainous landscapes. We analyzed two methods for estimating tree height: one using only spectral features and another combining spectral features with topographic factors (elevation, slope, aspect). We used 3-m resolution PlanetScope 8-band multispectral imagery, with 710 field-measured individual tree heights averaged to obtain 471 pixel-level tree height values as ground-truth, to develop tree height prediction models using eXtreme Gradient Boosting (XGBoost), Random Forest (RF), and Gradient Boosting Machine (GBM) models. The results show that the XGBoost model consistently presented the highest accuracy for both methods evaluated. Specifically, the XGBoost model that combined spectral data with elevation and slope variables with an R² of 0.75 and an RMSE of 2.69 m. Using the XGBoost model, we generated the tree height map for the Mentougou area at 3 m resolution, showing tree heights ranging from 0.5 to 30.4 m, and the model’s prediction error standard deviations ranged from 2.50 to 4.71 m, indicating reliable performance across varied terrain. Additionally, we compared and evaluated the global tree height products, identifying limitations in the accuracy within complex terrains. This study demonstrates the potential for accurately predicting tree heights by combining high-resolution multispectral satellites with a terrain factor modeling approach.

Complex terrain↗
DOE OSTI2024

Addressing bias in bagging and boosting regression models

As artificial intelligence (AI) becomes widespread, there is increasing attention on investigating bias in machine learning (ML) models. Previous research concentrated on classification problems, with little emphasis on regression models. This paper presents an easy-to-apply and effective methodology for mitigating bias in bagging and boosting regression models, that is also applicable to any model trained through minimizing a differentiable loss function. Our methodology measures bias rigorously and extends the ML model's loss function with a regularization term to penalize high correlations between model errors and protected attributes. We applied our approach to three popular tree-based ensemble models: a random forest model (RF), a gradient-boosted model (GBT), and an extreme gradient boosting model (XGBoost). We implemented our methodology on a case study for predicting road-level traffic volume, where RF, GBT, and XGBoost models were shown to have high accuracy. Despite high accuracy, the ML models were shown to perform poorly on roads in minority-populated areas. Our bias mitigation approach reduced minority-related bias by over 50%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Learning-based CO 2 concentration prediction: Application to indoor air quality control using demand-controlled ventilation

There have been increasing concerns over the air quality inside buildings as high levels of bio-effluents can cause nausea, dizziness, headaches, and fatigue to the people working in those spaces. First published in 2004 as Standard 62.1, ASHRAE Standard 62.2-2019 requires highly occupied spaces to implement heating, ventilation, and air conditioning (HVAC) that can dilute contaminants produced by occupants. In this regard, occupant-centric ventilation control has been regarded as an effective practice to maintain a satisfactory indoor air quality (IAQ) when dealing with highly variable occupancy environments. However, few established models in current literature and practice consider dynamic occupancy behavior and adaptive IAQ control. To address this gap, a dynamic indoor CO2 model is constructed using machine learning algorithms to forecast CO2concentrations across a range of forecasting horizons. Herein, we tuned and compared six state-of-the-algorithms—including Support Vector Machine, Ada Boost, Random Forest, Gradient Boosting, Logistic Regression, and Multilayer Perceptron. The algorithms’ performances are validated using CO 2 and historical meteorological data collected from a campus classroom with a variable occupancy rate. Simulation results showed that Multilayer Perceptron can strongly predict the volatile CO 2 behavior and also outperforms other algorithms in terms of accuracy. Furthermore, a control strategy capable of modeling and detecting dynamic patterns of CO 2 level is utilized to modulate the ventilation rate in real-time and also reduce the energy consumption. The proposed controller reduced the HVAC fan’s energy consumption by 51.4% and provide ventilation as needed per the ASHRAE standards.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗