Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “decision trees”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Tree-based algorithms for weakly supervised anomaly detection

Weakly supervised methods have emerged as a powerful tool for model-agnostic anomaly detection at the Large Hadron Collider (LHC). While these methods have shown remarkable performance on specific signatures such as dijet resonances, their application in a more model-agnostic manner requires dealing with a larger number of potentially noisy input features. In this paper, we show that using boosted decision trees as classifiers in weakly supervised anomaly detection gives superior performance compared to deep neural networks. Boosted decision trees are well known for their effectiveness in tabular data analysis. Our results show that they not only offer significantly faster training and evaluation times, but they are also robust to a large number of noisy input features. By using advanced gradient boosted decision trees in combination with ensembling techniques and an extended set of features, we significantly improve the performance of weakly supervised methods for anomaly detection at the LHC. This advance is a crucial step toward a more model-agnostic search for new physics. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

The analysis of rapidly developing fog at the Kennedy Space Center

This report documents fog precursors and fog climatology at Kennedy Space Center (KSC) Florida from 1986 to 1990. The major emphasis of this report focuses on rapidly developing fog events that would affect the less than 7-statute mile visibility rule for End-Of-Mission (EOM) Shuttle landing at KSC (Rule 4-64(A)). The Applied Meteorology Unit's (AMU's) work is to: develop a data base for study of fog associated weather conditions relating to violations of this landing constraint; develop forecast techniques or rules-of-thumb to determine whether or not current conditions are likely to result in an acceptable condition at landing; validate the forecast techniques; and transition techniques to operational use. As part of the analysis the fog events were categorized as either advection, pre-frontal or radiation. As a result of these analyses, the AMU developed a fog climatological data base, identified fog precursors and developed forecaster tools and decision trees. The fog climatological analysis indicates that during the fog season (October to April) there is a higher risk for a visibility violation at KSC during the early morning hours (0700 to 1200 UTC), while 95 percent of all fog events have dissipated by 1600 UTC. A high number of fog events are characterized by a westerly component to the surface wind at KSC (92 percent) and 83 percent of the fog events had fog develop west of KSC first (up to 2 hours). The AMU developed fog decision trees and forecaster tools that would help the forecaster identify fog precursors up to 12 hours in advance. Using the decision trees as process tools ensures the important meteorological data are not overlooked in the forecast process. With these tools and a better understanding of fog formation in the local KSC area, the Shuttle weather support forecaster should be able to give the Launch and Flight Directors a better KSC fog forecast with more confidence.

Wheeler, Mark M.↗

A Model Tree Generator (MTG) Framework for Simulating Hydrologic Systems: Application to Reservoir Routing

Data-driven algorithms have been widely used as effective tools to mimic hydrologic systems. Unlike black-box models, decision tree algorithms offer transparent representations of systems and reveal useful information about the underlying process. A popular class of decision tree models is model tree (MT), which is designed for predicting continuous variables. Most MT algorithms employ an exhaustive search mechanism and a pre-defined splitting criterion to generate a piecewise linear model. However, this approach is computationally intensive, and the selection of the splitting criterion can significantly affect the performance of the generated model. These drawbacks can limit the application of MTs to large datasets. To overcome these shortcomings, a new flexible Model Tree Generator (MTG) framework is introduced here. MTG is equipped with several modules to provide a flexible, efficient, and effective tool for generating MTs. The application of the algorithm is demonstrated through simulation of controlled discharge from several reservoirs across the Contiguous United States (CONUS).

54 ENVIRONMENTAL SCIENCES↗

Avatar Tools

Supervised machine learning is the process of using past experience to predict the future. "Ensembles" are a machine-learning meta-method that can be applied to most machine learning algorithms. Ensembles generally greatly improve accuracy, reduce or remove most of the design issues presented by machine learning, and are admirably suited to parallel and distributed computation. The Avatar Tools codes are an implementation of ensembles specifically for decision trees. Some features that distinguish Avatar Tools from other "ensembles for decision trees" codes are: (1) Does the bookkeeping necessary for out of bag (OOB) validation. (2) Can use OOB validation to automatically determine optimal ensemble size. (3) Provides an MPI-based parallel implementation, for distributed operation. (4) Provides convenient tools for cross-validation, to assess the accuracy provided by a training set. SAND2020-3858 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Siefert, Christopher↗

Deriving Optimal Operational Rules for Mitigating Inter-area Oscillations

This paper introduces a new method to mitigate inter-area oscillations of a large scale interconnected power system by means of generation re-dispatch. The optimal operational control procedures are derived as the shortest distance from the current operating condition to a desired damping ratio of the oscillation mode by adjusting generator outputs. A sensitivity based method is used to select the most effective generators for generation re-dispatch and decision tree is trained to approximate the security boundary in a space characterized by the selected generators. The optimal operational rules can be found by solving an optimization problem where the boundary constraints are provided by the decision tree rules. This method is tested on a Western Electricity Coordinating Council (WECC) 179-bus simplified network model and simulation results have demonstrated the proof of concept and shown promising application in real time operation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A comparison of machine learning methods to classify radioactive elements using prompt-gamma-ray neutron activation data

The detection of illicit radiological materials is critical to establishing a robust second line of defence in nuclear security. Neutron-capture prompt-gamma activation analysis (PGAA) can be used to detect multiple radioactive materials across the entire Periodic Table. However, long detection times and a high rate of false positives pose a significant hindrance in the deployment of PGAA-based systems to identify the presence of illicit substances in nuclear forensics. In the present work, six different machine-learning algorithms were developed to classify radioactive elements based on the PGAA energy spectra. The model performance was evaluated using standard classification metrics and trend curves with an emphasis on comparing the effectiveness of algorithms that are best suited for classifying imbalanced datasets. We analyse the classification performance based on Precision, Recall, F1-score, Specificity, Confusion matrix, ROC-AUC curves, and Geometric Mean Score (GMS) measures. The tree-based algorithms (Decision Trees, Random Forest and AdaBoost) have consistently outperformed Support Vector Machine and K-Nearest Neighbours. Based on the results presented, AdaBoost is the preferred classifier to analyse data containing PGAA spectral information due to the high recall and minimal false negatives reported in the minority class.

97 MATHEMATICS AND COMPUTING↗

Data mining of space heating system performance in affordable housing

The space heating in residential buildings accounts for a considerable amount of the primary energy use. Therefore, understanding the operation and performance of space heating systems becomes crucial in improving occupant comfort while reducing energy use. This study investigated the behavior of occupants adjusting their thermostat settings and heating system operations in a 62-unit affordable housing complex in Revere, Massachusetts, USA. The data mining methods, including clustering approach and decision trees, were used to ascertain occupant behavior patterns. Data tabulating ON/OFF space heating states was assessed, to provide a better understanding of the intermittent operation of space heating systems in terms of system cycling frequency and the duration of each operation. The decision tree was used to verify the link between room temperature settings, house and heating system characteristics and the heating energy use. The results suggest that the majority of apartments show fairly constant room temperature profiles with limited variations during a day or between weekday and weekend. Data clustering results revealed six typical patterns of room temperature profiles during the heating season. Space heating systems cycled more frequently than anticipated due to a tight range of room thermostat settings and potentially oversized heating capacities. In conclusion, from this study affirm data mining techniques are an effective method to analyze large datasets and extract hidden patterns to inform design and improve operations.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Entropy removal of medical diagnostics

Shannon entropy is a core concept in machine learning and information theory, particularly in decision tree modeling. To date, no studies have extensively and quantitatively applied Shannon entropy in a systematic way to quantify the entropy of clinical situations using diagnostic variables (true and false positives and negatives, respectively). Decision tree representations of medical decision-making tools can be generated using diagnostic variables found in literature and entropy removal can be calculated for these tools. This concept of clinical entropy removal has significant potential for further use to bring forth healthcare innovation, such as quantifying the impact of clinical guidelines and value of care and applications to Emergency Medicine scenarios where diagnostic accuracy in a limited time window is paramount. This analysis was done for 623 diagnostic tools and provided unique insights into their utility. For studies that provided detailed data on medical decision-making algorithms, bootstrapped datasets were generated from source data to perform comprehensive machine learning analysis on these algorithms and their constituent steps, which revealed a novel and thorough evaluation of medical diagnostic algorithms.

97 MATHEMATICS AND COMPUTING↗

The Characteristic Ellipsoid Methodology and Its Application in Power Systems

The characteristic ellipsoid (CELL) method to monitor dynamic behaviors of a power system is proposed. Multidimensional minimum-volume-enclosing characteristic ellipsoids are built using synchronized phasor measurements. System dynamic behaviors are identified by tracking the change rate of the CELL’s characteristic indices. Decision tree techniques are used to link the CELL’s characteristic indices and the system’s dynamic behaviors and to determine types, locations and related information about the dynamic behaviors. The knowledge base of representative transient events is created by offline simulations based on the full Western Electric Coordinating Council (WECC) model. Two case studies demonstrate that the CELL method combined with the decision trees can detect transient events and their features with good accuracy.

Characteristic ellipsoid, decision tree, event det↗

Photometric classification of type Ia supernovae in the SuperNova Legacy Survey with supervised learning

In the era of large astronomical surveys, photometric classification of supernovae (SNe) has become an important research field due to limited spectroscopic resources for candidate follow-up and classification. In this work, we present a method to photometrically classify type Ia supernovae based on machine learning with redshifts that are derived from the SN light-curves. This method is implemented on real data from the SNLS deferred pipeline, a purely photometric pipeline that identifies SNe Ia at high-redshifts (0.2 < z < 1.1). Our method consists of two stages: feature extraction (obtaining the SN redshift from photometry and estimating light-curve shape parameters) and machine learning classification. We study the performance of different algorithms such as Random Forest and Boosted Decision Trees. We evaluate the performance using SN simulations and real data from the first 3 years of the Supernova Legacy Survey (SNLS), which contains large spectroscopically and photometrically classified type Ia samples. Using the Area Under the Curve (AUC) metric, where perfect classification is given by 1, we find that our best-performing classifier (Extreme Gradient Boosting Decision Tree) has an AUC of 0.98.We show that it is possible to obtain a large photometrically selected type Ia SN sample with an estimated contamination of less than 5%. When applied to data from the first three years of SNLS, we obtain 529 events. We investigate the differences between classifying simulated SNe, and real SN survey data. In particular, we find that applying a thorough set of selection cuts to the SN sample is essential for good classification. This work demonstrates for the first time the feasibility of machine learning classification in a high- z SN survey with application to real SN data.

79 ASTRONOMY AND ASTROPHYSICS↗

Machine learning to predict final fire size at the time of ignition

Fires in boreal forests of Alaska are changing, threatening human health and ecosystems. Given expected increases in fire activity with climate warming, insight into the controls on fire size from the time of ignition is necessary. Such insight may be increasingly useful for fire management, especially in cases where many ignitions occur in a short time period. Here we investigated the controls and predictability of final fire size at the time of ignition. Using decision trees, we show that ignitions can be classified as leading to small, medium or large fires with 50.4 ± 5.2% accuracy. This was accomplished using two variables: vapour pressure deficit and the fraction of spruce cover near the ignition point. The model predicted that 40% of ignitions would lead to large fires, and those ultimately accounted for 75% of the total burned area. Other machine learning classification algorithms, including random forests and multi-layer perceptrons, were tested but did not outperform the simpler decision tree model. Applying the model to areas with intensive human management resulted in overprediction of large fires, as expected. This type of simple classification system could offer insight into optimal resource allocation, helping to maintain a historical fire regime and protect Alaskan ecosystems.

54 ENVIRONMENTAL SCIENCES↗

Diffusion and perfusion MR parameters to assess preoperative short-course radiotherapy response in locally advanced rectal cancer: a comparative explorative study among Standardized Index of Shape by DCE-MRI, intravoxel incoherent motion- and diffusion kurtosis imaging-derived parameters

Purpose: To assess preoperative short-course radiotherapy (SCR) tumor response in locally advanced rectal cancer (LARC) by means of Standardized Index of Shape (SIS) by dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), apparent diffusion coefficient (ADC), intravoxel incoherent motion (IVIM) and diffusion kurtosis imaging (DKI) parameters derived from diffusion-weighted MRI (DW-MRI). Materials and methods: Thirty-four patients with LARC who underwent MRI scans before and after SCR followed by delayed surgery, retrospectively, were enrolled. SIS, ADC, IVIM parameters [tissue diffusion (D{sub t}), pseudo-diffusion (D{sub p}), perfusion fraction (f{sub p})] and DKI parameters [mean diffusivity (MD), mean of diffusional kurtosis (MK)] were calculated for each patient. IVIM parameters were estimated using two methods, namely conventional biexponential fitting (CBFM) and variable projection (VARPRO). After surgery, the pathological TNM and tumor regression grade (TRG) were estimated. For each parameter, percentage changes between before and after SCR were evaluated. Furthermore, an artificial neural network was trained for outcome prediction. Nonparametric sample tests and receiver operating characteristic curve (ROC) analysis were performed. Results: Fifteen patients were classified as responders (TRG ≤ 2) and 19 as not responders (TRG > 3). Seven patients had TRG 1 (pathological complete response, pCR). Mean and standard deviation values of pre-treatment CBFM D{sub p} and mean value of VARPRO D{sub p} pre-treatment showed statistically significant differences to predict pCR. (p value at Mann–Whitney test was 0.05, 0.03 and 0.008, respectively.) Exclusively SIS percentage change showed significant differences between responder and non-responder patients after SCR (p value << 0.001) and to assess pCR after SCR (p value << 0.001). The best results to predict pCR were obtained by VARPRO Fp mean value pre-treatment with area under ROC of 0.84, a sensitivity of 96.4%, a specificity of 71.4%, a positive predictive value (PPV) of 92.9%, a negative predictive value (NPV) of 83.3% and an accuracy of 91.2%. The best results to assess after treatment complete pathological response were obtained by SIS with an area under ROC of 0.89, a sensitivity of 85.7%, a specificity of 92.6%, a PPV of 75.0%, a NPV of 96.1% and an accuracy of 91.2%. Moreover, the best results to differentiate after treatment responders vs. non-responders were obtained by SIS with an area under ROC of 0.94, a sensitivity of 93.3%, a specificity of 84.2%, a PPV of 82.4%, a NPV of 94.1% and an accuracy of 88.2%. Promising initial results were obtained using a decision tree tested with all ADC, IVIM and DKI extracted parameter: we reached high accuracy to assess pathological complete response after SCR in LARC (an accuracy of 85.3% to assess pathological complete response after SCR using VARPRO D{sub p} mean value post-treatment, ADC standard deviation value pre-treatment, MD standard deviation value post-treatment). Conclusion: SIS is a hopeful DCE-MRI angiogenic biomarker to assess preoperative treatment response after SCR with delayed surgery. Furthermore, an important prognostic role was obtained by VARPRO F{sub p} mean value pre-treatment and by a decision tree composed by diffusion parameters derived by DWI and DKI to assess pathological complete response.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Algorithm For A Self-Growing Neural Network

CID3 algorithm simulates self-growing neural network. Constructs decision trees equivalent to hidden layers of neural network. Based on ID3 algorithm, which dynamically generates decision tree while minimizing entropy of information. CID3 algorithm generates feedforward neural network by use of either crisp or fuzzy measure of entropy.

Cios, Krzysztof J.↗

Increasing the Scale of the Mass Spectrometry Query Language Compendium with Explainable AI

A significant bottleneck in metabolomics data interpretation is the effective use of domain knowledge to assign structural information based on fragmentation patterns. The mass spectrometry query language (MassQL) aims to make this process accessible and applicable across multiple analysis platforms. While advanced computational methods are capable of predicting compound structures from fragmentation data, AI/ML approaches often rely on complex, opaque criteria that are difficult to interpret or modify. As a result, their predictive patterns cannot be readily translated into human-readable rules, such as those used in MassQL. Here, in this study, we introduce ChemEcho, a machine learning embedding method that converts tandem mass spectrometry data into sparse feature vectors containing peak and neutral mass subformulae to enhance explainable AI/ML-based methods. An advantage of this approach is that decision trees trained using these feature vectors can be directly translated to MassQL. Using a battery of decision trees trained using ChemEcho embeddings to predict molecular attributes, we generated over 1500 MassQL queries for 765 molecular features and evaluated their precision and recall. From these queries, the 50 highest-performing queries were integrated into the MassQL compendium. This set of generated MassQL queries included environmentally and biologically relevant classes such as PFAS and molecules containing phosphate or sulfate substructures. To illustrate the impact these queries would have on a typical metabolomics experiment, these MassQL queries were applied to a public metabolomics data set─resulting in a marked increase in the structural information derived from tandem mass spectra. Access and reuse of these queries is expected to enhance structural annotation in untargeted experiments, leading to more specific claims and advancing many applications in metabolomics.

Harwood, Thomas V. [USDOE Joint Genome Institute (↗

An improved classification tree analysis of high cost modules based upon an axiomatic definition of complexity

Identification of high cost modules has been viewed as one mechanism to improve overall system reliability, since such modules tend to produce more than their share of problems. A decision tree model was used to identify such modules. In this current paper, a previously developed axiomatic model of program complexity is merged with the previously developed decision tree process for an improvement in the ability to identify such modules. This improvement was tested using data from the NASA Software Engineering Laboratory.

Tian, Jianhui↗

QuantifyML: How good is my machine learning model?

This paper presents an approach, QuantifyML, which employs model counting to assess the learnability and robustness of machine learning models. Typically the efficacy of machine learning models is determined by computing their accuracy statistically on test data sets. However, this may be misleading, if the test data is not representative of the problem that is being studied. Further, two different models may have the same accuracy on a given data set, measured statistically, but may be very different in their behavior on unseen data. Also, models with high accuracy could have poor adversarial robustness. In QuantifyML, our goal is to precisely quantify the extent to which machine learning models have learned and generalized from the given data. In QuantifyML, a trained model is translated into a C program, which is fed to the CBMC model checking tool to produce a formula in Conjunctive Normal Form (CNF), which in turn is analyzed with state-of-the-art model counters to efficiently obtain precise counts w.r.t different outputs. QuantifyML enables i) evaluating the learnability of models by comparing the counts for the outputs to ground truth, expressed as logical predicates (if available), ii) comparing the performance of different models that may be built with different machine learning algorithms (e.g., decision-trees vs. neural networks), and iii) quantifying the robustness of trained models around given inputs. Our evaluation demonstrates these applications of QuantifyML on decision trees and neural networks trained to learn relational properties of graphs, for which we know the ground truth, and to perform image classification, for which we do not have the ground truth, but we can quantify local robustness.

Deep Neural Networks↗

Prognostic analysis of high-flow nasal cannula therapy and non-invasive ventilation in mild to moderate hypoxemia patients and construction of a machine learning model for 48-h intubation prediction—a retrospective analysis of the MIMIC database

Background This study aims to investigate the clinical outcome between high-flow nasal cannula (HFNC) and non-invasive ventilation (NIV) therapy in mild to moderate hypoxemic patients on the first ICU day and to develop a predictive model of 48-h intubation. Methods The study included adult patients from the MIMIC III and IV databases who first initiated HFNC or NIV therapy due to mild to moderate hypoxemia (100 < PaO2/FiO2 ≤ 300). The 48-h and 30-day intubation rates were compared using cross-sectional and survival analysis. Nine machine learning and six ensemble algorithms were deployed to construct the 48-h intubation predictive models, of which the optimal model was determined by its prediction accuracy. The top 10 risk and protective factors were identified using the Shapley interpretation algorithm. Result A total of 123,042 patients were screened, of which, 673 were from the MIMIC IV database for ventilation therapy comparison (HFNC n = 363, NIV n = 310) and 48-h intubation predictive model construction (training dataset n = 471, internal validation set n = 202) and 408 were from the MIMIC III database for external validation. The NIV group had a lower intubation rate (23.1% vs. 16.1%, p = 0.001), ICU 28-day mortality (18.5% vs. 11.6%, p = 0.014), and in-hospital mortality (19.6% vs. 11.9%, p = 0.007) compared to the HFNC group. Survival analysis showed that the total and 48-h intubation rates were not significantly different. The ensemble AdaBoost decision tree model (internal and external validation set AUROC 0.878, 0.726) had the best predictive accuracy performance. The model Shapley algorithm showed Sequential Organ Failure Assessment (SOFA), acute physiology scores (APSIII), the minimum and maximum lactate value as risk factors for early failure and age, the maximum PaCO 2 and PH value, Glasgow Coma Scale (GCS), the minimum PaO 2 /FiO 2 ratio, and PaO 2 value as protective factors. Conclusion NIV was associated with lower intubation rate and ICU 28-day and in-hospital mortality. Further survival analysis reinforced that the effect of NIV on the intubation rate might partly be attributed to the other impact factors. The ensemble AdaBoost decision tree model may assist clinicians in making clinical decisions, and early organ function support to improve patients’ SOFA, APSIII, GCS, PaCO 2 , PaO 2 , PH, PaO 2 /FiO 2 ratio, and lactate values can reduce the early failure rate and improve patient prognosis.

Fu, Wei↗

Prediction of Weather Impacted Airport Capacity using Ensemble Learning

Ensemble learning with the Bagging Decision Tree (BDT) model was used to assess the impact of weather on airport capacities at selected high-demand airports in the United States. The ensemble bagging decision tree models were developed and validated using the Federal Aviation Administration (FAA) Aviation System Performance Metrics (ASPM) data and weather forecast at these airports. The study examines the performance of BDT, along with traditional single Support Vector Machines (SVM), for airport runway configuration selection and airport arrival rates (AAR) prediction during weather impacts. Testing of these models was accomplished using observed weather, weather forecast, and airport operation information at the chosen airports. The experimental results show that ensemble methods are more accurate than a single SVM classifier. The airport capacity ensemble method presented here can be used as a decision support model that supports air traffic flow management to meet the weather impacted airport capacity in order to reduce costs and increase safety.

Weather impact↗