Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supervised machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Enhancing transfer learning in angle-resolved photoemission spectroscopy (ARPES) with spatially-aware representations via graph convolution

A recent application of machine learning has been to spatially-resolved angle-resolved photoemission spectroscopy (ARPES). Here we advance the state-of-the-art by applying representational learning to transform ARPES data into an embedding space of a pre-trained self-supervised learning model, thus enhancing the pipeline that improves the bandstructure classification and domain assignment/segmentation performance compared to a k-means clustering method. In the current iteration, the real-space information is entered into the domain assignment through the graph convolution method, which improves the transfer learning performance of the original self-supervised model. Lastly, an unsupervised automated tool is developed that incorporates these techniques to enable automatic domain assignment.

ARPES↗

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While traditional ensemble classifiers yield improved task performance, such methods come with a steep cost in model complexity. Through knowledge distillation, it is possible to retain some performance improvements from an ensemble, while minimizing costs to model complexity. In this work, we implement a weakly supervised ensemble using clustering-constrained-attention multiple-instance learners (CLAM), which uses attention and instance-level clustering to identify task salient regions and feature extraction in whole slides. By applying logit-based and attention-based knowledge distillation, we show it is possible to retain some performance improvements resulting from the ensemble at zero cost to model complexity.

Alamudun, Folami↗

Fast Evaluation of Aircraft Icing Severity Using Machine Learning Based on XGBoost

Aircraft icing represents a serious hazard in aviation which has caused a number of fatal accidents over the years. In addition, it can lead to substantial increase in drag and weight, thus reducing the aerodynamics performance of the airplane. The process of ice accretion on a solid surface is a complex interaction of aerodynamic and environmental variables. The complex relationship makes machine learning-based methods an attractive alternative to traditional numerical simulation-based approaches. In this study, we introduce a purely data-driven approach to find the complex pattern between different flight conditions and aircraft icing severity prediction. The supervised learning algorithm Extreme Gradient Boosting (XGBoost) is applied to establish the prediction framework which makes prediction based on any set of observations. The input flight conditions for the proposed prediction framework are liquid water content, droplet diameter and exposure time. The proposed approach is demonstrated in three cases: maximum ice thickness prediction, icing area prediction and icing severity level evaluation. Performance comparison studies and error analysis are also conducted to verify the effectiveness and performance of the proposed method. Results show that the proposed method has reasonable capability in evaluating aircraft icing severity.

Li, Sibo (ORCID:000000021705844X)↗

Combustion machine learning: Principles, progress and prospects

Progress in combustion science and engineering has led to the generation of large amounts of data from large-scale simulations, high-resolution experiments, and sensors. This corpus of data offers enormous opportunities for extracting new knowledge and insights—if harnessed effectively. Machine learning (ML) techniques have demonstrated remarkable success in data analytics, thus offering a new paradigm for data-intense analyses and scientific investigations through combustion machine learning (CombML). While data-driven methods are utilized in various combustion areas, recent advances in algorithmic developments, the accessibility of open-source software libraries, the availability of computational resources, and the abundance of data have together rendered ML techniques ubiquitous in scientific analysis and engineering. This article examines ML techniques for applications in combustion science and engineering. Starting with a review of sources of data, data-driven techniques, and concepts, we examine supervised, unsupervised, and semi-supervised ML methods. Various combustion examples are considered to illustrate and to evaluate these methods. Next, we review past and recent applications of ML approaches to problems in combustion, spanning fundamental combustion investigations, propulsion and energy-conversion systems, and fire and explosion hazards. Challenges unique to CombML are discussed and further opportunities are identified, focusing on interpretability, uncertainty quantification, robustness, consistency, creation and curation of benchmark data, and the augmentation of ML methods with prior combustion-domain knowledge.

33 ADVANCED PROPULSION SYSTEMS↗

Machine learning the dynamics of quantum kicked rotor

Highlights: • Quantum kicked rotor. • Quantum phase transition. • Machine learning. • LSTM network. • Convolutional neural network. Using the multilayer convolutional neural network (CNN), we can detect the quantum phases in random electron systems, and phase diagrams of two and higher dimensional Anderson transitions and quantum percolations as well as disordered topological systems have been obtained. Here, instead of using CNN to analyze the wave functions, we analyze the dynamics of wave packets via long short-term memory network (LSTM). We adopt the quasi-periodic quantum kicked rotors, which simulate the three and four dimensional Anderson transitions. By supervised training, we let LSTM extract the features of the time series of wave packet displacements in localized and delocalized phases. We then simulate the wave packets in unknown phases and let LSTM classify the time series to localized and delocalized phases. We compare the phase diagrams obtained by LSTM and those obtained by CNN.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Predicting non-linear stress–strain response of mesostructured cellular materials using supervised autoencoder

Recent breakthroughs in advanced manufacturing capabilities have made it possible to design and print sophisticated topologies of cellular structures using diverse engineering materials such as metals, polymers, and ceramics. In these architectured materials, it is often desirable to tailor the mechanical properties by altering the unit cell topology. This necessitates an in-depth understanding of how the topology of the unit cell structure affects the macroscopic behavior of the material in both the linear and the non-linear regimes encountered under large compression. Here, we have developed a machine learning (ML) approach capable of accelerating the prediction of the stress–strain response of a polymer-based cellular structure under uniaxial confined compression. As part of generating the training data for ML, 60,000 mesostructures were generated using a relatively novel approach based on cellular automata, and their corresponding stress–strain responses were obtained from the finite element simulations. Principal component analysis (PCA) was used to reduce the dimensionality of the stress–strain curves. With only 20 principal components, PCA captured 99.89% of the variance in the stress–strain curves while reducing the dimensionality by 5X. ML using supervised autoencoder was able to successfully speed up the prediction of the non-linear stress–strain response of a unit cell by up to 4600X. The proposed method can serve as an efficient data generation tool and a rapid means for predicting the structure–property relationship through accelerated forward modeling of cellular materials under compaction, in cases where the macroscopic stress–strain response is governed by the unit-cell topology.

36 MATERIALS SCIENCE↗

Supervised learning with word embeddings derived from PubMed captures latent knowledge about protein kinases and cancer

Abstract Inhibiting protein kinases (PKs) that cause cancers has been an important topic in cancer therapy for years. So far, almost 8% of >530 PKs have been targeted by FDA-approved medications, and around 150 protein kinase inhibitors (PKIs) have been tested in clinical trials. We present an approach based on natural language processing and machine learning to investigate the relations between PKs and cancers, predicting PKs whose inhibition would be efficacious to treat a certain cancer. Our approach represents PKs and cancers as semantically meaningful 100-dimensional vectors based on word and concept neighborhoods in PubMed abstracts. We use information about phase I-IV trials in ClinicalTrials.gov to construct a training set for random forest classification. Our results with historical data show that associations between PKs and specific cancers can be predicted years in advance with good accuracy. Our tool can be used to predict the relevance of inhibiting PKs for specific cancers and to support the design of well-focused clinical trials to discover novel PKIs for cancer therapy.

59 BASIC BIOLOGICAL SCIENCES↗

The Pixel Anomaly Detection Tool : a user-friendly GUI for classifying detector frames using machine-learning approaches

Data collection at X-ray free electron lasers has particular experimental challenges, such as continuous sample delivery or the use of novel ultrafast high-dynamic-range gain-switching X-ray detectors. This can result in a multitude of data artefacts, which can be detrimental to accurately determining structure-factor amplitudes for serial crystallography or single-particle imaging experiments. Here, a new data-classification tool is reported that offers a variety of machine-learning algorithms to sort data trained either on manual data sorting by the user or by profile fitting the intensity distribution on the detector based on the experiment. This is integrated into an easy-to-use graphical user interface, specifically designed to support the detectors, file formats and software available at most X-ray free electron laser facilities. The highly modular design makes the tool easily expandable to comply with other X-ray sources and detectors, and the supervised learning approach enables even the novice user to sort data containing unwanted artefacts or perform routine data-analysis tasks such as hit finding during an experiment, without needing to write code.

47 OTHER INSTRUMENTATION↗

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗

High-dimensional anomaly detection with radiative return in $\mathcal{e}$⁺$\mathcal{e}$⁻ collisions

Experiments at a future $\mathcal{e}$⁺$\mathcal{e}$⁻ collider will be able to search for new particles with masses below the nominal centre-of-mass energy by analyzing collisions with initial-state radiation (radiative return). We show that machine learning methods that use imperfect or missing training labels can achieve sensitivity to generic new particle production in radiative return events. In addition to presenting an application of the classification without labels (CWoLa) search method in $\mathcal{e}$⁺$\mathcal{e}$⁻ collisions, our study combines weak supervision with variable-dimensional information by deploying a deep sets neural network architecture. We have also investigated some of the experimental aspects of anomaly detection in radiative return events and discuss these in the context of future detector design.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

Connectivity‐based parcellation of normal and anatomically distorted human cerebral cortex

Abstract For over a century, neuroscientists have been working toward parcellating the human cortex into distinct neurobiological regions. Modern technologies offer many parcellation methods for healthy cortices acquired through magnetic resonance imaging. However, these methods are suboptimal for personalized neurosurgical application given that pathology and resection distort the cerebrum. We sought to overcome this problem by developing a novel connectivity‐based parcellation approach that can be applied at the single‐subject level. Utilizing normative diffusion data, we first developed a machine‐learning (ML) classifier to learn the typical structural connectivity patterns of healthy subjects. Specifically, the Glasser HCP atlas was utilized as a prior to calculate the streamline connectivity between each voxel and each parcel of the atlas. Using the resultant feature vector, we determined the parcel identity of each voxel in neurosurgical patients ( n = 40) and thereby iteratively adjusted the prior. This approach enabled us to create patient‐specific maps independent of brain shape and pathological distortion. The supervised ML classifier re‐parcellated an average of 2.65% of cortical voxels across a healthy dataset ( n = 178) and an average of 5.5% in neurosurgical patients. Our patient dataset consisted of subjects with supratentorial infiltrating gliomas operated on by the senior author who then assessed the validity and practical utility of the re‐parcellated diffusion data. We demonstrate a rapid and effective ML parcellation approach to parcellation of the human cortex during anatomical distortion. Our approach overcomes limitations of indiscriminately applying atlas‐based registration from healthy subjects by employing a voxel‐wise connectivity approach based on individual data.

59 BASIC BIOLOGICAL SCIENCES↗

Deep learning with mixup augmentation for improved pore detection during additive manufacturing

In additive manufacturing (AM), process defects such as keyhole pores are difficult to anticipate, affecting the quality and integrity of the AM-produced materials. Hence, considerable efforts have aimed to predict these process defects by training machine learning (ML) models using passive measurements such as acoustic emissions. This work considered a dataset in which keyhole pores of a laser powder bed fusion (LPBF) experiment were identified using X-ray radiography and then registered both in space and time to acoustic measurements recorded during the LPBF experiment. Due to AM’s intrinsic process controls, where a pore-forming event is relatively rare, the acoustic datasets collected during monitoring include more non-pores than pores. In other words, the dataset for ML model development is imbalanced. Moreover, this imbalanced and sparse data phenomenon remains ubiquitous across many AM monitoring schemes since training data is nontrivial to collect. Hence, we propose a machine learning approach to improve this dataset imbalance and enhance the prediction accuracy of pore-labeled data. Specifically, we investigate how data augmentation helps predict pores and non-pores better. This imbalance is improved using recent advances in data augmentation called Mixup, a weak-supervised learning method. Convolutional neural networks (CNNs) are trained on original and augmented datasets, and an appreciable increase in performance is reported when testing on five different experimental trials. When ML models are trained on original and augmented datasets, they achieve an accuracy of 95% and 99% on test datasets, respectively. We also provide information on how dataset size affects model performance. Lastly, we investigate the optimal Mixup parameters for augmentation in the context of CNN performance.

36 MATERIALS SCIENCE↗

Deep learning with mixup augmentation for improved pore detection during additive manufacturing

In additive manufacturing (AM), process defects such as keyhole pores are difficult to anticipate, affecting the quality and integrity of the AM-produced materials. Hence, considerable efforts have aimed to predict these process defects by training machine learning (ML) models using passive measurements such as acoustic emissions. This work considered a dataset in which keyhole pores of a laser powder bed fusion (LPBF) experiment were identified using X-ray radiography and then registered both in space and time to acoustic measurements recorded during the LPBF experiment. Due to AM’s intrinsic process controls, where a pore-forming event is relatively rare, the acoustic datasets collected during monitoring include more non-pores than pores. In other words, the dataset for ML model development is imbalanced. Moreover, this imbalanced and sparse data phenomenon remains ubiquitous across many AM monitoring schemes since training data is nontrivial to collect. Hence, we propose a machine learning approach to improve this dataset imbalance and enhance the prediction accuracy of pore-labeled data. Specifically, we investigate how data augmentation helps predict pores and non-pores better. This imbalance is improved using recent advances in data augmentation called Mixup, a weak-supervised learning method. Convolutional neural networks (CNNs) are trained on original and augmented datasets, and an appreciable increase in performance is reported when testing on five different experimental trials. When ML models are trained on original and augmented datasets, they achieve an accuracy of 95% and 99% on test datasets, respectively. We also provide information on how dataset size affects model performance. Lastly, we investigate the optimal Mixup parameters for augmentation in the context of CNN performance.

42 ENGINEERING↗

Spread Spectrum Time Domain Reflectometry (SSTDR) and Frequency Domain Reflectometry (FDR) for Detection of Cable Anomalies Using Machine Learning

Cables are initially qualified for nuclear power plant use for 40 years. As plants extend their operating license to 60 and 80 years, continued use of these cables must shift to a performance-based approach since it is cost prohibitive to completely replace cables that are likely still capable of performing their design function. A variety of cable tests are available and are commonly applied during outages when the cables can be taken out of service. Frequency domain reflectometry (FDR) is one of these test methods that is being more broadly accepted and used because it not only detects anomalies along the cable with a low-voltage signal that does not stress the cable insulation, but the technique also locates the anomalies. This supports follow-up local inspection and local repair or partial replacement of a damaged cable segment. Currently, FDR testing is only applied to cables that are taken out of service since the test instrument would be damaged by operational voltages. A related technology that has found some acceptance in the aircraft and rail industry is spread spectrum time domain reflectometry (SSTDR). This technology has been implemented with a custom commercial instrument by LiveWire Innovation Inc. that is designed to operate on live cables up to 1000 volts. One of the main conclusions of a previous effort was that cable reflectometry plots can be difficult for humans to analyze due to baseline noise, low or noisy anomaly response peaks, or large responses from cable ends. Detection of cable anomalies for many of these frequencies and test conditions was challenging for manual analysis. This presented an ideal opportunity for ML analysis to distinguish undamaged cable indications from anomalous cable indications. This research discusses application of machine learning (ML) to reflectometry cable test methods. The goal was to assess feasibility to distinguish undamaged cable reflectometry responses from damaged or anomalous cable reflectometry responses. The assessment considered the 3 instruments, multiple frequency bandwidths from each instrument, multiple cable anomalies and test conditions, and both supervised and unsupervised ML approaches. Although approaches and analysis methods were not identical or directly comparable, both outputs were encouraging. The unsupervised prediction weighted accuracy was assessed by instrument and by frequency. It performed better at high frequencies with the highest prediction accuracy of 0.84 for the higher frequency FDR, 0.79 for the 48-MHz LiveWire SSTDR, and 0.77 for 300-MHz PNNL SSTDR. The initial weighted accuracy average across all frequencies for using supervised ML was 0.56 to 0.68. The supervised analysis was repeated with noisier training data removed resulting in weighted accuracies of 0.69 to 0.87. These weighted accuracies are not directly comparable due to differences in the supervised and unsupervised analysis details but do indicate an encouraging trend. Even with limited and unbalanced data, strong prediction accuracies seem encouraging for further work including more data under a wider range of conditions.

42 ENGINEERING↗

Machine Learning-Driven Reliability Estimation of PV Inverters Considering Alert-Ambient Variability

Weather-induced spatio-temporal degradation limits outdoor PV inverter lifetime and reliability, necessitating advanced data analysis. This study employs a top-down, data-driven approach utilizing multiple machine learning (ML) algorithms to estimate inverter reliability in a 1.4 MW PV power plant, considering factors such as irradiance, humidity, temperature, time of day, and weather conditions. An extensive alert dataset from 17 identical inverters, including alert types, propagation, and frequency, reveals significant correlations with environmental factors and inverter output power, enabling the construction of a performance reliability model. Dual-stage supervised-ML models are evaluated for accuracy, with the ‘classification-regression’ model by an artificial neural network (ANN) tested on the averaged “Alert-Ambient” dataset, which is outperformed by ‘clustering-regression’ models using random forest (RF) and K-Nearest Neighbors (KNN) on individual inverter datasets. K-means clustering applies principal component analysis to reduce dimensions, achieving improved accuracy beyond the 80% achieved by ANN on the averaged dataset. Second-stage regression estimates inverter reliability with a mean square error of 0.0195 on the averaged dataset and as low as 0.002 on individual inverter datasets using RF. Furthermore, these findings highlight the method's suitability for estimating PV inverter output reliability under ambient conditions, essential for digital twin development and related applications.

14 SOLAR ENERGY↗

Development of Automated Pipeline for Time-Resolved Link-Wise Vehicular Energy Consumption in the Chattanooga, TN Road Network

The Department of Energy (DOE) has shown strong interest in detecting energy inefficiencies in regional road networks, so as to derive energy consumed at a high spatial temporal resolution. We have developed a workflow to automate the estimation of time-resolved vehicular energy consumption over each link in a road network of interest. The road network used in the current work is centered around the city of Chattanooga, Tennessee and its bordering regions. Utilizing the most mature road network for the Chattanooga, TN region, vehicle speed & count data from TomTom in conjunction with machine learning methods, we have developed an automated pipeline to estimate energy consumption for every link in the network. The first step in the pipeline is ingesting vehicle probe counts and speed estimates from TomTom API. In the next step, the probe counts, speed profiles and other exogenous data (i.e. road types, weather data, ground-truth volume counts and more) were used as input to a supervised learning algorithm to estimate the number of vehicles throughout the entire region for each road segment. These volume estimates were then mapped to a unified road network that contained additional important information such as percentage change in gradient across a link, number of lanes and link lengths that are features in pre-trained single vehicle energy-consumption models available with the RouteE software developed at NREL. The per vehicle energy consumption on each road link predicted using appropriate RouteE vehicular models were multiplied by the volume estimate for the corresponding link over a given time period to predict energy consumed per link for the time interval of interest. Currently, work is underway to improve both the RouteE per vehicle energy estimate and the volume estimates derived from TomTom probe counts. We have also explored the correlation of the link-wise energy estimates with the features of the pre-trained RouteE machine learning model in order to gain insight into what factors contribute most to the link-wise energy consumption.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION,↗

Semi-Supervised Disaggregation of Load Profiles at Transmission Buses with Significant Behind-the-Meter Solar Generations

It is of imperative interests for regional transmission organizations (RTOs) to effectively extract daily load profiles at transmission buses, which remains a gap in existing technology paradigm. This digest proposes an explicit yet efficient linear estimator, to disaggregate metered load profiles at buses with significant behind-the-meter (BTM) solar generations in a data driven manner. The proposed estimator is based on utility zonal load profiles and proxy solar irradiance profiles, which in reality is the aggregated waveform at each transmission bus and equivalent to the mix of summed load profiles minus actual BTM solar generation. To overcome technical challenges in the lack of “ground truth” and validate the performance of supervised learning algorithms, we propose semi-supervised mechanisms with parameter tuning, and leverage the unique characteristics of zero-crossing points in BTM solar peaking behaviors.

machine leaning, behind-the-meter, power grid anal↗