Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supervised learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling

We introduce a novel semi-supervised learning approach, named Teacher-Student Bayesian Optimization (TSBO ), integrating the teacher-student paradigm into BO to minimize expensive labeled data queries for the first time. TSBO incorporates a teacher model, an unlabeled data sampler, and a student model. The student is trained on unlabeled data locations generated by the sampler, with pseudo labels predicted by the teacher. The interplay between these three components implements a unique selective regularization to the teacher in the form of student feedback. This scheme enables the teacher to predict high-quality pseudo labels, enhancing the generalization of the GP surrogate model in the search space. To fully exploit TSBO , we propose two optimized unlabeled data samplers to construct effective student feedback that well aligns with the objective of Bayesian optimization. Furthermore, we quantify and leverage the uncertainty of the teacher-student model for the provision of reliable feedback to the teacher in the presence of risky pseudo-label predictions. TSBO demonstrates significantly improved sample-efficiency in several global optimization tasks under tight labeled data budgets. The implementation is available at https://github.com/reminiscenty/TSBO-Official.

Yin, Yuxuan↗

Reverse-mode differentiation in arbitrary tensor network format: with application to supervised learning.

This paper describes an efficient reverse-mode differentiation algorithm for contraction operations of tensor networks that may have arbitrary and unconventional network topologies. The approach leverages the tensor contraction tree of Evenbly and Pfeifer (2014), which provides an instruction set for the contraction sequence of a network. We show that this tree can be efficiently leveraged for differentiation of a full tensor network contraction using a recursive scheme that exploits (1) the bilinear property of contraction and (2) the property that trees have single path from root to leaves. While differentiation of tensor-tensor contraction is already possible in most automatic differentiation packages, we show that exploiting these two additional properties in the specific context of contraction sequences can improve efficiency. Following a description of the algorithm and computational complexity analysis, we investigate its utility for gradient-based supervised learning for low-rank function recovery and for fitting real-world unstructured datasets. We demonstrate improved performance over alternating least-squares optimization approaches and the capability to handle heterogeneous and arbitrary tensor network formats. When compared to alternating minimization algorithms, we find that the gradient-based approach requires a smaller oversampling ratio (number of samples compared to number model parameters) for recovery. This increased efficiency extends to fitting unstructured data of varying dimensionality and when employing a variety of tensor network formats. Here, we show improved learning using the hierarchical Tucker method over the tensor-train in high-dimensional settings on a number of benchmark problems.

97 MATHEMATICS AND COMPUTING↗

Prototype Vector Machine for Large Scale Semi-Supervised Learning

Practicaldataminingrarelyfalls exactlyinto the supervisedlearning scenario. Rather, the growing amount of unlabeled data poses a big challenge to large-scale semi-supervised learning (SSL). We note that the computationalintensivenessofgraph-based SSLarises largely from the manifold or graph regularization, which in turn lead to large models that are dificult to handle. To alleviate this, we proposed the prototype vector machine (PVM), a highlyscalable,graph-based algorithm for large-scale SSL. Our key innovation is the use of"prototypes vectors" for effcient approximation on both the graph-based regularizer and model representation. The choice of prototypes are grounded upon two important criteria: they not only perform effective low-rank approximation of the kernel matrix, but also span a model suffering the minimum information loss compared with the complete model. We demonstrate encouraging performance and appealing scaling properties of the PVM on a number of machine learning benchmark data sets.

97 MATHEMATICS AND COMPUTING↗

Beyond optimization—supervised learning applications in relativistic laser-plasma experiments

We explore the applications of machine learning techniques in relativistic laser-plasma experiments beyond optimization purposes. We predict the beam charge of electrons produced in a laser wakefield accelerator given the laser wavefront change caused by a deformable mirror. Machine learning enables feature analysis beyond merely searching for an optimal beam charge, showing that specific aberrations in the laser wavefront are favored in generating higher beam charges. Supervised learning models allow characterizing the measured data quality as well as recognizing irreproducible data and potential outliers. Furthermore, we also include virtual measurement errors in the experimental data to examine the model robustness under these conditions. This work demonstrates how machine learning methods can benefit data analysis and physics interpretation in a highly nonlinear problem of relativistic laser-plasma interaction.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Dense autoencoders, clustering techniques, and semi-supervised learning for HPGe $γ$-spectra

Classifying high-resolution gamma spectra by their isotopic content is an essential task in nuclear forensics and other applications. Traditional analysis methods are often time-intensive, but machine learning (ML) may help analysts quickly process many spectra. Such methods tend to rely on abundant, well-labeled data for training. Historical gamma data exists in various fields but is not uniformly useful for supervised ML due to inconsistent labeling. Here, to address some of these challenges, we present a method to classify and organize unlabeled data from high-purity germanium detectors using an autoencoding neural network (autoencoder). We trained dense autoencoders to compress gamma data into latent representations that enable efficient data characterization. By clustering the encoded spectra or lower-dimensional mappings of them, we identified and removed portions of over-abundant data categories, resulting in a more balanced dataset and improved autoencoder performance. This encoding and clustering pipeline also enabled the organization of spectra into self-consistent categories. Finally, we found that encoded representations showed potential as inputs for semi-supervised learning of nuclide identification (NID) labels, achieving an average F1 score of 0.85 ± 0.03 when mapping encodings to a set of 65 isotope labels.

Autoencoders↗

A Semi-Supervised Learning Method for the Identification of Bad Exposures in Large Imaging Surveys

As the data volume of astronomical imaging surveys rapidly increases, traditional methods for image anomaly detection, such as visual inspection by human experts, are becoming impractical. We introduce a machine-learning-based approach to detect poor-quality exposures in large imaging surveys, with a focus on the DECam Legacy Survey (DECaLS) in regions of low extinction (i.e., E ( B − V ) < 0.04 ). Our semi-supervised pipeline integrates a vision transformer (ViT), trained via self-supervised learning (SSL), with a k-Nearest Neighbor (kNN) classifier. We train and validate our pipeline using a small set of labeled exposures observed by surveys with the Dark Energy Camera (DECam). A clustering-space analysis of where our pipeline places images labeled in good and bad categories suggests that our approach can efficiently and accurately determine the quality of exposures. Applied to new imaging being reduced for DECaLS Data Release 11, our pipeline identifies 780 problematic exposures, which we subsequently verify through visual inspection. Being highly efficient and adaptable, our method offers a scalable solution for quality control in other large imaging surveys.

Luo, Yufeng (ORCID:0000000246230683)↗

Advanced Semi-Supervised Learning with Uncertainty Estimation for Phase Identification in Distribution Systems

The integration of advanced metering infrastructure (AMI) into power distribution networks generates valuable data for tasks such as phase identification; however, the limited and unreliable availability of labeled data in the form of customer phase connectivity presents challenges. To address this issue, we propose a semi-supervised learning (SSL) framework that effectively leverages labeled and unlabeled data. Our approach incorporates self-training, label spreading, and Bayesian neural networks (BNNs) to enhance phase identification with AMI data. Our method uses an ensemble of multilayer perceptron classifiers in a self-training setup, iteratively adding high-confidence pseudo-labels to improve robustness. We also apply label spread to propagate labels based on data similarity, which enhances generalization across diverse distributions. In addition, we employ a BNNs with uncertainty estimation, boosting confidence in predictions and reducing phase identification errors. In our case study, we achieved approximately 98% +/- 0.08 accuracy with uncertainty using minimal and unreliable labeled data from a real U.S. utility, Duquesne Light Company. Our SSL approach, combined with uncertainty estimation, provides an efficient solution for phase identification in AMI data, ultimately improving the reliability of smart grid applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Advanced Semi-Supervised Learning With Uncertainty Estimation for Phase Identification in Distribution Systems

The integration of advanced metering infrastructure (AMI) into power distribution networks generates valuable data for tasks such as phase identification; however, the limited and unreliable availability of labeled data in the form of customer phase connectivity presents challenges. To address this issue, we propose a semi-supervised learning (SSL) framework that effectively leverages labeled and unlabeled data. Our approach incorporates self-training, label spreading, and Bayesian neural networks (BNNs) to enhance phase identification with AMI data. Our method uses an ensemble of multilayer perceptron classifiers in a self-training setup, iteratively adding high-confidence pseudo-labels to improve robustness. We also apply label spread to propagate labels based on data similarity, which enhances generalization across diverse distributions. In addition, we employ a BNNs with uncertainty estimation, boosting confidence in predictions and reducing phase identification errors. In our case study, we achieved approximately 98% +/- 0.08 accuracy with uncertainty using minimal and unreliable labeled data from a real U.S. utility, Duquesne Light Company. Our SSL approach, combined with uncertainty estimation, provides an efficient solution for phase identification in AMI data, ultimately improving the reliability of smart grid applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Supervised Learning Applied to Air Traffic Trajectory Classification

Given the recent increase of interest in introducing new vehicle types and missions into the National Airspace System, a transition towards a more autonomous air traffic control system is required in order to enable and handle increased density and complexity. This paper presents an exploratory effort of the needed autonomous capabilities by exploring supervised learning techniques in the context of aircraft trajectories. In particular, it focuses on the application of machine learning algorithms and neural network models to a runway recognition trajectory-classification study. It investigates the applicability and effectiveness of various classifiers using datasets containing trajectory records for a month of air traffic. A feature importance and sensitivity analysis are conducted to challenge the chosen time-based datasets and the ten selected features. The study demonstrates that classification accuracy levels of 90% and above can be reached in less than 40 seconds of training for most machine learning classifiers when one track data point, described by the ten selected features at a particular time step, per trajectory is used as input. It also shows that neural network models can achieve similar accuracy levels but at higher training time costs.

Bosson, Christabelle S.↗

Supervised Learning Applied to Air Traffic Trajectory Classification

Given the recent increase of interest in introducing new vehicle types and missions into the National Airspace System, a transition towards a more autonomous air traffic control system is required in order to enable and handle increased density and complexity. This paper presents an exploratory effort of the needed autonomous capabilities by exploring supervised learning techniques in the context of aircraft trajectories. In particular, it focuses on the application of machine learning algorithms and neural network models to a runway recognition trajectory-classification study. It investigates the applicability and effectiveness of various classifiers using datasets containing trajectory records for a month of air traffic. A feature importance and sensitivity analysis are conducted to challenge the chosen time-based datasets and the ten selected features. The study demonstrates that classification accuracy levels of 90% and above can be reached in less than 40 seconds of training for most machine learning classifiers when one track data point, described by the ten selected features at a particular time step, per trajectory is used as input. It also shows that neural network models can achieve similar accuracy levels but at higher training time costs.

Bosson, Christabelle↗

Virtual sensor supervised learning for cyber-attack neutralization

An industrial asset may have monitoring nodes that generate current monitoring node values. A dynamic, resilient estimator may split a temporal monitoring node space into normal and one or more abnormal subspaces associated with different kinds of attack vectors. According to some embodiments, a neutralization model is constructed and trained for each attack vector using supervised learning and the associated abnormal subspace. In other embodiments, a single model is created using out-of-range values for abnormal monitoring nodes. Responsive to an indication of a particular abnormal monitoring node or nodes, the system may automatically invoke the appropriate neutralization model to determine estimated values of the particular abnormal monitoring node or nodes (e.g., by selecting the correct model or using out-of-range values). The series of current monitoring node values from the abnormal monitoring node or nodes may then be replaced with the estimated values.

97 MATHEMATICS AND COMPUTING↗

Monitoring the propagation of mechanical discontinuity using data-driven causal discovery and supervised learning

Mechanical wave transmission through a material is influenced by the mechanical discontinuity in the material. The propagation of embedded discontinuities can be monitored by analyzing the wave-transmission measurements recorded by a multipoint sensor system placed on the surface of the material. The proposed workflow monitors the propagation of mechanical discontinuity through three stages, namely initial, intermediate, and final stages, by using supervised learning followed by data-driven causal discovery. To the end, the workflow processes the multipoint waveform measurements resulting from a single impulse source, while considering the effects of wave attenuation, dispersion and multiple wave-propagation modes due to the discontinuity and material boundaries. Among various feature reduction techniques ranging from decomposition methods to manifold approximation methods, the features derived based on statistical parameterizations of the measured waveforms lead to reliable monitoring that is robust to changes in precision, resolution, and signal-to-noise ratio of the multipoint sensor measurements. The numbers of zero-crossing, negative-turning, and positive turning in the waveforms are the strongest causal signatures of the propagation of mechanical discontinuity. Higher order moments of the waveforms, such as variance, skewness and kurtosis, are also strong causal signatures of the propagation. Finally, the newly discovered causal signatures confirm that the statistical correlations and conventional feature rankings are not always statistically significant indicators of causality.

42 ENGINEERING↗

Predicting Air Traffic Management Initiatives Using Supervised Learning

Terminal Traffic Management Initiatives (TMIs) such as Ground Stops (GS) and Ground Delay Programs (GDP) are implemented to manage excess demand or lowered capacity at an airport. Air Traffic Flow Management (TFM) specialists identify situations such as aviation constraints, current and forecasted weather conditions, airport demand and capacity, and initiate TMIs for safe and orderly movement of air traffic. In this paper, we outline supervised learning techniques that can be used to predict and recommend TMIs at an airport based on current weather and airport conditions. Our research involves building classic Machine Learning (ML) models such as Logistic Regression, K-Nearest Neighbor, Random Forest and XGBoost, as well as Long short-term memory (LSTM) networks. We trained the models on 3-year historical data (weather, airport demand, capacity and TMIs) from Newark (EWR) airport which was selected based on its higher TMI implementation rates and varied weather conditions. Although Random Forest and XGBoost algorithms are able to predict if a TMI is needed or not, they have difficulty in predicting specific program type. For this purpose, we found that LSTM time-series forecasting models performed better as they also learn from past TMI program type sequences. This study also lays down the foundation for advanced modeling techniques and architectures to predict TMIs in advance for future periods. The ability to predict TMIs in advance will be highly beneficial to the traffic controllers and managers as this will help them to prepare for and manage TMIs more efficiently.

Manoj Agrawal↗

Semi-supervised learning approach for crack detection and identification in advanced gas-cooled reactor graphite bricks - 111

One of the life-limiting components of an Advanced Gas cooled Reactor (AGR) is its graphite core. The bricks present in the core undergo radiolytic oxidation throughout their lifetime which causes graphite weight loss and irradiation which can result in some of the bricks developing cracks. Understanding the nature and extent of brick cracking within the core is key to ensuring continued and extended operation of the AGR fleet. A semi-supervised machine learning classification algorithm is proposed as a method for improving the detection of cracked graphite bricks, by combining the labels derived from infrequent, detailed inspections of the core, with unlabeled, more frequent monitoring measurements taken during refueling operations. Semi-supervised machine learning, which is an emerging field in nuclear power condition monitoring, is the combination of ideas from both supervised and unsupervised machine learning whereby the data that is used to train the algorithm is a combination of labeled and unlabeled data. This paper introduces the initial research that has been undertaken in creating a semi-supervised self-training algorithm to detect the presence of graphite brick cracks and then proceeds to show that there is an improvement in the classification of graphite bricks using a semi-supervised machine learning classifier compared to supervised machine learning classifiers. This improved classification performance is encouraging as it does not require time consuming and costly human analysis to obtain extra learning information from available data. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Multisource Data Classification Using A Hybrid Semi-supervised Learning Scheme

In many practical situations thematic classes can not be discriminated by spectral measurements alone. Often one needs additional features such as population density, road density, wetlands, elevation, soil types, etc. which are discrete attributes. On the other hand remote sensing image features are continuous attributes. Finding a suitable statistical model and estimation of parameters is a challenging task in multisource (e.g., discrete and continuous attributes) data classification. In this paper we present a semi-supervised learning method by assuming that the samples were generated by a mixture model, where each component could be either a continuous or discrete distribution. Overall classification accuracy of the proposed method is improved by 12% in our initial experiments.

54 ENVIRONMENTAL SCIENCES↗

Feature extraction for subtle anomaly detection using semi-supervised learning

The demand for automated and effective monitoring techniques has soared with the increased digitization of industrial monitoring systems. State-of-the-art machine learning methods are effectively detecting abrupt changes in system states. However, these methods lack comparable maturity in detecting subtle changes that may be signs of incipient faults. This manuscript argues that the current anomaly detection methods can be enhanced by exploring weak patterns to enable subtle variation detection. Specifically, the concept of semi-supervised learning is employed, with labels representing knowledge about some anomalous conditions of a system. The basic idea is to extract a candidate set of weak patterns discarded by state-of-the-art baselining algorithms. With few labeled anomalous data, the algorithm selects the weak patterns and allows for their possible fusion using the highest sensitivity to the labeled anomalies. Here, the method’s applicability is demonstrated using a representative pressurized water reactor (PWR) model simulated by Dymola.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Cavity contour segmentation in chest radiographs using supervised learning and dynamic programming

Purpose: Efficacy of tuberculosis (TB) treatment is often monitored using chest radiography. Monitoring size of cavities in pulmonary tuberculosis is important as the size predicts severity of the disease and its persistence under therapy predicts relapse. The authors present a method for automatic cavity segmentation in chest radiographs. Methods: A two stage method is proposed to segment the cavity borders, given a user defined seed point close to the center of the cavity. First, a supervised learning approach is employed to train a pixel classifier using texture and radial features to identify the border pixels of the cavity. A likelihood value of belonging to the cavity border is assigned to each pixel by the classifier. The authors experimented with four different classifiers:k-nearest neighbor (kNN), linear discriminant analysis (LDA), GentleBoost (GB), and random forest (RF). Next, the constructed likelihood map was used as an input cost image in the polar transformed image space for dynamic programming to trace the optimal maximum cost path. This constructed path corresponds to the segmented cavity contour in image space. Results: The method was evaluated on 100 chest radiographs (CXRs) containing 126 cavities. The reference segmentation was manually delineated by an experienced chest radiologist. An independent observer (a chest radiologist) also delineated all cavities to estimate interobserver variability. Jaccard overlap measure Ω was computed between the reference segmentation and the automatic segmentation; and between the reference segmentation and the independent observer's segmentation for all cavities. A median overlap Ω of 0.81 (0.76 ± 0.16), and 0.85 (0.82 ± 0.11) was achieved between the reference segmentation and the automatic segmentation, and between the segmentations by the two radiologists, respectively. The best reported mean contour distance and Hausdorff distance between the reference and the automatic segmentation were, respectively, 2.48 ± 2.19 and 8.32 ± 5.66 mm, whereas these distances were 1.66 ± 1.29 and 5.75 ± 4.88 mm between the segmentations by the reference reader and the independent observer, respectively. The automatic segmentations were also visually assessed by two trained CXR readers as “excellent,” “adequate,” or “insufficient.” The readers had good agreement in assessing the cavity outlines and 84% of the segmentations were rated as “excellent” or “adequate” by both readers. Conclusions: The proposed cavity segmentation technique produced results with a good degree of overlap with manual expert segmentations. The evaluation measures demonstrated that the results approached the results of the experienced chest radiologists, in terms of overlap measure and contour distance measures. Automatic cavity segmentation can be employed in TB clinics for treatment monitoring, especially in resource limited settings where radiologists are not available.

62 RADIOLOGY AND NUCLEAR MEDICINE↗