Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Feature Extraction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Cosmic ray muon clustering for the MicroBooNE liquid argon time projection chamber using sMask-RCNN

In this article, we describe a modified implementation of Mask Region-based Convolutional Neural Networks (Mask-RCNN) for cosmic ray muon clustering in a liquid argon TPC and applied to MicroBooNE neutrino data. Our implementation of this network, called sMask-RCNN, uses sparse submanifold convolutions to increase processing speed on sparse datasets, and is compared to the original dense version in several metrics. The networks are trained to use wire readout images from the MicroBooNE liquid argon time projection chamber as input and produce individually labeled particle interactions within the image. These outputs are identified as either cosmic ray muon or electron neutrino interactions. We find that sMask-RCNN has an average pixel clustering efficiency of 85.9% compared to the dense network's average pixel clustering efficiency of 89.1%. We demonstrate the ability of sMask-RCNN used in conjunction with MicroBooNE's state-of-the-art Wire-Cell cosmic tagger to veto events containing only cosmic ray muons. The addition of sMask-RCNN to the Wire-Cell cosmic tagger removes 70% of the remaining cosmic ray muon background events at the same electron neutrino event signal efficiency. Furthermore, this event veto can provide 99.7% rejection of cosmic ray-only background events while maintaining an electron neutrino event-level signal efficiency of 80.1%. In addition to cosmic ray muon identification, sMask-RCNN could be used to extract features and identify different particle interaction types in other 3D-tracking detectors.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Analysis and Benchmarking of feature reduction for classification under computational constraints

Abstract Machine learning is most often expensive in terms of computational and memory costs due to training with large volumes of data. Current computational limitations of many computing systems motivate us to investigate practical approaches, such as feature selection and reduction, to reduce the time and memory costs while not sacrificing the accuracy of classification algorithms. In this work, we carefully review, analyze, and identify the feature reduction methods that have low costs/overheads in terms of time and memory. Then, we evaluate the identified reduction methods in terms of their impact on the accuracy, precision, time, and memory costs of traditional classification algorithms. Specifically, we focus on the least resource intensive feature reduction methods that are available in Scikit-Learn library. Since our goal is to identify the best performing low-cost reduction methods, we do not consider complex expensive reduction algorithms in this study. In our evaluation, we find that at quadratic-scale feature reduction, the classification algorithms achieve the best trade-off among competitive performance metrics. Results show that the overall training times are reduced 61%, the model sizes are reduced 6×, and accuracy scores increase 25% compared to the baselines on average with quadratic scale reduction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sub-millisecond keyhole pore detection in laser powder bed fusion using sound and light sensors and machine learning

Laser powder bed fusion is a mainstream additive manufacturing technology widely used to manufacture complex parts in prominent sectors, including aerospace, biomedical, and automotive industries. However, during the printing process, the presence of an unstable vapor depression can lead to a type of defect called keyhole porosity, which is detrimental to the part quality. In this study, we developed an effective approach to locally detect the generation of keyhole pores during the printing process by leveraging machine learning and a suite of optical and acoustic sensors. Simultaneous synchrotron x-ray imaging allows the direct visualization of pore generation events inside the sample, offering high-fidelity ground truth. A neural network model adopting SqueezeNet architecture using single-sensor data was developed to evaluate the fidelity of each sensor for capturing keyhole pore generation events. Our comparative study shows that the near infrared images gave the highest prediction accuracy, followed by 100 kHz and 20 kHz microphones, and the photodiode sensitive to processing laser wavelength had the lowest accuracy. Using a single sensor, over 90% prediction accuracy can be achieved with a temporal resolution as short as 0.1 ms. A data fusion scheme was also developed with features extracted using SqueezeNet neural network architecture and classification using different machine learning algorithms. Our work demonstrates the correlation between the characteristic optical and acoustic emissions and the keyhole oscillation behavior, and thereby provides strong physics support for the machine learning approach.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

Abstract Motivation Millions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. Results We developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementation The source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

The PAU Survey: Photometric redshifts using transfer learning from simulations

In this paper, we introduce the DEEPZ deep learning photometric redshift (photo-z) code. As a test case, we apply the code to the PAU survey (PAUS) data in the COSMOS field. DEEPZ reduces the σ68 scatter statistic by 50 percent at iAB = 22.5 compared to existing algorithms. This improvement is achieved through various methods, including transfer learning from simulations where the training set consists of simulations as well as observations, which reduces the need for training data. The redshift probability distribution is estimated with a mixture density network (MDN), which produces accurate redshift distributions. Our code includes an autoencoder to reduce noise and extract features from the galaxy SEDs. It also benefits from combining multiple networks, which lowers the photo-z scatter by 10 percent. Furthermore, training with randomly constructed coadded fluxes adds information about individual exposures, reducing the impact of photometric outliers. In addition to opening up the route for higher redshift precision with narrow bands, these machine learning techniques can also be valuable for broad-band surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Anomaly detection in the Zwicky Transient Facility DR3

We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of three stages: feature extraction, search of outliers with machine learning algorithms, and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of four automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinized by an expert. From these, 188 (68 per cent) were found to be bogus light curves – including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24 per cent) were previously reported sources whereas 23 (8 per cent) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e.g. one spectroscopically confirmed RS Canum Venaticorum star, four supernovae candidates, one red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation that can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Data-driven approach for synchrotron X-ray Laue microdiffraction scan analysis

A novel data-driven approach is proposed for analyzing synchrotron Laue X-ray microdiffraction scans based on machine learning algorithms. The basic architecture and major components of the method are formulated mathematically. It is demonstrated through typical examples including polycrystalline BaTiO 3 , multiphase transforming alloys and finely twinned martensite. The computational pipeline is implemented for beamline 12.3.2 at the Advanced Light Source, Lawrence Berkeley National Laboratory. The conventional analytical pathway for X-ray diffraction scans is based on a slow pattern-by-pattern crystal indexing process. Here, this work provides a new way for analyzing X-ray diffraction 2D patterns, independent of the indexing process, and motivates further studies of X-ray diffraction patterns from the machine learning perspective for the development of suitable feature extraction, clustering and labeling algorithms.

42 ENGINEERING↗
DOE OSTI2017

Adaptive Neuron Apoptosis for Accelerating Deep Learning on Large Scale Systems

Machine Learning and Data Mining (MLDM) algorithms are becoming ubiquitous in {\em model learning} from the large volume of data generated using simulations, experiments and handheld devices. Deep Learning algorithms -- a class of MLDM algorithms -- are applied for automatic feature extraction, and learning non-linear models for unsupervised and supervised algorithms. Naturally, several libraries which support large scale Deep Learning -- such as TensorFlow and Caffe -- have become popular. In this paper, we present novel techniques to accelerate the convergence of Deep Learning algorithms by conducting low overhead removal of redundant neurons -- {\em apoptosis} of neurons -- which do not contribute to model learning, during the training phase itself. We provide in-depth theoretical underpinnings of our heuristics (bounding accuracy loss and handling apoptosis of several neuron types), and present the methods to conduct adaptive neuron apoptosis. We implement our proposed heuristics with the recently introduced TensorFlow and using its recently proposed extension with MPI. Our performance evaluation on two difference clusters -- one connected with Intel Haswell multi-core systems, and other with nVIDIA GPUs -- using InfiniBand, indicates the efficacy of the proposed heuristics and implementations. Specifically, we are able to improve the training time for several datasets by 2-3x, while reducing the number of parameters by 30x (4-5x on average) on datasets such as ImageNet classification. For the Higgs Boson dataset, our implementation improves the accuracy (measured by Area Under Curve (AUC)) for classification from 0.88/1 to 0.94/1, while reducing the number of parameters by 3x in comparison to existing literature, while achieving a 2.44x speedup in comparison to the default (no apoptosis) algorithm.

Siegel, Charles M.↗
DOE OSTI2022

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While traditional ensemble classifiers yield improved task performance, such methods come with a steep cost in model complexity. Through knowledge distillation, it is possible to retain some performance improvements from an ensemble, while minimizing costs to model complexity. In this work, we implement a weakly supervised ensemble using clustering-constrained-attention multiple-instance learners (CLAM), which uses attention and instance-level clustering to identify task salient regions and feature extraction in whole slides. By applying logit-based and attention-based knowledge distillation, we show it is possible to retain some performance improvements resulting from the ensemble at zero cost to model complexity.

Alamudun, Folami↗
DOE OSTI2021

Identifying Light-Duty Vehicle Travel from Large-Scale Multimodal Wearable GPS Data with Novelty Detection Algorithms

Identifying travel mode within travel survey data sets, especially light-duty vehicle (LDV) travel, is foundational, though nontrivial, to travel behavior analysis and fuel consumption estimation. Current travel mode detection approaches require well-sampled and balanced data sets with ground truth travel mode labels. They are rarely applied and validated on large-scale, real-world data sets, which may not satisfy the data requirements. This paper proposes an LDV travel mode detection model as a supplement to current travel mode detection methods, for the case when the training set is highly (and/or completely) unbalanced, to the extent that classical machine-learning approaches become difficult or impossible to deploy. The proposed model uses a novelty detection technique-one-class support vector machines (OCSVMs)-and a novel exhaustive feature extraction (EFE) technique on continuous time series data (i.e., Global Positioning System [GPS] speed profiles) for single-mode trip trajectories. Training and validation of the model are conducted on a large-scale, real-world data set. The proposed method accurately identifies LDV trips from a broad set of multimodal trips by leveraging a wealth of preexisting in-vehicle GPS travel data. Additional sensitivity analysis sheds light on the optimal training size, which will benefit applications limited by highly imbalanced data. The paper also discusses performance comparison with regular machine-learning approaches, the model's robustness, and the potential to extend the proposed model to multimodal prediction.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Introducing SpaceNet 9 - Cross-Modal Satellite Imagery Registration for Natural Disaster Responses

Computer vision algorithms are increasingly leveraged to accelerate geospatial analysis for disaster response and recovery. As the diversity of remote sensing imagery grows with optical, SAR, and other modalities, a perquisite for analytics is cross-modal image registration. There is a high potential to harness computer vision for this pre-processing requirement toward enabling downstream analytics such as heterogeneous change detection, automated feature extraction, and data fusion. Advancement in these areas has the potential to simplify data wrangling tasks and further accelerate disaster response timelines. The SpaceNet 9 challenge (launching in mid-2024) focuses on addressing the cross-modal image registration problem and demonstrating the utility of such modules on earthquake impacted scenarios. This paper describes the motivation for the SpaceNet 9 and provides a first overview of the dataset, the baseline algorithm, and implications for seeking cross-modal image registration in Earth observation. Code is available at https://github.com/SpaceNetChallenge/SpaceNet9.

Hansch, Ronny↗
DOE OSTI2024

Data Security Defense: Modeling and Detection of Synchrophasor Data Spoofing Attack for Grid Edge

Data security and cyberattack have become critical issues in the distributed power system where adversaries can swap the source information of sensors or even spoof and alter measurements. However, the cyber security of the power system is challenged by the unpredictability and stealth of the spoofing attacks. Here, to protect the data security at the grid edge, this paper developed a synchrophasor data spoofing attack detection framework based on the time-frequency feature extraction techniques including the short-time Fourier transform (STFT) and object detection network for real-time synchrophasor data categorization and spoofing attack localization. The proposed approach outperforms earlier work in terms of spoofing attack detection and offers a vital localization function employing distributed synchrophasor sensors.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

A Physics-Aligned Multi-Domain Machine Learning Framework for Time-Localised Diagnosis of Power Electronics Faults

This paper presents a physics-aligned framework for fault diagnosis in multi-phase power-electronic systems using cycle-synchronous windowing and multi-domain features derived from Fourier, wavelet, and Hilbert–Huang representations. While both logistic regression and multilayer perceptron (MLP) models achieve perfect performance under standard evaluation, blind unseen testing reveals a critical failure in a baseline MLP. This is shown to arise from model selection based on validation accuracy. Using validation-loss-based selection restores correct unseen performance and improves confidence. Feature ablation shows that Fourier and wavelet features dominate, while computational analysis indicates that feature extraction, particularly HHT, governs runtime.

Kumar, Praveen [ORNL] (ORCID:0000000291877857)↗
DOE OSTI2024

Detecting Anomalies for Fire Prevention in Distribution Systems: Challenges and Analytical Techniques

Electric utilities in California have historically been linked to up to 10% of wildfires. To mitigate this risk, Southern California Edison has invested significantly in wildfire prevention strategies, including undergrounding cables and enhancing equipment inspections. This article explores a novel approach to fire prevention by detecting anomalies in the distribution system that may indicate potential fire hazards. The focus is on identifying arcing conditions through high-resolution point-on-wave (POW) measurements. Arcing, a precursor to fires, is challenging to detect due to its subtle transients and complex system topology. The article discusses the use of advanced signal processing and machine learning techniques, such as spectral correlation function and discrete wavelet transform, to extract features from POW data and accurately identify arcing events. The study demonstrates a high accuracy rate in detecting arcing, paving the way for improved fire prevention measures in electric distribution systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

In Situ Transmission Electron Microscopy: Signal processing challenges and examples

Transmission electron microscopy (TEM) is a powerful tool for imaging material structure and characterizing material chemistry. Recent advances in data collection technology for TEM have enabled high-volume and high-resolution data collection at a microsecond frame rate. Here, taking advantage of these advances in data collection rates requires the development and application of data processing tools, including image analysis, feature extraction, and streaming data processing techniques. In this article, we highlight a few areas in materials science that have benefited from combining signal processing and statistical analysis with data collection capabilities in TEM and present a future outlook on opportunities of integrating signal processing with automated TEM data analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2023

On the Investigation of Phase Fault Classification in Power Grid Signals: A Case Study for Support Vector Machines, Decision Tree and Random Forest

In monitoring the power grid, an ability to differentiate between fault types is essential to ensuring electrical safety. Accordingly, this study introduces a fault detection and classification method by considering different machine learning (ML) and feature extraction (FE) methods combinations. Specifically, the proposed method is established in two classification layers; the first layer determines the fault, and the second layer distinguishes the type of fault. Based on the proposed system model, this study seeks to determine the influential data attributes in a power grid signal using FE methods, including fast Fourier transform, power spectral density (PSD), auto-correlation, and wavelet transform (WT). A cross-comparison of the effectiveness of the Support Vector Machine (SVM), Decision Tree (DT), and Random Forest (RF) is also performed to accomplish the classification layers of the proposed method. The designed algorithm is analyzed under the various combinations of FE and ML methods, and outcomes are presented by considering the trade-off between computational complexity and prediction accuracy. The results reveal that the RF-based ML algorithm shows the most accurate classification performance with PSD, and the most time-saving of the models is the DT WT. Also, SVM emerges superior on a subsequent test of the simulated models on real-world signals.

Galbraith, Kelli↗
DOE OSTI2024

CNN-Based Phase Fault Classification in Real and Simulated Power Systems Data

This study proposes a convolutional neural network (CNN)–based two-step phase fault detection and identification method to classify anomalies in the power grid signal. Specifically, the first step checks the fault’s existence and determines the need for the second step. Subsequently, in the case of anomalies in the power grid signal, the second step identifies the type of fault, including line-to-line, single-line-to-ground, double-line-to-ground, and triple-line. Accordingly, the CNN architecture is both designed for the classification layers and trained with simulated data. To provide maximum prediction accuracy with minimum processing time, this study investigates the combinations of various feature extraction (FE) techniques, such as fast Fourier transform (FFT), amplitude and phase (AP), auto-correlation function, power spectral density, and wavelet transform (WT). Consequently, simulated and real-world results demonstrate that the proposed two-step method outperforms conventional one-step techniques, with the best performance obtained by using the combination of AP-AP, AP-WT, FFT-AP, and FFT-WT–based FE methods.

Alaca, Ozgur↗
DOE OSTI2023

Model Agnostic Bayesian Framework for Online Anomaly/Event Detection in PMU Data

Phasor measurement units (PMU) are integral to the modernization and automation plan of the electric power industry. A PMU data signature contains system-level events (e.g., faults, generation/load change, etc.) and any measurement/device-related errors. Therefore, the reliable and resilient operation of power systems is equivalent to the quality of the PMU data and the situation awareness provided by its data signature. Despite recent progress, current state-of-the-art methods are not fool-proof and have certain limitations tracing an error/abnormality to sensor sub-components and grid systems. This is because of technical challenges imposed by the scarcity of the labeled information, loss of data quality, and non-stationarity of data. In this paper, we consider the online PMU data stream as an output of a stochastic process and pose the anomaly/event detection as a changepoint detection problem dealing with detecting parameter changes in the underlying stochastic processes. The proposed model-agnostic framework relies on: (a) feature extraction utilizing the minimum volume enclosing ellipsoids (MVEE) method from raw PMU observations and (b) a Bayesian framework of changepoint detection. The validity of the proposed methodology is discussed through numerical experiments on real-world utility-scale PMU data.

Hossain, Ramij Raja↗