Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convolution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Assessing the Feasibility of Bordered Block Diagonal Reordering in Power System Matrices using Fully Convolutional Network

In electromagnetic transient (EMT) simulations for power systems and inverter-based resources (IBRs), the arrangement of states within the system's linear equations, represented by matrix A in Ax=b, is critical. The state ordering in matrix A can highlight distinct characteristics of the system's graph, and identifying an optimal state ordering is crucial for efficient computation. The choice of state ordering, however, is dependent on the solver used, as each solver may perform optimally with different matrix patterns. With a wide array of matrix reordering algorithms available, selecting the most suitable one becomes challenging without insights into the matrix's ideal configuration. To address this, the paper proposes a fully convolutional network (FCN) to evaluate the reordering potential of the A matrix into a bordered block diagonal (BBD) pattern, which is commonly observed in power system and IBR modeling. The FCN's assessment aims to streamline the solver's operation, which in turn could substantially reduce the computational time required to find a solution.

Xia, Qianxue↗
DOE OSTI2025

RE-INTEGRATE EMT Simulation Software: Graph Convolutional Network for Sparse Matrix Pattern Detection

The increasing complexity of power networks, driven by proliferation of inverters, presents analytical challenges that simplified models often fail to capture, necessitating Electromagnetic Transient (EMT) simulations. EMT models are represented as discretized differential-algebraic equations (DAEs), forming a linear system Ax = b that is computationally intensive to solve. Due to inherent sparsity of adjacency matrix A, distinct patterns emerge that, when accurately identified, enable efficient solver selection to minimize computation time. However, identifying ideal pattern is complicated by numerous reordering algorithms and limited structural insights. To address this, we introduce a Graph Convolutional Network (GCN) model for classifying sparse matrix patterns common in power system analysis. The model, achieving 96% test accuracy, is validated using PV plant models of 125 MW capacities connected to New England 39-bus transmission system (TS), and further scaled to a 4,992-bus network with 384 PV plants, yielding 191, 616 × 191, 616 sized A matrix. For all cases, the GCN model accurately identifies the matrix’s intrinsic sparse pattern, demonstrating its potential to enhance solver performance in EMT analysis.

Hossain, Md Rifat [Florida International Universit↗
DOE OSTI2023

Early Alarm: Robust Event Analysis for Power Systems using 1-D Fully Convolutional Network

This work presents a novel deep learning model for early, accurate, and robust detection, recognition, and temporal localization of multi-type events in large-scale power systems. The proposed method develops a unified 1-D fully convolutional network (FCN) model that takes time series of raw frequency signals measured from a power system as input, extracts distinguishing features, and predicts at every temporal point in the time series if an event is happening and what the type of the event is. Compared to existing methods, the proposed model eliminates the necessity for hand-crafted feature extraction or complicated data pre-processing, can flexibly handle input signals of arbitrary length, and precisely infer the event occurrence time. Most importantly, the model is intentionally trained with incomplete patterns, such that it is more robust to partial features of an event which is common in real-world online recognition, resulting in early alarm for power system failures. Extensive experimental results demonstrate that the proposed method achieves superior performance to the state-of-the-art, and also shows strong robustness to noise and system oscillations.

Li, Chengcheng↗
DOE OSTI2022

Effects-Based Monitoring of Geomagnetically-Induced Current Using a Convolutional Neural Network

Geomagnetically-induced current (GIC) due to space weather can flow in the power grid causing undesirable effects such as transformer overheating, misoperation of protection devices, and potential blackouts. It is therefore important to monitor GIC in the power grid to improve online situational awareness and decision-making of system operators during a geomagnetic disturbance. To avoid the costly installation of GIC monitors at transformers’ neutrals, it is desirable to find correlations between GIC and already-monitored parameters. Hence, this work proposed the use of a convolutional neural network (CNN) to compute GIC amplitudes from learned patterns in the time-series data of transformer even harmonic currents. Using an electromagnetic transient program, GIC injection simulations were performed for a modeled Dominion Energy Virginia (DEV) substation with two 504 MVA, 500/230 kV transformers. Data collected from these offline simulations were used to train the CNN to provide online GIC monitoring. Testing the CNN performance involved using real GIC measurements from published literature and from a physical GIC monitor in the DEV area. Finally, the results showed that the proposed method was able to provide GIC readings with a root mean squared error of 1.56 A/phase (equivalent to an average accuracy of 94%) for these real GIC waveforms.

42 ENGINEERING↗
DOE OSTI2021

Graph Convolutional Network-Based Topology Embedded Deep Reinforcement Learning for Voltage Stability Control

Topological variations in power system is a common phenomenon and can impose significant challenges to traditional controllers of power system. Recent study revealed the strength of deep reinforcement learning (DRL) based approaches in power system preventive and corrective control. But topological variations are difficult to capture using classical fully connected neural network (FCN) model and has not been explicitly modeled in previous work. Hence, we develop a Graph Convolutional Network (GCN) based DRL framework to tackle topology changes in control design of power system. The GCN model exploits the graph structure of the power network and helps the DRL agent to embed the topology information during learning process. Our GCN based approach is evaluated using the IEEE-39 bus system and it outperforms the FCN-based DRL scheme in terms of training convergence and control performance considering grid topology changes.

Hossain, Ramij Raja↗
DOE OSTI2021

Automatic detection of impact craters on Al foils from the Stardust interstellar dust collector using convolutional neural networks

NASA's Stardust mission utilized a sample collector composed of aerogel and aluminum foil to return cometary and interstellar particles to Earth. Analysis of the aluminum foil begins with locating craters produced by hypervelocity impacts of cometary and interstellar dust. Interstellar dust craters are typically less than one micrometer in size and are sparsely distributed, making them difficult to find. In this paper, we describe a convolutional neural network based on the VGG16 architecture that achieves high specificity and sensitivity in locating impact craters in the Stardust interstellar collector foils. Finally, we evaluate its implications for current and future analyses of Stardust samples.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Use of Convolutional Neural Network Image Classification and High-Speed Ion Probe Data Toward Real-Time Detonation Characterization in a Water-Cooled Rotating Detonation Engine

As rotating detonation engines (RDEs) progress in maturity, the importance of monitoring advancements toward development of active control becomes more critical. Experimental RDE data processing at time scales which satisfy real-time diagnostics will likely require the use of machine learning. This study aims to develop and deploy a novel real-time monitoring technique capable of determining detonation wave number, direction, frequency, and individual wave speeds throughout experimental RDE operational windows. To do so, the diagnostic integrates image classification by a convolutional neural network (CNN) and ionization current signal analysis. Wave mode identification through single-image CNN classification bypasses the need to evaluate sequential images and offers instantaneous identification of the wave mode present in the RDE annulus. Here, real-time processing speeds are achieved due to low data volumes required by the methodology, namely one short-exposure image and a short window of sensor data to generate each diagnostic output. The diagnostic acquires live data using a modified experimental setup alongside Pylon and PyDAQmx libraries within a python data acquisition environment. Lab-deployed diagnostic results are presented across varying wave modes, operating conditions, and data quality, currently executed at 3–4 Hz with a variety of iteration speed optimization options to be considered as future work. These speeds exceed that of conventional techniques and offer a proven structure for real-time RDE monitoring. The demonstrated ability to analyze detonation wave presence and behavior during RDE operation will certainly play a vital role in the development of RDE active control, necessary for RDE technology maturation toward industrial integration.

42 ENGINEERING↗
DOE OSTI2021

Evaluating Automated Face Identity-Masking Methods with Human Perception and a Deep Convolutional Neural Network

Face de-identification (or “masking”) algorithms have been developed in response to the prevalent use of video recordings in public places. Here, we evaluated the success of face identity masking for human perceivers and a deep convolutional neural network (DCNN). Eight de-identification algorithms were applied to videos of drivers’ faces, while they actively operated a motor vehicle. These masks were pre-selected to be applicable to low-quality video and to maintain coarse information about facial actions. Humans studied high-resolution images to learn driver identities and were tested on their recognition of active drivers in low-resolution videos. Faces in the videos were either unmasked or were masked by one of the eight algorithms. When participants were tested immediately after learning (Experiment 1), all masks reduced identification, with six of eight masks reducing identification to extremely poor performance. In a second experiment, two of the most effective masks were tested after a delay of 7 or 28 days. The delay did not further reduce identification of the masked faces. In all masked conditions, participants maintained stringent decision criteria, with low confidence in recognition, further indicating the effectiveness of the masks. Next, the DCNN performed an identity-matching task between high-resolution images and masked videos—a task analogous to that done by humans. The pattern of accuracy for the DCNN mirrored some, but not all, aspects of human performance, highlighting the need to test the effectiveness of identity masking for both humans and machines. The DCNN was also tested on its ability to match identity between masked and unmasked versions of the same video, based only on the face. DCNN performance for the eight masks offers insight into the nature of the information in faces that is coded in these networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

GRAPH CONVOLUTIONAL NEURAL NETWORK (GCNN)FOR POLYMER PROPERTY PREDICTION

SF-22-121 Graph convolutional neural networks (GCNN) are powerful machine learning models that apply a message-passing algorithm on adjacent nodes in a graph. Due to the natural representation of molecules via discrete graphs (where nodes encode atoms and edges encode bonds), GCNNs are an intuitive machine learning model chemical space informatics, including the prediction of polymer properties. Here, molecules are represented in feature and adjacency matrices where they are directly used as inputs to a GCNN.

FEINSTEIN, JEREMY↗
DOE OSTI2024

Convolution Neural Network for Fault Identification in Distribution Feeder with High Penetration Solar PV

Identification and zonal classification of the faults is a decisive factor in the relay’s decision to trip or not. Different types of fault like three-phase, line-to-line-to-ground and single-line-to-ground can occur at various locations in the feeder. These faults are seen as the variation in the instantaneous values of three-phase voltages and currents, i.e., waveforms, that are measured at the relay location. The objective of this work is to develop a machine learning model that can identify a fault and classify it to various protection zones based on measured waveforms. In this work, a data-driven relay based on Convolutional Neural Network (CNN) is proposed for fault identification in distribution feeders with high penetration solar PV. The proposed CNN model takes local current and voltage waveforms as input and classify it into fault, no-fault or a capacitor switching. Further, the CNN also attempts to identify fault zones based on the images of waveforms. The overall testing accuracy of the trained model exceeds 95%.

Ramesh, Meghana↗
DOE OSTI2025

Detecting Living-off-the-land Attacks Using K-means And Graph Convolutional Networks

The code ingests Zeek logs derived from network packet captures and goes through data preprocessing before it gets passed into a K-Means model that labels each device as either a client or server. Graph Convolutional Network (GCN) model is used to obtain the embeddings to represent the features in lower dimension. Last, K-means cluster analysis is used to cluster the embeddings for each class.

Quach, Anna [Idaho National Laboratory (INL), Idah↗
DOE OSTI2022

Classification of Cloud Particle Imagery from Aircraft Platforms Using Convolutional Neural Networks

Abstract A vast amount of ice crystal imagery exists from a variety of field campaign initiatives that can be utilized for cloud microphysical research. Here, nine convolutional neural networks are used to classify particles into nine regimes on over 10 million images from the Cloud Particle Imager probe, including liquid and frozen states and particles with evidence of riming. A transfer learning approach proves that the Visual Geometry Group (VGG-16) network best classifies imagery with respect to multiple performance metrics. Classification accuracies on a validation dataset reach 97% and surpass traditional automated classification. Furthermore, after initial model training and preprocessing, 10 000 images can be classified in approximately 35 s using 20 central processing unit cores and two graphics processing units, which reaches real-time classification capabilities. Statistical analysis of the classified images indicates that a large portion (57%) of the dataset is unusable, meaning the images are too blurry or represent indistinguishable small fragments. In addition, 19% of the dataset is classified as liquid drops. After removal of fragments, blurry images, and cloud drops, 38% of the remaining ice particles are largely intersecting the image border (≥10% cutoff) and therefore are considered unusable because of the inability to properly classify and dimensionalize. After this filtering, an unprecedented database of 1 560 364 images across all campaigns is available for parameter extraction and bulk statistics on specific particle types in a wide variety of storm systems, which can act to improve the current state of microphysical parameterizations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

TULIP: An RNA-seq-based Primary Tumor Type Prediction Tool Using Convolutional Neural Networks

Background: With cancer as one of the leading causes of death worldwide, accurate primary tumor type prediction is critical in identifying genetic factors that can inhibit or slow tumor progression. There have been efforts to categorize primary tumor types with gene expression data using machine learning, and more recently with deep learning, in the last several years. Methods In this paper, we developed four 1-dimensional (1D) Convolutional Neural Network (CNN) models to classify RNA-seq count data as one of 17 highly represented primary tumor types or 32 primary tumor types regardless of imbalanced representation. Additionally, we adapted the models to take as input either all Ensembl genes (60,483) or protein coding genes only (19,758). Unlike previous work, we avoided selection bias by not filtering genes based on expression values. RNA-seq count data expressed as FPKM-UQ of 9,025 and 10,940 samples from The Cancer Genome Atlas (TCGA) were downloaded from the Genomic Data Commons (GDC) corresponding to 17 and 32 primary tumor types respectively for training and validating the models. Results: All 4 1D-CNN models had an overall accuracy of 94.7% to 97.6% on the test dataset. Further evaluation indicates that the models with protein coding genes only as features performed with better accuracy compared to the models with all Ensembl genes for both 17 and 32 primary tumor types. For all models, the accuracy by primary tumor type was above 80% for most primary tumor types. Conclusions: We packaged all 4 models as a Python-based deep learning classification tool called TULIP (TUmor CLassIfication Predictor) for performing quality control on primary tumor samples and characterizing cancer samples of unknown tumor type. Further optimization of the models is needed to improve the accuracy of certain primary tumor types.

Jones, Sara↗
DOE OSTI2021

DeepDist: real-value inter-residue distance prediction with deep residual convolutional network

Abstract Background Driven by deep learning, inter-residue contact/distance prediction has been significantly improved and substantially enhanced ab initio protein structure prediction. Currently, most of the distance prediction methods classify inter-residue distances into multiple distance intervals instead of directly predicting real-value distances. The output of the former has to be converted into real-value distances to be used in tertiary structure prediction. Results To explore the potentials of predicting real-value inter-residue distances, we develop a multi-task deep learning distance predictor (DeepDist) based on new residual convolutional network architectures to simultaneously predict real-value inter-residue distances and classify them into multiple distance intervals. Tested on 43 CASP13 hard domains, DeepDist achieves comparable performance in real-value distance prediction and multi-class distance prediction. The average mean square error (MSE) of DeepDist’s real-value distance prediction is 0.896 Å 2 when filtering out the predicted distance ≥ 16 Å, which is lower than 1.003 Å 2 of DeepDist’s multi-class distance prediction. When distance predictions are converted into contact predictions at 8 Å threshold (the standard threshold in the field), the precision of top L/5 and L/2 contact predictions of DeepDist’s multi-class distance prediction is 79.3% and 66.1%, respectively, higher than 78.6% and 64.5% of its real-value distance prediction and the best results in the CASP13 experiment. Conclusions DeepDist can predict inter-residue distances well and improve binary contact prediction over the existing state-of-the-art methods. Moreover, the predicted real-value distances can be directly used to reconstruct protein tertiary structures better than multi-class distance predictions due to the lower MSE. Finally, we demonstrate that predicting the real-value distance map and multi-class distance map at the same time performs better than predicting real-value distances alone.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

Abstract Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN models on HPC systems to predict material properties of millions of molecules. We use HydraGNN, our in-house library for large-scale GCNN training, leveraging distributed data parallelism in PyTorch. We use ADIOS, a high-performance data management framework for efficient storage and reading of large molecular graph data. We perform parallel training on two open-source large-scale graph datasets to build a GCNN predictor for an important quantum property known as the HOMO-LUMO gap. We measure the scalability, accuracy, and convergence of our approach on two DOE supercomputers: the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) and the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). We present our experimental results with HydraGNN showing (i) reduction of data loading time up to 4.2 times compared with a conventional method and (ii) linear scaling performance for training up to 1024 GPUs on both Summit and Perlmutter.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Experimental Observations of the Topology of Convolutional Neural Network Activations

Topological data analysis (TDA) is a branch of computational mathematics, bridging algebraic topology and data science, that provides compact, noise-robust representations of complex structures. Deep neural networks (DNNs) learn millions of parameters associated with a series of transformations defined by the model architecture resulting in high-dimensional, difficult to interpret internal representations of input data. As DNNs become more ubiquitous across multiple sectors of our society, there is increasing recognition that mathematical methods are needed to aid analysts, researchers, and practitioners in understanding and interpreting how these models' internal representations relate to the final classification. In this paper we apply cutting edge techniques from TDA with the goal of gaining insight towards interpretability of convolutional neural networks used for image classification. We use two common TDA approaches to explore several methods for modeling hidden layer activations as high-dimensional point clouds, and provide experimental evidence that these point clouds capture valuable structural information about the model's process. First, we demonstrate that a distance metric based on persistent homology can be used to quantify meaningful differences between layers and discuss these distances in the broader context of existing representational similarity metrics for neural network interpretability. Second, we show that a mapper graph can provide semantic insight as to how these models organize hierarchical class knowledge at each layer. These observations demonstrate that TDA is a useful tool to help deep learning practitioners unlock the hidden structures of their models.

topological data analysis, deep learning↗
DOE OSTI2023

Data-Driven Validation of NOvA's Convolutional Neural Network for Electron (Anti)Neutrino Selection

NOvA is a long-baseline neutrino oscillation experiment, designed to make measurements of several oscillation parameters using muon neutrino disappearance and electron neutrino appearance. It consists of two functionally equivalent detectors and utilizes the Fermilab NuMI neutrino beam. NOvA uses a convolutional neural network for particle identification of electron neutrino events with a validation process that includes several data-driven techniques. These Muon Removed studies ensure that our classifier performs the same on data as it does on simulation. In particular, Muon Removed Electron-Added studies involve selecting muon neutrino charged current events from both data and simulation and replacing the muon with a simulated electron of similar energy. For Muon Removed Bremsstrahlung and Muon Removed Decay-in-Flight studies, we remove muonic hits from cosmic muons that have either experienced Bremsstrahlung radiation or decayed in flight, producing samples of pure electromagnetic showers. Each of these electron neutrino-like samples are then evaluated by our classifier to obtain selection efficiencies. Our most recent analysis showed good agreement in the electron selection efficiency between data and simulation using these techniques. Furthermore, these cross-checks can be extended to corrections to our predicted electron neutrino signal. The impact of one such set of corrections on the overall analysis results were also evaluated in thesis.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Classification and Localization of Fracture-Hit Events in Low-Frequency Distributed Acoustic Sensing Strain Rate with Convolutional Neural Networks

Summary Distributed acoustic sensing (DAS) has been used in the oil and gas industry as an advanced technology for surveillance and diagnostics. Operators use DAS to monitor hydraulic fracturing activities, examine well stimulation efficacy, and estimate complex fracture system geometries. Particularly, low-frequency DAS can detect geomechanical events such as fracture hits because hydraulic fractures propagate and create strain rate variations in the rock. Analysis of DAS data today is mostly done post-job and subject to interpretation methods. However, the continuous and dense data stream generated live by DAS poses the opportunity for more efficient and accurate real-time data-driven analysis. The objective of this study is to develop a machine learning-based workflow that can identify and locate fracture-hit events in simulated strain rate responses correlated with low-frequency DAS data. In this paper, “fracture hit” refers to a hydraulic fracture originating from a stimulated well intersecting an offset well. We start with building a single fracture propagation model to produce strain rate patterns observed at a hypothetical monitoring well. This model is used to generate two sets of strain rate responses with one set containing fracture-hit events. The labeled synthetic data are then used to train a custom convolutional neural network (CNN) model for identifying the presence of fracture-hit events. The same model is trained again for locating the event with the output layer of the model replaced with linear units. We achieved near-perfect predictions for both event classification and localization. These promising results prove the feasibility of using CNN for real-time event detection from fiber-optic sensing data. Additionally, we use edge detection techniques to recognize fracture-hit event patterns in strain rate images. The fracture-hit location can be identified using recognized pixels in the image. The accuracy of edge detection-based location identification is also plausible, but edge detection is dependent on the assumption of pattern shape and image quality, hence it is less robust compared to CNN models. This comparison further supports the need for CNN applications in image-based real-time fiber-optic sensing event detection.

Engineering↗