Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convolution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Differentiable Earth mover’s distance for data compression at the high-luminosity LHC

Abstract The Earth mover’s distance (EMD) is a useful metric for image recognition and classification, but its usual implementations are not differentiable or too slow to be used as a loss function for training other algorithms via gradient descent. In this paper, we train a convolutional neural network (CNN) to learn a differentiable, fast approximation of the EMD and demonstrate that it can be used as a substitute for computing-intensive EMD implementations. We apply this differentiable approximation in the training of an autoencoder-inspired neural network (encoder NN) for data compression at the high-luminosity LHC at CERN The goal of this encoder NN is to compress the data while preserving the information related to the distribution of energy deposits in particle detectors. We demonstrate that the performance of our encoder NN trained using the differentiable EMD CNN surpasses that of training with loss functions based on mean squared error.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Deep learning based event reconstruction for cyclotron radiation emission spectroscopy

The objective of the cyclotron radiation emission spectroscopy (CRES) technology is to build precise particle energy spectra. This is achieved by identifying the start frequencies of charged particle trajectories which, when exposed to an external magnetic field, leave semi-linear profiles (called tracks) in the time–frequency plane. Due to the need for excellent instrumental energy resolution in application, highly efficient and accurate track reconstruction methods are desired. Deep learning convolutional neural networks (CNNs) - particularly suited to deal with information-sparse data and which offer precise foreground localization—may be utilized to extract track properties from measured CRES signals (called events) with relative computational ease. In this work, we develop a novel machine learning based model which operates a CNN and a support vector machine in tandem to perform this reconstruction. A primary application of our method is shown on simulated CRES signals which mimic those of the Project 8 experiment—a novel effort to extract the unknown absolute neutrino mass value from a precise measurement of tritium β - -decay energy spectrum. When compared to a point-clustering based technique used as a baseline, we show a relative gain of 24.1% in event reconstruction efficiency and comparable performance in accuracy of track parameter reconstruction.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

A self-supervised workflow for particle picking in cryo-EM

High-resolution single-particle cryo-EM data analysis relies on accurate particle picking. To facilitate the particle picking process, a self-supervised workflow has been developed. This includes an iterative strategy, which uses a 2D class average to improve training particles, and a progressively improved convolutional neural network for particle picking. To automate the selection of particles, a threshold is defined (%/Res) using the ratio of percentage class distribution and resolution as a cutoff. This workflow has been tested using six publicly available data sets with different particle sizes and shapes, and can automatically pick particles with minimal user input. The picked particles support high-resolution reconstructions at 3.0 Å or better. This workflow is a step towards automated single-particle cryo-EM data analysis at the stage of particle picking. It may be used in conjunction with commonly used single-particle analysis packages such as Relion , cryoSPARC , cisTEM , SPHIRE and EMAN2 .

2D class averages↗
DOE OSTI2022

Ransomware Attack Modeling and Artificial Intelligence-Based Ransomware Detection for Digital Substations

Ransomware has become a serious threat to the current computing world, requiring immediate attention to prevent it. Ransomware attacks can also have disruptive impacts on operation of smart grids including digital substations. This paper provides a ransomware attack modeling method targeting disruptive operation of a digital substation and investigates an artificial intelligence (AI)-based ransomware detection approach. The proposed ransomware file detection model is designed by a convolutional neural network (CNN) using 2-D grayscale image files converted from binary files. Here, the experimental results show that the proposed method achieves 96.22% of ransomware detection accuracy.

artificial intelligence↗
DOE OSTI2025

Link Scheduling in Satellite Networks via Machine Learning Over Riemannian Manifolds

Low Earth Orbit (LEO) satellites play a crucial role in enhancing global connectivity, serving a complementary solution to existing terrestrial systems. In wireless networks, scheduling is a vital process that allocates time-frequency resources to users for interference management. However, LEO satellite networks face significant challenges in scheduling their links towards ground users due to the satellites’ mobility and overlapping coverage. This paper addresses the dynamic link scheduling problem in LEO satellite networks by considering spatio-temporal correlations introduced by the satellites’ movements. The first step in the proposed solution involves modeling the network over Riemannian manifolds, thanks to their representation as symmetric positive definite matrices. We introduce two machine learning (ML)-based link scheduling techniques that model the dynamic evolution of satellite positions and link conditions over time and space. To accurately predict satellite link states, we present a recurrent neural network (RNN) over Riemannian manifolds, which captures spatio-temporal characteristics over time. Furthermore, we introduce a separate model, the convolutional neural network (CNN) over Riemannian manifolds, which captures geometric relationships between satellites and users by extracting spatial features from the network topology across all links. Simulation results demonstrate that both RNN and CNN over Riemannian manifolds deliver comparable performance to the fractional programming-based link scheduling (FPLinQ) benchmark. Remarkably, unlike other ML-based models that require extensive training data, both models only need 30 training samples to achieve over 99% of the sum rate while maintaining similar computational complexity relative to the benchmark.

42 ENGINEERING↗
DOE OSTI2022

Accelerating Hydraulic Fracture Imaging by Deep Transfer Learning

Deep transfer learning has a great success story in computer vision (CV), natural language processing, and many other fields. In this communication, we are going to push forward the deep transfer learning to the hydraulic fracture imaging problem by proposing a two-step approach: 1) train a convolutional neural network (CNN) to reconstruct target geometries by a relatively large amount of approximated field patterns generated from a simplified model and 2) fine-tune the top layers of transferred CNN by a small amount of true field patterns generated through a full model. The advantages include the rapid generation of large amount of data through the simplified model and the high reconstruction accuracy through a careful design of the deep transfer learning. Here, the CNN trained through the deep transfer learning can accurately reconstruct the lateral extent and direction of fractures with unseen conductivity and white Gaussian noise, showing a notable acceleration/accuracy enhancement over the previous CNN trained by mixing a small number of true data into the approximated data as data augmentation.

02 PETROLEUM↗
DOE OSTI2021

Fault Detection on Seismic Structural Images Using a Nested Residual U-Net

Automatic identification of faults on seismic structural images is a challenging yet crucial task in quantitative seismic interpretation. Human picking or attribute-based fault detection methods may misidentify faults on noisy, complex seismic images. In this work, we develop a new automatic fault detection method using a nested residual U-shaped convolutional neural network. Each of the encoders and decoders in this neural network is a residual U-Net, leading to a nested architecture. The final fault map results from the fusion of three fault maps with low, medium, and high fault resolutions. We demonstrate the excellent fault-detection capability of our nested neural network using a series of synthetic and field seismic images. We find that our approach produces clearer and more interpretable fault maps than the current state-of-the-art U-Net fault detection method, particularly on noisy seismic images. Our new automatic fault detection method can facilitate reliable quantitative seismic interpretation on field seismic images.

58 GEOSCIENCES↗
DOE OSTI2021

InversionNet3D: Efficient and Scalable Learning for 3-D Full-Waveform Inversion

Seismic full-waveform inversion (FWI) techniques aim to find a high-resolution subsurface geophysical model provided with waveform data. Some recent effort in data-driven FWI has shown some encouraging results in obtaining 2-D velocity maps. However, due to high computational complexity and large memory consumption, the reconstruction of 3-D high-resolution velocity maps via deep networks is still a great challenge. Here, in this article, we present InversionNet3D (InvNet3D), an efficient and scalable encoder–decoder network for 3-D FWI. The proposed method employs group convolution in the encoder to establish an effective hierarchy for learning information from multiple sources while cutting down unnecessary parameters and operations at the same time. The introduction of invertible layers further reduces the memory consumption of intermediate features during training and, thus, enables the development of deeper networks with more layers and higher capacity as required by different application scenarios. Experiments on the 3-D Kimberlina dataset demonstrate that InvNet3D achieves state-of-the-art reconstruction performance with lower computational cost and lower memory footprint compared to the baseline.

58 GEOSCIENCES↗
DOE OSTI2024

Rapid Monitoring and Defense Approach for Resilience Improvement of Grid Cyber Security

Cyber-physical systems and electric utilities significantly depend on the reliability and efficiency of information and operational technology. However, false data injection attacks based on synchrophasor measurement data pose a serious threat to the safe and reliable operation of modern power systems. Here, to mitigate this problem, a rapid monitoring and defense approach is proposed to defend against cyber attacks. Initially, the Time and Frequency based Convolutional neural Network (TFCN) is proposed to detect different types of attacks. Within the TFCN, the advances are that both time and frequency domain information can be fused without extra spectrum analysis methods, and can save detection time to speed the calculation efficiency using the developed time-frequency block. Next, a comprehensive defense strategy is developed for multiple cyber attacks to ensure the stability and resilience of the power system according to the feedback detection results. The advances of this strategy are that different control strategies can be automatically selected to recover the stability to the greatest extent according to the detected attacks. To verify the effectiveness of the proposed approach, the high-speed frequency measurements collected from the wide-area monitoring system are used. The results demonstrate that the cyber attack detection performance can reach 95.57% accuracy, outperforming both traditional and some advanced neural networks. Importantly, the defense strategy is conducted and verified in a modified IEEE 39 bus system as well, which illustrates profound performance in faster stability restoration.

Comprehensive defense strategy↗
DOE OSTI2025

Topology-Aware Reinforcement Learning for Voltage Control: Centralized and Decentralized Strategies

Volt-VAR control (VVC) methods based on deep reinforcement learning (DRL) can effectively control distribution grid voltage and minimize power loss by implementing corrective and preventive control measures on the reactive power output of inverter-based distributed energy resources (DERs). However, model-free DRL-based VVC approaches usually cannot capture the important topological feature of the power system since they use a fully-connected network (FCN) to deliver the action. Therefore, this paper proposes a graph convolutional network (GCN)-based DRL approach that can employ the topological information of the network to take better control action for regulating the voltage. Our implementation allows for both centralized and decentralized configurations, utilizing a single agent and multiple agents respectively. Although the centralized GCN-based DRL approach has its advantages of minimizing voltage fluctuation and power loss, it is not suitable for large scale power systems due to its challenges in terms of scalability, computation speed and potential single points of failure. Therefore, these problems can be resolved using the decentralized GCN-based DRL approach. Moreover, to ensure the safe operation of the model, our proposed approach incorporates an exponential barrier function while formulating the reward function for each agent. To validate performance of the proposed approaches, the proposed model is tested on modified IEEE test systems and the performances are measured in terms on voltage fluctuation reduction, minimization of power loss and computational speed. Finally, the results show that the proposed topology-aware approach outperforms the FCN-based DRL approach in terms of reducing voltage fluctuation and minimizing power loss of the network. Moreover, it is shown that the decentralized GCN-based DRL has faster computational speed than other approaches.

42 ENGINEERING↗
DOE OSTI2024

Deep Multitask Learning Models for Radiation Estimation at High Energy Accelerator Facility

Controlling the dose of radiation exposure in potential radioactive facilities is critical for ensuring the safety of staff and the public. Here, in this paper, we developed machine learning models to estimate radiation exposure efficiently at the Thomas Jefferson National Accelerator Facility (JLab), aiming to enhance safety in both accelerator facilities and public areas. Multiple sensors were deployed around the three experimental halls at JLab. Data on single-beam currents, energy levels, and radiation values at the sensor locations were collected during accelerator operation. We proposed a multi-task learning model for radiation estimation, utilizing either one-dimensional convolutional neural networks (1-D CNNs) or long short-term memory networks (LSTMs) as the backbone. The proposed model was trained to simultaneously estimate radiation levels at the sensor locations. Experimental results demonstrated that the proposed model with LSTM backbone achieved the best estimation performance, with an average R 2 score of 0.7557 for estimation within the same year and 0.7157 for estimation across different years. These results significantly surpassed those of competing models.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Low Activity Tritium Detection in CCDs Using Deep Learning Techniques

Here, this study explores the use of charge-coupled devices (CCDs) for detecting low-energy beta particles from tritium decay - a critical signal for nuclear safety, nuclear nonproliferation, and environmental monitoring. We employ a dual approach utilizing both measured CCD data and detailed Geant4 simulations. Our analysis compares classical techniques with advanced deep learning methods, including convolutional neural networks (CNNs), autoencoders trained exclusively on tritium data, and preliminary studies on boosted decision trees (BDTs). The CNN, trained on mixed signal/background datasets, demonstrates superior classification performance, while the autoencoder shows the potential of unsupervised, background-agnostic strategies when background characteristics are poorly defined. These results highlight the excellent sensitivity achievable thanks to the background rejection made possible by information-rich CCD data, paving the way for improved portable tritium monitoring.

Autoencoder↗
DOE OSTI2020

The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism

Here, we present scalable hybrid-parallel algorithms for training large-scale 3D convolutional neural networks. Deep learning-based emerging scientific workflows often require model training with large, high-dimensional samples, which can make training much more costly and even infeasible due to excessive memory usage. We solve these challenges by extensively applying hybrid parallelism throughout the end-to-end training pipeline, including both computations and I/O. Our hybrid-parallel algorithm extends the standard data parallelism with spatial parallelism, which partitions a single sample in the spatial domain, realizing strong scaling beyond the mini-batch dimension with a larger aggregated memory capacity. We evaluate our proposed training algorithms with two challenging 3D CNNs, CosmoFlow and 3D U-Net. Our comprehensive performance studies show that good weak and strong scaling can be achieved for both networks using up to 2K GPUs. More importantly, we enable training of CosmoFlow with much larger samples than previously possible, realizing an order-of-magnitude improvement in prediction accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Practical Event Location Estimation Algorithm for Power Transmission System Based on Triangulation and Oscillation Intensity

Event location in power systems is quite essential information for system operators to enhance control-room situational awareness capability. Therefore, it is of great importance to develop an event location estimation algorithm for transmission systems with high accuracy. With the development of wide-area measurement system (WAMS) such as FNET/GridEye, and the synchrophasor measurement devices (SMDs) such as frequency disturbance recorders (FDRs), the synchronous measurement data including frequency, voltage amplitude and phase angle can be collected and used for event location estimation. First, the phase angle and rate of change of frequency (RoCoF) trajectories are respectively used for determining two sets of wave arrival time associated with each FDR. Then, a convolutional neural network (CNN) is utilized to determine the wave arrival order to select the more suitable set of wave arrival times for a given case and to perform corresponding modifications. Next, the oscillation intensity associated with each FDR is determined based on phase angle trajectories in the center of inertia (COI) coordinate system. Finally, the multiple criteria for event location estimation are represented. In conclusion, case studies and comparisons between the proposed and previous algorithms using actual and confirmed cases in U.S. power systems are performed to demonstrate the effectiveness and improvement of the proposed algorithm in practical applications.

frequency disturbance recorder (FDR)↗
DOE OSTI2021

Frequency Injection Based HVDC Attack-Defense Control Via Squeeze-Excitation Double CNN

Due to the independent controllability and fast power regulation capability, the High Voltage Direct Current (HVDC) system could be a prospective technology to provide multiple ancillary services to the system besides conventional bulk power transmission. However, with the increase of False Data Injection Attacks (FDIAs) on PMU data, the HVDC system could have the wrong response once the collected data that the HVDC system relied on is attacked, thus threatening the system operating security. How to ensure the security of the PMU-based HVDC ancillary service control become an urgent issue. To mitigate the risk, this paper proposed an HVDC attack-defense control based on the FDIAs detection method. Firstly, the Squeeze-Excitation based Double Convolutional Neural Networks (SE-DCNN) is proposed to realize fast identification of the attacking frequency type based on the time and frequency domain signals. Furthermore, the duration time of FDIAs is detected by the local outlier factor. Then, utilizing the results from SE-DCNN, HVDC ancillary service control framework is reorganized and an HVDC attack defense control is proposed for suppressing the potential influence of various types of FDIAs on the HVDC system ancillary service. Different experiments results demonstrate that the proposed method has the ability to significantly mitigate the frequency deviation and oscillation under the FDIA.

42 ENGINEERING↗
DOE OSTI2022

Wavelet and Deep-Learning-Based Approach for Generation System Problematic Parameters Identification and Calibration

Accurate models of generation systems are critical for maintaining reliable and secure grid operations. In this paper, a novel and systematic approach is proposed to identify and calibrate the generation system problematic parameters using continuous wavelet transform (CWT) and advanced deep-learning technology. The phasor measurement unit (PMU) data are used through “event playback” to check whether the parameter calibration is required, and if yes, a group of suspicious parameters will be identified as the primary problematic parameter candidates (PPCs). These primary PPCs are randomly perturbed to generate the event playback simulation data, which are used by the CWT and convolutional neural networks (CNNs) to further narrow down the primary PPCs into a smaller set of candidates. Then, the identified candidates are perturbed again to generate massive event playback simulation data for training a parameter calibration neural network. Here, we designed a multi-output neural network structure to find the mappings between the perturbed parameters and the simulation data using both CNN and long short-term memory (LSTM) models. Finally, the well-trained and tested CNN-LSTM model is used to estimate the accurate value of the suspicious parameters with actual PMU measurements. The proposed CNN-LSTM network can accurately and reliably estimate the generation-system problematic parameters, and has better performance when compared to other machine-learning methods, such as the multilayer perceptron network and the conditional variational autoencoder method. The accuracy and effectiveness of the proposed approach have been validated through simulation and real-world data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

FeederGAN: Synthetic Feeder Generation via Deep Graph Adversarial Nets

This paper presents a novel, automated, generative adversarial networks (GAN) based synthetic feeder generation mechanism, abbreviated as FeederGAN. FeederGAN digests real feeder models represented by directed graphs via a deep learning framework powered by GAN and graph convolutional networks (GCN). Information of a distribution feeder circuit is extracted from its model input files so that the device connectivity is mapped onto the adjacency matrix and the device characteristics, such as circuit types (i.e., 3-phase, 2-phase, and 1-phase) and component attributes (e.g., length and current ratings), are mapped onto the attribute matrix. Then, Wasserstein distance is used to optimize the GAN and GCN is used to discriminate the generated graphs from the actual ones. A greedy method based on graph theory is developed to reconstruct the feeder using the generated adjacency and attribute matrices. Furthermore, our results show that the GAN generated feeders resemble the actual feeder in both topology and attributes verified by visual inspection and by empirical statistics obtained from actual distribution feeders.

42 ENGINEERING↗