SEARCH · Engineering Papers

Results for “Convolutional neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Inverse Analysis with Variational Autoencoders: A Comparison of Shallow and Deep Networks

Inverse problems are applied to determine unknown properties by matching observational data with a physical model that often takes many parameters as input. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented involving regularization with a technique known as a variational autoencoder (VAE), which is trained to map a high-dimensional parameter space with a complex structure to a low-dimensional latent space with a simple structure. We apply this approach to unconditioned realizations of the parameters (heterogeneous hydraulic fields) for a hydrogeological inverse problem. Two types of hydraulic conductivity fields are used to evaluate the characterization for the different levels of heterogeneity complexity of the physical inputs. This approach keeps the computational cost of generating the training data low. The reason is unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any groundwater flow model forward runs. In addition, this approach applies regularization on a low-dimensional latent space from the VAE and increases optimization efficiency through automatic differentiation. Furthermore, two different neural network (NN) structures are tested for their utility in using the VAE for inverse analysis. The performance of a deep, convolutional neural network strongly depends on the dimensionality of the latent space, which requires tuning. In contrast, a shallow, dense neural network provides consistently accurate characterization without tuning. Furthermore, our approach evaluates the advantages of the shallow, dense neural network over the deep, convolutional one and enables future application to a wide range of inverse problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Measuring thermal profiles in high explosives using neural networks

We present a new method for calculating the temperature profile of high explosive (HE) material using a Convolutional Neural Network (CNN). To train/test the CNN, we have developed a hybrid experiment/simulation method for collecting acoustic and temperature data. We experimentally heat cylindrical containers of HE material until detonation/deflagration, where we continuously measure the acoustic bursts through the HE using multiple acoustic transducers lined around the exterior container circumference. However, measuring the temperature profile in the HE in an experiment would require inserting a large number of thermal probes, which would disrupt the heating process. Thus, we use two thermal probes, one at the HE center and one at the wall. We then use numerical simulation of the heating process to calculate the temperature distribution and correct the simulated temperatures based on the experimental center and wall temperatures. We calculate temperature errors on the order of 15 °C, which is ~12% of the range of temperatures in the experiment. We also investigate how the algorithm’s accuracy is affected by the number of acoustic receivers used to collect each measurement and the resolution of the temperature prediction. This work provides a means of assessing the safety status of HE material, which cannot be achieved using existing temperature measurement methods. In addition, it has implications for a range of other applications where internal temperature profile measurements would provide critical information. These applications include detecting chemical reactions, observing thermodynamic processes such as combustion, monitoring metal or plastic casting, determining the energy density in thermal storage capsules, and identifying abnormal battery operations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

WavPool: A New Block for Deep Neural Networks

Modern deep neural networks comprise many operational layers, such as dense or convolutional layers, which are often collected into blocks. In this work, we introduce a new, wavelet-transform-based network architecture that we call the multi-resolution perceptron: by adding a pooling layer, we create a new network block, the WavPool. The first step of the multi-resolution perceptron is transforming the data into its multi-resolution decomposition form by convolving the input data with filters of fixed coefficients but increasing size. Following image processing techniques, we are able to make scale and spatial information simultaneously accessible to the network without increasing the size of the data vector. WavPool outperforms a similar multilayer perceptron while using fewer parameters, and outperforms a comparable convolutional neural network by ~ 10% on relative accuracy on CIFAR-10.

McDermott, Samuel D.↗

DOE OSTI2020

A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence

In the present study, we investigate different data-driven parameterizations for large eddy simulation of two-dimensional turbulence in the a priori settings. These models utilize resolved flow field variables on the coarser grid to estimate the subgrid-scale stresses. We use data-driven closure models based on localized learning that employs a multilayer feedforward artificial neural network with point-to-point mapping and neighboring stencil data mapping, and convolutional neural network fed by data snapshots of the whole domain. The performance of these data-driven closure models is measured through a probability density function and is compared with the dynamic Smagorinksy model (DSM). The quantitative performance is evaluated using the cross-correlation coefficient between the true and predicted stresses. We analyze different frameworks in terms of the amount of training data, selection of input and output features, their characteristics in modeling with accuracy, and training and deployment computational time. We also demonstrate computational gain that can be achieved using the intelligent eddy viscosity model that learns eddy viscosity computed by the DSM instead of subgrid-scale stresses. Here, we detail the hyperparameters optimization of these models using the grid search algorithm.

42 ENGINEERING↗

DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

MicroBooNE, a Liquid Argon Time Projection Chamber (LArTPC) located in the $\nu_{\mu}$-dominated Booster Neutrino Beam at Fermilab, has been studying $\nu_{e}$ charged-current (CC) interaction rates to shed light on the MiniBooNE low energy excess. The LArTPC technology employed by MicroBooNE provides the capability to image neutrino interactions with mm-scale precision. Computer vision and other machine learning techniques are promising tools for image processing that could boost efficiencies for selecting $\nu_{e}$-CC and other rare signals, reduce cosmic and beam-induced backgrounds, and improve the reconstruction of neutrino energies. The MicroBooNE experiment has been at the forefront of developing and testing such techniques for use in physics analyses. In this poster we overview deep-learning based reconstruction methods. We will showcase the use of a recurrent neural network to estimate neutrino energies and present a new reconstruction framework that uses convolutional neural networks to locate neutrino interaction vertices, tag pixels with track and shower labels, and perform particle identification on reconstructed clusters. We will present studies characterizing the performance of these new tools and demonstrate their effectiveness through their use in an inclusive $\nu_{e}$-CC event selection.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2020

Simulated production of OH, HO 2 , CH 2 O, and CO 2 during dilute fuel oxidation can predict 1st-stage ignition delays

Chemical kinetics simulations are used to explore whether detailed measurements of relevant chemical species during the oxidation of very dilute fuels (less than 1 Torr partial pressure) in a high-pressure plug flow reactor (PFR) can predict autoignition propensity. We find that for many fuels the timescale for the onset of spontaneous oxidation in dilute fuel/air mixtures in a simple PFR is similar to the 1st-stage ignition delay time (IDT) at stoichiometric engine-relevant conditions. For those fuels that deviate from this simple trend, the deviation is closely related to the peak rate of production of OH, HO 2 , CH 2 O, and CO 2 formed during oxidation. We use these insights to show that an accurate correlation between simulated profiles of these species in a PFR and 1st-stage IDT can be developed using convolutional neural networks. Our simulations suggest that the accuracy of such a correlation is 10–50%, which is appropriate for rapid fuel screening and may be sufficient for predictive fuel performance modeling.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Ransomware Attack Modeling and Artificial Intelligence-Based Ransomware Detection for Digital Substations

Ransomware has become a serious threat to the current computing world, requiring immediate attention to prevent it. Ransomware attacks can also have disruptive impacts on operation of smart grids including digital substations. This paper provides a ransomware attack modeling method targeting disruptive operation of a digital substation and investigates an artificial intelligence (AI)-based ransomware detection approach. The proposed ransomware file detection model is designed by a convolutional neural network (CNN) using 2-D grayscale image files converted from binary files. Here, the experimental results show that the proposed method achieves 96.22% of ransomware detection accuracy.

artificial intelligence↗

DOE OSTI2024

Rapid Monitoring and Defense Approach for Resilience Improvement of Grid Cyber Security

Cyber-physical systems and electric utilities significantly depend on the reliability and efficiency of information and operational technology. However, false data injection attacks based on synchrophasor measurement data pose a serious threat to the safe and reliable operation of modern power systems. Here, to mitigate this problem, a rapid monitoring and defense approach is proposed to defend against cyber attacks. Initially, the Time and Frequency based Convolutional neural Network (TFCN) is proposed to detect different types of attacks. Within the TFCN, the advances are that both time and frequency domain information can be fused without extra spectrum analysis methods, and can save detection time to speed the calculation efficiency using the developed time-frequency block. Next, a comprehensive defense strategy is developed for multiple cyber attacks to ensure the stability and resilience of the power system according to the feedback detection results. The advances of this strategy are that different control strategies can be automatically selected to recover the stability to the greatest extent according to the detected attacks. To verify the effectiveness of the proposed approach, the high-speed frequency measurements collected from the wide-area monitoring system are used. The results demonstrate that the cyber attack detection performance can reach 95.57% accuracy, outperforming both traditional and some advanced neural networks. Importantly, the defense strategy is conducted and verified in a modified IEEE 39 bus system as well, which illustrates profound performance in faster stability restoration.

Comprehensive defense strategy↗

DOE OSTI2020

Detection of Synchrophasor False Data Injection Attack using Feature Interactive Network

The synchrophasor data recorded by Phasor Measurement Units (PMUs) plays an increasingly critical role in the regulation and situational awareness of power systems. However, the widely installed PMUs are vulnerable to multiple malicious attacks from cyber hackers during data transmission and storage. To address this problem, a Modified Ensemble Empirical Mode Decomposition (MEEMD) is proposed first to extract the intrinsic mode functions of each Synchrophasor Data Attacks (SDA). The frequency-based adaptive screening criterion embedded in MEEMD is used to eliminate the false intrinsic mode functions. Next, a Multivariate Convolutional Neural Network (MCNN) is proposed to identify multiple SDA by utilizing the extracted intrinsic mode functions and original SDA as input vectors. A fusion block as the main structure of MCNN is also leveraged to increase the diversity of features and compress the model parameters. Integrating MEEMD and MCNN, a framework with automatic feature extraction and multi-source information fusion capability, referred to as Feature Interactive Network (FIN), is proposed to detect multiple SDA. Based on the proposed FIN framework, six types of SDA are explored for the first time using actual synchrophasor data in FNET/Grideye that was collected from different locations in the U.S. Eastern Interconnection. Finally, a large quantity of experiments with different attack strengths are used to evaluate the adaptability and classification performance of the proposed FIN.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

Computationally Efficient Multiscale Neural Networks Applied to Fluid Flow in Complex 3D Porous Media

Abstract The permeability of complex porous materials is of interest to many engineering disciplines. This quantity can be obtained via direct flow simulation, which provides the most accurate results, but is very computationally expensive. In particular, the simulation convergence time scales poorly as the simulation domains become less porous or more heterogeneous. Semi-analytical models that rely on averaged structural properties (i.e., porosity and tortuosity) have been proposed, but these features only partly summarize the domain, resulting in limited applicability. On the other hand, data-driven machine learning approaches have shown great promise for building more general models by virtue of accounting for the spatial arrangement of the domains’ solid boundaries. However, prior approaches building on the convolutional neural network (ConvNet) literature concerning 2D image recognition problems do not scale well to the large 3D domains required to obtain a representative elementary volume (REV). As such, most prior work focused on homogeneous samples, where a small REV entails that the global nature of fluid flow could be mostly neglected, and accordingly, the memory bottleneck of addressing 3D domains with ConvNets was side-stepped. Therefore, important geometries such as fractures and vuggy domains could not be modeled properly. In this work, we address this limitation with a general multiscale deep learning model that is able to learn from porous media simulation data. By using a coupled set of neural networks that view the domain on different scales, we enable the evaluation of large ( $$>512^3$$ > 512 3 ) images in approximately one second on a single graphics processing unit. This model architecture opens up the possibility of modeling domain sizes that would not be feasible using traditional direct simulation tools on a desktop computer. We validate our method with a laminar fluid flow case using vuggy samples and fractures. As a result of viewing the entire domain at once, our model is able to perform accurate prediction on domains exhibiting a large degree of heterogeneity. We expect the methodology to be applicable to many other transport problems where complex geometries play a central role.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Multi-task graph neural networks for simultaneous prediction of global and atomic properties in ferromagnetic systems *

Abstract We introduce a multi-tasking graph convolutional neural network, HydraGNN, to simultaneously predict both global and atomic physical properties and demonstrate with ferromagnetic materials. We train HydraGNN on an open-source ab initio density functional theory (DFT) dataset for iron-platinum with a fixed body centered tetragonal lattice structure and fixed volume to simultaneously predict the mixing enthalpy (a global feature of the system), the atomic charge transfer, and the atomic magnetic moment across configurations that span the entire compositional range. By taking advantage of underlying physical correlations between material properties, multi-task learning (MTL) with HydraGNN provides effective training even with modest amounts of data. Moreover, this is achieved with just one architecture instead of three, as required by single-task learning (STL). The first convolutional layers of the HydraGNN architecture are shared by all learning tasks and extract features common to all material properties. The following layers discriminate the features of the different properties, the results of which are fed to the separate heads of the final layer to produce predictions. Numerical results show that HydraGNN effectively captures the relation between the configurational entropy and the material properties over the entire compositional range. Overall, the accuracy of simultaneous MTL predictions is comparable to the accuracy of the STL predictions. In addition, the computational cost of training HydraGNN for MTL is much lower than the original DFT calculations and also lower than training separate STL models for each property.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2024

DLSIA: Deep Learning for Scientific Image Analysis

DLSIA (Deep Learning for Scientific Image Analysis) is a Python-based machine learning library that empowers scientists and researchers across diverse scientific domains with a range of customizable convolutional neural network (CNN) architectures for a wide variety of tasks in image analysis to be used in downstream data processing. DLSIA features easy-to-use architectures, such as autoencoders, tunable U-Nets and parameter-lean mixed-scale dense networks (MSDNets). Additionally, this article introduces sparse mixed-scale networks (SMSNets), generated using random graphs, sparse connections and dilated convolutions connecting different length scales. For verification, several DLSIA-instantiated networks and training scripts are employed in multiple applications, including inpainting for X-ray scattering data using U-Nets and MSDNets, segmenting 3D fibers in X-ray tomographic reconstructions of concrete using an ensemble of SMSNets, and leveraging autoencoder latent spaces for data compression and clustering. As experimental data continue to grow in scale and complexity, DLSIA provides accessible CNN construction and abstracts CNN complexities, allowing scientists to tailor their machine learning approaches, accelerate discoveries, foster interdisciplinary collaboration and advance research in scientific image analysis.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2023

Statistical Decomposition and Machine Learning to Clean In Situ Spaceflight Magnetic Field Measurements

Robust in situ magnetic field measurements are critical to understanding the various mechanisms that couple mass, momentum, and energy throughout our solar system. However, the spacecraft on which magnetometers are often deployed contaminate the magnetic field measurements via onboard subsystems including reaction wheels and magnetorquers. Two magnetometers can be deployed at different distances from the spacecraft to determine an approximation of the interfering field for subsequent removal, but constant data streams from both magnetometers can be impractical due to power and telemetry limitations. Here we propose a method to identify and remove time-varying magnetic interference from sources such as reaction wheels using statistical decomposition and convolutional neural networks, providing high-fidelity magnetic field data even in cases where dual-sensor measurements are not constantly available. For example, a measurement interval from the Parker Solar Probe outboard magnetometer experienced a 95.1% reduction in reaction wheel interference following application of the proposed technique.

magnetic field data↗

DOE OSTI2022

Wavelet and Deep-Learning-Based Approach for Generation System Problematic Parameters Identification and Calibration

Accurate models of generation systems are critical for maintaining reliable and secure grid operations. In this paper, a novel and systematic approach is proposed to identify and calibrate the generation system problematic parameters using continuous wavelet transform (CWT) and advanced deep-learning technology. The phasor measurement unit (PMU) data are used through “event playback” to check whether the parameter calibration is required, and if yes, a group of suspicious parameters will be identified as the primary problematic parameter candidates (PPCs). These primary PPCs are randomly perturbed to generate the event playback simulation data, which are used by the CWT and convolutional neural networks (CNNs) to further narrow down the primary PPCs into a smaller set of candidates. Then, the identified candidates are perturbed again to generate massive event playback simulation data for training a parameter calibration neural network. Here, we designed a multi-output neural network structure to find the mappings between the perturbed parameters and the simulation data using both CNN and long short-term memory (LSTM) models. Finally, the well-trained and tested CNN-LSTM model is used to estimate the accurate value of the suspicious parameters with actual PMU measurements. The proposed CNN-LSTM network can accurately and reliably estimate the generation-system problematic parameters, and has better performance when compared to other machine-learning methods, such as the multilayer perceptron network and the conditional variational autoencoder method. The accuracy and effectiveness of the proposed approach have been validated through simulation and real-world data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Unveiling mesoscopic structures in distorted lamellar phases through deep learning-based small angle neutron scattering analysis

Hypothesis: The formation of distorted lamellar phases, distinguished by their arrangement of crumpled, stacked layers, is frequently accompanied by the disruption of long-range order, leading to the formation of interconnected network structures commonly observed in the sponge phase. Nevertheless, traditional scattering functions grounded in deterministic modeling fall short of fully representing these intricate structural characteristics. Our hypothesis posits that a deep learning method, in conjunction with the generalized leveled wave approach used for describing structural features of distorted lamellar phases, can quantitatively unveil the inherent spatial correlations within these phases. Experiments and Simulations: This report outlines a novel strategy that integrates convolutional neural networks and variational autoencoders, supported by stochastically generated density fluctuations, into a regression analysis framework for extracting structural features of distorted lamellar phases from small angle neutron scattering data. To evaluate the efficacy of our proposed approach, we conducted computational accuracy assessments and applied it to the analysis of experimentally measured small angle neutron scattering spectra of AOT surfactant solutions, a frequently studied lamellar system. Findings: The findings unambiguously demonstrate that deep learning provides a dependable and quantitative approach for investigating the morphology of wide variations of distorted lamellar phases. It is adaptable for deciphering structures from the lamellar to sponge phase including intermediate structures exhibiting fused topological features. In conclusion, this research highlights the effectiveness of deep learning methods in tackling complex issues in the field of soft matter structural analysis and beyond.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Deep learning model to detect various synchrophasor data anomalies

High-density synchrophasors provide valuable information for power grid situational awareness, operation and control. Unfortunately, due to factors including communication instability and hardware failure, their data quality can be greatly deteriorated by anomalies. Since the anomalies can impact the performance of the synchrophasor applications, it is of paramount significance to propose a model to detect anomalies in synchrophasor. In this study, a convolutional neural network model is established to detect and classify the anomalies in the synchrophasor measurements. Additionally, four types of anomalies observed in actual synchrophasors including erroneous patterns, random spikes, missing points and high-frequency interferences are considered in this study. The proposed model is extensively evaluated via field-collected measurements from the synchrophasor network in Jiangsu grid, China. The superior performance of the proposed model indicates the great potential of using deep learning for the detection of abnormal synchrophasor measurements.

42 ENGINEERING↗

DOE OSTI2023

Simple and sophisticated models of Taylor’s cylinder impact

We present a study based on the simplest possible models for Taylor’s cylinder impact problem, in addition to examina-tion of using convolutional neural networks (CNNs) to map cylinder profiles to strength calibrations. We find that the approximate treatments of Taylor and Hawkyard compare well with hydrodynamic simulations using an equivalent assumption of constant flow stress. The CNN models prove to be well suited to successfully infer parameterizations of the Preston-Tonks-Wallace model of plastic deformation based on the deformed profile of an impacted cylinder.

36 MATERIALS SCIENCE↗