Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convolution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,693 records · Page 94
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Applying Machine Learning to the Classification of DC-DC Converters (Milestone 2 Deliverable Report)

Since extending the Autodetector to a convolutional neural network (CNN) machine learning classifier model, an effort has been executed to demonstrate its ability to distinguish not only a switching DC-DC converter as high voltage, but identify the make and model of a converter on which it was trained. This was achieved by collecting data in a noisy environment, pre-processing the time domain data to obtain composite images using a method that improves upon that of the prior research, then validating a trained CNN model to an accuracy of 100% on a selected candidate converter.

42 ENGINEERING↗
DOE OSTI2020

How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection

Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Reification of latent microstructures: On supervised unsupervised and semi-supervised deep learning applications for microstructures in materials informatics

Machine learning (ML), including deep learning (DL), has become increasingly popular in the last few years due to its continually outstanding performance. In this context, we apply machine learning techniques to "learn" the microstructure using both supervised and unsupervised DL techniques. In particular, we focus (1) on the localization problem bridging (micro)structure (localized) property using supervised DL and (2) on the microstructure reconstruction problem in latent space using unsupervised DL. The goal of supervised and semi-supervised DL is to replace crystal plasticity finite element model (CPFEM) that maps from (micro)structure (localized) property, and implicitly the (micro)structure (homogenized) property relationships, while the goal of unsupervised DL is (1) to represent high-dimensional microstructure images in a non-linear low-dimensional manifold, and (2) to discover a way to interpolate microstructures via latent space associating with latent microstructure variables. At the heart of this report is the applications of several common DL architectures, including convolutional neural networks (CNN), autoencoder (AE), and generative adversarial network (GAN), to multiple microstructure datasets, and the quest of neural architecture search for optimal DL architectures.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Start-to-end self-consistent simulation of the UEM beamline

Addressing the preliminary design review report of the ultrafast electron microscope (UEM), start-to-end self-consistent simulations were carried out to explore fundamental aspects affecting the resolution (space-charge effects, energy spread, shot-to-shot energy jitter, contrast formation, etc.). The electron beam energy spread in the single-shot mode and the effective energy spread in the accumulation mode, equivalent to the convolution of the single-shot energy spread and shot-to-shot energy jitter, have been identified as the ultimate limits of the instrument resolution. A full self-consistent simulation from the photocathode to the detector is being performed using the GPT code to find out what resolution can be achieved. Preliminary results of the start-to-end simulations are reported in this note and the plan of further simulations is presented.

43 PARTICLE ACCELERATORS↗
DOE OSTI2020

Microseismic Event Denoising: Removal of Borehole Waves

The vertical borehole array at Farnsworth Unit, TX is used to monitor microseismic activity in the subsurface around the Carbon Capture and Sequestration (CCS) reservoir. The array consists of 16 3-component seismometers spaced vertically in a single borehole. Tube or borehole waves traveling up or down the borehole can corrupt signals of interest, such as microseismic events. A denoising convolutional neural network (DCNN) was trained to remove borehole waves from seismic waveforms of microseismic events for the purpose of reducing unwanted signal detections and better characterizing events of interest. This R&D leverages the work of Sandia colleague Rigo Tibi, who used a DCNN developed by Greg Beroza at Stanford University to improve the signal-to-noise ratio (SNR) of teleseismic events detected by the International Monitoring System.

58 GEOSCIENCES↗
DOE OSTI2021

Low-Power Deep Learning Inference using the SpiNNaker Neuromorphic Platform

n this presentation we will discuss recent results on using the SpiNNaker neuromorphic platform (48-chip model) for deep learning neural network inference. We use the Sandia Labs developed Whet stone spiking deep learning library to train deep multi-layer perceptrons and convolutional neural networks suitable for the spiking substrate on the neural hardware architecture. By using the massively parallel nature of SpiNNaker, we are able to achieve, under certain network topologies, substantial network tiling and consequentially impressive inference throughput. Such high-throughput systems may have eventual application in remote sensing applications where large images need to be chipped, scanned, and processed quickly. Additionally, we explore complex topologies that push the limits of the SpiNNaker routing hardware and investigate how that impacts mapping software-implemented networks to on-hardware instantiations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Identifying precursors of daily to seasonal hydrological extremes over the USA using deep learning techniques and climate model ensembles

Focal Area(s): We focus on two areas of crosscutting interest for DOE: 1) predictability of extreme precipitation and drought in the USA and 2) the integration of climate models with new AI tools, such as convolutional neural networks (CNN) and methods to understand their output (e.g. layer-wise relevance propagation; LRP). This project fits into focus area 3 of this call for white paper using AI to gain insight from complex data, including explainable AI tools. Science Challenge: Predicting hydrological extremes is important due to their impacts on people, agriculture and infrastructure. This prediction is difficult due to the infrequent occurrence of extremes and their complexity. However, extreme events can be related to more predictable conditions in the ocean, such as El Nino, long-term soil moisture or large scale modes of climate variability, such as the North Atlantic Oscillation (NAO).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

DeepMerge: Studying Distant Merging Galaxies with Deep Neural Networks

The hierarchical merging of galaxies is a probe of the cosmos to test the canonical ?CDM cosmology paradigm. A particularly interesting period is "cosmic high noon" at redshifts z? 2?3, during which star formation rates were the highest, and significant amounts of stellar mass assembled into galaxy-scale bodies. Detecting galaxy mergers in observations by conventional automated methods (which use extracted parameters of galaxy structure - asymmetry, clumpiness, concentration etc.) or by visual inspection has proven to be quite time-consuming and prone to errors. Convolutional Neural Network (CNNs) are a primary representative of deep learning algorithms which are used in computer vision tasks by training to detect features in images. CNNs were already used for classification of low-redshift merging galaxies [1, 2]. Here we will use CNN to to learn directly from images (without the need to extract morphology parameters) of distant merging galaxies in order to distinguish between merging and non-merging objects.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Insights into cosmological structure formation with machine learning.

Our modern understanding of cosmological structure formation posits that small matter density fluctuations present in the early Universe, as traced by the cosmic microwave background, grow via gravitational instability to form extended haloes of dark matter. A theoretical understanding of the structure, evolution and formation of dark matter haloes is an essential step towards unravelling the intricate connection between halo and galaxy formation, needed to test our cosmological model against data from upcoming galaxy surveys. Physical understanding of the process of dark matter halo formation is made difficult by the highly non-linear nature of the haloes' evolution. I describe a new approach to gain physical insight into cosmological structure formation based on machine learning. This approach combines the ability of machine learning algorithms to learn non-linear relationships, with techniques that enable us to physically interpret the learnt mapping. I describe applicatio ns of th e method, with the aim of investigating which aspects of the early universe density field impact the later formation of dark matter haloes. First I present a case where the process of halo formation is turned into a binary classification problem; the algorithm predicts whether or not dark matter `particles' in the initial conditions of a simulation will collapse into haloes of a given mass range. Second, I present its generalization to regression, where the algorithm infers the final mass of the halo to which each particle will later belong. I show that the initial tidal shear does not play a significant role compared to the initial density field in establishing final halo masses. Finally, I demonstrate that extending the framework to deep learning algorithms such as convolutional neural networks allows us to explore connections between the early universe and late time haloes beyond those studied by existing analytic approximations of halo collapse.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Bridging Power System Protection Gaps with Data-driven Approaches

Protection is a critical function in power systems to avoid equipment damage, maintain personnel safety, and support system reliability. However, current protective relay technology cannot adequately protect equipment and personnel from effects of some events; these deficiencies are termed protection gaps. In this research, a data-driven approach is proposed to complement traditional protection technology and distinguish fault conditions from transients caused by normal operations. A convolutional neural network (CNN) based fault detection approach is implemented to achieve data translation invariance of the time-series input data. As a result, the data-driven method can accurately detect system faults despite variation and noise in the input data. In addition, using the CNN–based method avoids the complicated manual feature extraction procedure required by many traditional data-driven methods. The effectiveness of the proposed approach is tested on four kinds of protection gaps: high impedance faults, transformer/generator inter-turn faults, distribution system PV circuit faults, and the mis-operation situations of Zone 3 line protection relays operating under system stress. Finally, a transfer learning method is also proposed to address the common issue of data-driven methods for which real-world training data are scarce. Extensive study results demonstrate that the proposed approach can accurately bridge power system protection gaps.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Constraints on Neutrino Oscillation Parameters from Neutrinos and Antineutrinos with Machine Learning

NOvA is a two detector, long baseline neutrino oscillation experiment measuring the oscillations of muon neutrinos from the \numi neutrino beam over a baseline of \SI{810}{km}. The experiment uses four oscillation channels, $\numu \rightarrow \numu$, $\numubar \rightarrow \numubar$, $\numu \rightarrow \nue$, and $\numubar \rightarrow \nuebar$, with a peak neutrino energy of \SI{1.8}{GeV}. This dissertation describes the analysis of these channels using a dataset of $13.6\times10^{20}$ protons on target neutrino beam mode and $12.5\times10^{20}$ protons on target antineutrino beam mode. The analysis makes use of improvements in the treatment of systematic uncertainties and machine learning techniques to reconstruct neutrino interactions. A technique for decorrelating systematic errors using principle component analysis was utilized to reduce and optimize neutrino cross section and beam related uncertainties. The improved machine learning algorithms make use of convolutional ne ural net works for neutrino event classification, particle classification, and instance segmentation. The selection of neutrino signal events utilizing the neutrino event classifier shows an efficiency of 63\% for the selection of electron neutrinos in neutrino beam mode and 75\% for electron antineutrinos in antineutrino beam mode. Using this algorithm, 82 appearing electron neutrino candidates and 33 appearing electron antineutrino candidates were observed with expected backgrounds of 26.8 and 14.0 respectively. In addition, 211 surviving muon neutrino candidates and 105 muon antineutrino candidates were identified with a purity of more than 96\% using the same neutrino event classifier. Fitting these data to the three flavor neutrino oscillation model, using constraints on \thetaonetwo, \thetaonethree, and \dmsqonetwo from solar and reactor neutrino experiments, the oscillation parameters are measured to be $\sintwothree = 0.57^{+0.04}_{-0.03}$, $\dmsqthreetwo = \SI[parse-numbers= false]{+ 2.41\pm0.07 \times 10^{-3}}{eV^2}$, and $\dcp=0.82^{+0.27}_{-0.87}\pi$ with a preference for the normal neutrino mass hierarchy. Leading systematic uncertainties for these measurements come from detector calibration and neutrino interaction models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Scaling Inference Using Triton to Accelerate Particle Physics at the LHC and DUNE

DUNE and the LHC experiments consist of unique and cutting-edge particle detectors that create massive, complex, and rich datasets with billions of events. They require sophisticated algorithms to reconstruct and interpret the data. Modern machine learning algorithms provide a powerful toolset to detect and classify particles, from familiar image processing convolutional neural networks to newer graph neural network architectures. A full reconstruction of these particle collisions requires novel approaches to handle the computing challenge of processing so much raw data. In a series of studies, physicists from Fermilab, CERN, and university groups explored how to accelerate their data processing using the Triton Inference Server.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Verification of Triso Fuel Burnup Using Machine Learning Algorithms

Pebble Bed Reactors are fueled with fuel pebbles that are circulated multiple times through the reactor vessel before discharge. During the normal operation of a PBR, ejected pebbles are returned to the reactor or discharged depending on the fuel burnup and physical condition of the pebbles. The burnup measurement is usually based on detected radiation signatures of fission products accumulated in the pebble fuel over burnup. Previous research has shown that height of photopeaks of fission products, such as 134 Cs, 137 Cs, 154 Eu, etc., can be used independently or in combination to infer or predict the level of burnup in the fuel. However, it remains challenging to measure such complex sources due to self-shielding effects, strong radiation background and intervening materials. Another operational challenge is the required high throughput of burnup measurement, which necessitates limited measurement time and thus impacts quality of measured gamma-ray spectra. Hence, advanced spectral analysis methods are needed to analyze the noisy gamma spectra and predict the burnup values. We propose to use machine learning (ML) method to interpret gamma-ray spectra and predict the burnup values of the pebbles. ML has achieved widespread success and adoption across a few domains that require pattern recognition and analysis in varied data types. In this work, we apply three proven ML approaches - multilayer perceptrons, convolutional neural networks, and transformers - to the task of predicting fuel burnup from measured gamma spectra, and compile a dataset of simulated spectra for training and validation of the ML models. In this paper, we will discuss the network architecture of these three ML approaches and compare the performance of the simplest of these (MLP) to a standard linear regression.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

LDMS-GPU: Lightweight Distributed Metric Service (LDMS) for NVIDIA GPGPUs

GPUs are now a fundamental accelerator for many high-performance computing applications. They are viewed by many as a technology facilitator for the surge in fields like machine learning and Convolutional Neural Networks. To deliver the best performance on a GPU, we need to create monitoring tools to ensure that we optimize the code to get the most performance and efficiency out of a GPU. Since NVIDIA GPUs are currently the most commonly implemented in HPC applications and systems, NVIDIA tools are the solution for performance monitoring. The Light-Weight Distributed Metric System (LDMS) at Sandia is an infrastructure widely adopted for large-scale systems and application monitoring. Sandia has developed CPU application monitoring capability within LDMS. Therefore, we chose to develop a GPU monitoring capability within the same framework. In this report, we discuss the current limitations in the NVIDIA monitoring tools, how we overcame such limitations, and present an overview of the tool we built to monitor GPU performance in LDMS and its capabilities. Also, we discuss our current validation results. Most of the performance counter results are the same in both vendor tools and our tool when using LDMS to collect these results. Furthermore, our tool provides these statistics during the entire runtime of the tool as a time series and not just aggregate statistics at the end of the application run. This allows the user to see the progress of the behavior of the applications during their lifetime.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

Recently, Graph Neural Networks (GNNs) have received a lot of interest because of their success in learning representations from graph structured data. However, GNNs exhibit different compute and memory characteristics compared to traditional Deep Neural Networks (DNNs). Graph convolutions require feature aggregations from neighboring nodes (known as the aggregation phase), which leads to highly irregular data accesses. GNNs also have a very regular compute phase that can be broken down to matrix multiplications (known as the combination phase). All recently proposed GNN accelerators utilize different dataflows and microarchitecture optimizations for these two phases. Different communication strategies between the two phases have been also used. However, as more custom GNN accelerators are proposed, the harder it is to qualitatively classify them and quantitatively contrast them. In this work, we present a taxonomy to describe several diverse dataflows for running GNN inference on accelerators. This provides a structured way to describe and compare the design-space of GNN accelerators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Deep Learning Systems for Increased Safeguards Surveillance Review Productivity

Nuclear safeguards inspectors expend significant time and maintain intense focus in reviewing video surveillance for safeguards relevant events. To increase efficiency and reduce the time burden of safeguards inspectors performing surveillance data review, this paper presents a novel deep learning (DL) systems concept to integrate generalized DL models into the safeguards surveillance review workflow. The Agency is investigating several DL algorithms for object recognition, localization, tracking, and flagging relevant activities. The project team is working closely with nuclear safeguards inspectors to identify review use cases (based on specific safeguards objectives) and collect their associated requirements. We focused on CANDU and LWR Nuclear Power Plants (NPPs) and their associated dry storage areas as these present a particularly heavy burden on the inspector surveillance review process due to the number of these facilities under safeguards worldwide. Initial DL algorithm results on safeguards data are promising. Using a convolutional neural network, the team attained a mean average precision (mAP) of 92.9% identifying and localizing spent fuel (SF) casks from a 475 surveillance image dataset. Further, the team had initial success in training a recurrent neural network to identify reactor area activities in video clips, successfully indicating when SF casks enter or exit a pool. We discuss how such DL algorithms would be integrated into the Next Generation Surveillance Review (NGSR) software application. Another issue impacting review productivity is the long time inspectors may have to wait when running these algorithms in NGSR. We present a concept to pre-process remotely collected surveillance data with DL models as the data arrives to IAEA headquarters so that results are already available when starting a new review in NGSR. The proposed DL system concept shows a pathway and workflow for increasing an inspector’s surveillance review productivity by quickly and accurately identifying declared and undeclared safeguards relevant objects and activities in large quantities of surveillance imagery data.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2021

Performance of Pulsed Thermal Tomography Imaging with Machine Learning-Based Classification of Defects in Additively Manufactured Structures

Additive manufacturing (AM) is an emerging method for cost-efficient fabrication of complex topology nuclear reactor parts from high-strength corrosion resistance alloys, such as stainless steel and Inconel. AM of metallic structures for nuclear energy applications is currently based on laser powder bed fusion (LPBF) process. Some of the challenges with using LPBF method for nuclear manufacturing include the possibility of introducing pores into metallic structures. Integrity of AM structures needs to be evaluated nondestructively because material flaws could lead to premature failures in high temperature nuclear reactor environment. Currently, there exist limited capabilities to evaluate actual AM structures non-destructively. Pulsed Thermal Tomography Imaging (PTT) provides a capability for non-destructive evaluation (NDE) of subsurface defects in arbitrary size structures. The PTT method is based on recording material surface temperature transients with infrared (IR) camera following thermal pulse delivered on material surface with flash light. The PTT method has advantages for NDE of actual AM structures because the method involves one-sided non-contact measurements and fast processing of large sample areas captured in one image. Following initial qualification of an AM component for deployment in a nuclear reactor, a PTT system can also be used for in-service nondestructive evaluation (NDE) applications. In this report, we describe recent progress in enhancing PTT capabilities in detecting and visualizing microscopic defects in metallic specimens. The thermal tomography (TT) algorithm obtains depth reconstructions of spatial effusivity from the data cube of sequentially recorded surface temperatures. However, interpretation of TT images is non-trivial because of blurring of images with increasing depth. To address this challenge, we have developed a deep learning convolutional neural network (CNN) to classify size and orientation subsurface defects in simulated TT images. CNN is trained on a database of TT images created for a set of simulated metallic structures with elliptical subsurface voids. Test of CNN performance demonstrate the ability to classify radii and angular orientation of subsurface defects in TT images. In addition, we have shown that CNN trained on elliptical defects is capable of classifying irregular-shaped defects obtained from scanning electron microscopy (SEM) of stainless steel sections printed with LPBF.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Integrating static PRA information with risk informed safety margin characterization (RISMC) simulation methods

The overall objective of the project was to develop a computationally feasible and user-friendly mechanized process to integrate traditional probabilistic risk assessment (PRA) and dynamic PRA (DPRA) results. Starting with the systematic identification of items in an existing PRA that need dynamic augmentation, the project used a generic 4-loop pressurized reactor (PWR) and 3-loop PWR as example plants. Station blackout (SBO) and large break loss of coolant accident SBLOCA) were selected as the example initiating events. Using the traditional event-tree (ET)/fault-tree (FT) methodology augmented by dynamic evet tree approach, the potential consequences of the initiating events were simulated with RELAP-3D and MELCOR/RASCAL codes to cover Level 1 through Level 3 of PRA. RAVEN and ADAPT software were used to generate Level 1 simulations with RELAP-3D and Level 2/3 simulations with MELCOR (Level 2)/RASCAL (Level 3), respectively. Example branching conditions (BCs) for SBO included AC power recovery time, valve repair failure time, reactor coolant pump leak time/break size and emergency power supply duration to a total of 9. Example BCs for LOCA included off-site power recovery time, diesel generator power recovery time, auxiliary feed water system operation time, safety relief valve failure to open upon demand, reactor coolant pump seal break time and size to a total of 21. Each RELAP-3D simulation (9,587 scenarios) was labelled OK or Core Damage based on the maximum allowed peak clad temperature (2,100oF). Each MELCOR simulation (4610 scenarios) was labeled as Bin over 10rem or Bin 0-10rem based on the dose at the site boundary. The scenarios were clustered based on the criteria above using the mean shift methodology. Classical PRA (CPRA) and DPRA results were compared to identify the ET sequences that need DPRA augmentation. Several approaches were proposed for the incorporation of these sequences into CPRA using clustering with the mean shift methodology, restructuring the CPRA ETs by adding new BCs/sequences, and using the concept of a limit surface. Procedures for decision making regarding the possible consequences of an initiating event (e.g. core damage or not, site evacuation or not) were developed using a convolutional neural network (CNN), a recurrent neural network (RNN) and a transformer neural network (TNN). The project has led to two PhD degrees, three archival journal papers and five refereed conference proceedings.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗