Results for “deep neural network” — Page 7

DOE OSTI2022

Enhanced deep neural networks with transfer learning for distribution LMP considering load and PV uncertainties

As the flexibility of generation and demand increases in distribution systems, the residential loads are emerging as a promising means to participate in demand response and the transactive energy market. Market pricing is an instrumental mechanism for the distribution system operator to exploit the full potential of the flexible resources. The distribution locational marginal price (DLMP) can be used to guide the residential load consumption. This type of market signal helps the distribution system operator to optimize the scheduling of all resources while satisfying related network constraints through a day-ahead market. However, solving the optimization problem for large-scale systems can be computationally expensive. To address the scalability and practicability limitations of the DLMP framework, a learning-based approach is proposed in this paper to complement the day-ahead distribution market framework. Here, the proposed approach combines long short-term memory and transfer learning to develop deep neural network that can capture the spatial–temporal correlation of the input data. The model can determine the optimal DLMP for each node in a distribution system without the system parameters required to formulate the optimization problem. Testing results on IEEE 33-bus and 123-bus systems show that the proposed approach can generate a comparable DLMP against the optimization solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

Optimization of the deep neural network parameters for generating homogenized fuel assembly data for nodal codes

Homogenized fuel assembly (FA) data is a typical input data for nodal codes. Generating that data, however, could be time-consuming. One of promising ways to mitigate the computational burden of generating macroscopic cross-sections is to use trained artificial neural network (ANN) models for predicting nuclear data. However, there is a challenge to make the model support variable FA geometry. In this work, two most common types of FA were combined in one ANN model. Since there could be multiple ways of converting 2-dimensional FA data into 1-dimensional input vector for ANN, three different approaches of data flattening were evaluated. The input parameters included each fuel pin enrichment, fuel temperature, moderator temperature and boron concentration. The output parameters were 2-group macroscopic cross-sections (XS) and pin power distribution (HFF). A fully connected deep neural network (DNN) model was trained and tested using pre-generated data obtained with lattice physics code STREAM. The results of this study showed no statistically significant difference in the accuracy of XS and HFF generation for all 3 tested input vector orders. This means that fully connected DNN for XS generation demonstrated input sequence invariance. Results of comparing predicted XS data with reference solutions were found sufficiently close considering the reduction of computation time offered by ANN. Mean relative difference (MRD) for all output XS parameters was found below 0.7%, while HFF MRD was found higher compared to XS values, in some cases slightly exceeding 1%, mostly near guide tube locations. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

Measurement of a radial flow profile with eddy current flow meters and deep neural networks

Eddy current flow meters (ECFMs) measure flows of conductive fluids. Recent interest in ECFMs has increased due to applications in advanced nuclear reactors. ECFMs are well suited for such applications, as they can provide non-invasive measurements of flow in fluids that are often difficult to measure. Traditionally, ECFMs are operated using an alternating current at a single frequency, limiting ECFMs to measure average fluid velocities, blockages, or voids. Here, we expand the capabilities of ECFMs by measuring the fluid radial velocity profile of liquid mercury. To accomplish this, we made several ECFM sensitivity measurements at a range of frequencies. Different frequencies vary the electromagnetic skin depth of the device. By adjusting frequencies, we probed the fluid velocity at various radial locations and constructed a flow-velocity profile. The relationship between the ECFM measurements and velocity profile is nonlinear and requires solving an inverse problem. Using electromagnetic finite-element simulations to train a deep neural network (DNN), we created a model that provides a stable general relationship between the sensitivity measurements of an ECFM and the fluid velocity profile. Using ECFM measurements of liquid mercury, our DNN model calculates a flow profile that agrees well with computational fluid dynamics (CFD) simulations. This technique has potential to improve flow monitoring for optimization, safe operation of conductive fluid loops, and/or validating complex CFD models.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗

NASA NTRSDate not supplied

CERES FluxByCldTyp NB2BB Fluxes Improvement based on Deep Neural Network

The NASA Clouds and the Earth's Radiant Energy System (CERES) product provides over 20 years of accurately observed top-of-the-atmosphere (TOA) and surface flux data record for climate monitoring and diagnostic studies. The interaction between clouds and radiation interaction is a key factor that dominate climate feedbacks but is not well understood. To further advance our understanding of the cloud-radiation interaction, a new CERES FluxByCldTyp (FBCT) product has been developed that contains radiative fluxes by cloud-type, which can provide more stringent constraints when validating models and reveal more insight into the interactions between clouds and climate. For CERES partly cloudy and multiple cloud-type footprints, the FBCT product utilizes Moderate Resolution Imaging Spectroradiometer (MODIS) narrow-band (NB) imager channel radiances partitioned by cloud-type within a CERES footprint to estimate the cloud-type broadband fluxes. The MODIS multi-channel derived broadband fluxes were compared with the CERES observed footprint fluxes and were found to be within 1% and 2.5% for LW and SW, respectively, as well as being mostly free of cloud property dependencies. The FBCT all-sky and clear-sky monthly averaged fluxes were found to be consistent with the CERES SSF1deg product. This study takes advantage of recent progress in machine learning (ML) field by applying deep neural network algorithm to improve fluxes based on MODIS NB radiances. The preliminary study shows ML produce are an improvement over the current FBCT Edition 4 NB2BB algorithm. Furthermore, unlike Ed4 NB2BB, the new ML method convert NB radiances directly to broadband fluxes. For future Ed5, new NB radiances are proposed and used by ML to improve fluxes calculation. Prelimary results show significant LW improvement.

Moguo Sun↗

NASA NTRSDate not supplied

Improving Radiative Fluxes for the CERES FluxByCldTyp Data Product Using Deep Neural Network

The NASA Clouds and the Earth's Radiant Energy System (CERES) product provides over 20 years of accurately observed top-of-the-atmosphere (TOA) and surface flux data record for climate monitoring and diagnostic studies. The interaction between clouds and radiation interaction is a key factor that dominate climate feedbacks but is not well understood. To further advance our understanding of the cloud-radiation interaction, a new CERES FluxByCldTyp (FBCT) product has been developed that contains radiative fluxes by cloud-type, which can provide more stringent constraints when validating models and reveal more insight into the interactions between clouds and climate. For CERES partly cloudy and multiple cloud-type footprints, the FBCT product utilizes Moderate Resolution Imaging Spectroradiometer (MODIS) narrow-band (NB) imager channel radiances partitioned by cloud-type within a CERES footprint to estimate the cloud-type broadband fluxes. The MODIS multi-channel derived broadband fluxes were compared with the CERES observed footprint fluxes and were found to be within 1% and 2.5% for LW and SW, respectively, as well as being mostly free of cloud property dependencies. The FBCT all-sky and clear-sky monthly averaged fluxes were found to be consistent with the CERES SSF1deg product. This study takes advantage of recent progress in machine learning (ML) field by applying deep neural network algorithm to improve fluxes based on MODIS NB radiances. The preliminary study shows ML produce are an improvement over the current FBCT Edition 4 NB2BB algorithm. Furthermore, unlike Ed4 NB2BB, the new ML method convert NB radiances directly to broadband fluxes. For future Ed5, new NB radiances are proposed and used by ML to improve fluxes calculation. Preliminary results show significant LW improvement.

Sun, Moguo↗

DOE OSTI2020

DeepMerge: Classifying high-redshift merging galaxies with deep neural networks

In this work, we investigate and demonstrate the use of convolutional neural networks (CNNs) for the task of distinguishing between merging and non-merging galaxies in simulated images, and for the first time at high redshifts (i.e. $z=2$). We extract images of merging and non-merging galaxies from the Illustris-1 cosmological simulation and apply observational and experimental noise that mimics that from the Hubble Space Telescope; the data without noise form a "pristine" data set and that with noise form a "noisy" data set. The test set classification accuracy of the CNN is $79\%$ for pristine and $76\%$ for noisy. The CNN outperforms a Random Forest classifier, which was shown to be superior to conventional one- or two-dimensional statistical methods (Concentration, Asymmetry, the Gini, $M_{20}$ statistics etc.), which are commonly used when classifying merging galaxies. We also investigate the selection effects of the classifier with respect to merger state and star formation rate, finding no bias. Finally, we extract Grad-CAMs (Gradient-weighted Class Activation Mapping) from the results to further assess and interrogate the fidelity of the classification model.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Time‐Lapse Image Classification Using a Diffractive Neural Network

Diffractive deep neural networks (D 2 NNs), comprised of spatially engineered passive surfaces, collectively process optical input information at the speed of light propagation through a thin diffractive volume, without any external computing power. Diffractive networks were demonstrated to achieve all‐optical object classification and perform universal linear transformations. Herein, a “time‐lapse” image classification scheme using a diffractive network is demonstrated for the first time, significantly advancing its classification accuracy and generalization performance on complex input objects by using the lateral movements of the input objects and/or the diffractive network, relative to each other. In a different context, such relative movements of the objects and/or the camera are routinely being used for image super‐resolution applications; inspired by their success, a time‐lapse diffractive network is designed to benefit from the complementary information content created by controlled or random lateral shifts. The design space and performance limits of time‐lapse diffractive networks are numerically explored, revealing a blind testing accuracy of 62.03% on the optical classification of objects from the CIFAR‐10 dataset. This constitutes the highest inference accuracy achieved so far using a single diffractive network on the CIFAR‐10 dataset. Time‐lapse diffractive networks will be broadly useful for the spatiotemporal analysis of input signals using all‐optical processors.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Long-term missing value imputation for time series data using deep neural networks

We present an approach that uses a deep learning model, in particular, a MultiLayer Perceptron, for estimating the missing values of a variable in multivariate time series data. We focus on filling a long continuous gap (e.g., multiple months of missing daily observations) rather than on individual randomly missing observations. Our proposed gap filling algorithm uses an automated method for determining the optimal MLP model architecture, thus allowing for optimal prediction performance for the given time series. We tested our approach by filling gaps of various lengths (three months to three years) in three environmental datasets with different time series characteristics, namely daily groundwater levels, daily soil moisture, and hourly Net Ecosystem Exchange. We compared the accuracy of the gap-filled values obtained with our approach to the widely used R-based time series gap filling methods ImputeTS and mtsdi. The results indicate that using an MLP for filling a large gap leads to better results, especially when the data behave nonlinearly. Thus, our approach enables the use of datasets that have a large gap in one variable, which is common in many long-term environmental monitoring observations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Deep neural networks in real-time coherent diffraction imaging

Abstract not provided.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

Deep neural network uncertainty quantification for LArTPC reconstruction

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. Here we assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.

47 OTHER INSTRUMENTATION↗

NASA NTRS2017

Multi-Objective Reinforcement Learning-Based Deep Neural Networks for Cognitive Space Communications

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

space archtiecture↗

NASA NTRS2017

Multi-Objective Reinforcement Learning-based Deep Neural Networks for Cognitive Space Communications

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

space archtiecture↗

DOE OSTI2022

Deep Neural Network Algorithm for CMC Microstructure Characterization and Variability Quantification

Microstructure characterization and variability quantification are crucial for understanding ceramic matrix composites (CMCs) mechanical behavior and deformation mechanisms across length scales. Traditionally, analyses of the micrographs obtained from microscopy are labor-intensive. However, with the vast improvement in computer vision (CV) and deep learning (DL), an automated algorithm can be designed to extract essential microstructure variability from micrographs which can then be used to construct a statistically representative volume element (SRVE). The DL-based algorithm spans the taxonomy of microstructure analyses, including semantic segmentation of microstructure constituents, secondary phases, matrix/fiber interface, and defects, and quantifying the microstructure variability in terms of probability distributions. In this work, C/SiNC and SiC/SiNC CMCs microstructures are semantically segmented through a deep convolutional neural network, followed by variability quantification through the implementation of a fully connected regression layer, hence forming a deep regression network. The deep regression network operates in a feedforward regime, in which the neuron output signal traverses through the network in a unidirectional manner. The weight tensor associated with each layer is updated through a backpropagation stochastic gradient descent approach. The input gray-scale image obtained through in-house scanning electron microscope and confocal microscope micrographs is augmented through affine transformations to increase the training set size, which is then processed through four strided convolutional layers. This compresses the image resolution by half at each layer while increasing the image depth by applying different filters (image encoding). The class activation maps (CAMs) corresponding to the applied filters highlight the key architectural features and assist with the semantic segmentation of the microstructure.

Hamza, Mohamed H.↗

DOE OSTI2022

Deep neural network enabled corrective source term approach to hybrid analysis and modeling

Not Available

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Newton versus the machine: solving the chaotic three-body problem using deep neural networks

ABSTRACT Since its formulation by Sir Isaac Newton, the problem of solving the equations of motion for three bodies under their own gravitational force has remained practically unsolved. Currently, the solution for a given initialization can only be found by performing laborious iterative calculations that have unpredictable and potentially infinite computational cost, due to the system’s chaotic nature. We show that an ensemble of converged solutions for the planar chaotic three-body problem obtained using an arbitrarily precise numerical integrator can be used to train a deep artificial neural network (ANN) that, over a bounded time interval, provides accurate solutions at a fixed computational cost and up to 100 million times faster than the numerical integrator. In addition, we demonstrate the importance of training an ANN using converged solutions from an arbitrary precise integrator, relative to solutions computed by a conventional fixed precision integrator, which can introduce errors in the training data, due to numerical round-off and time discretization, that are learned by the ANN. Our results provide evidence that, for computationally challenging regions of phase space, a trained ANN can replace existing numerical solvers, enabling fast and scalable simulations of many-body systems to shed light on outstanding phenomena such as the formation of black hole binary systems or the origin of the core collapse in dense star clusters.

Breen, Philip G.↗

DOE OSTI2023

Recurrent Convolutional Deep Neural Networks for Modeling Time-Resolved Wildfire Spread Behavior

The increasing incidence and severity of wildfires underscores the necessity of accurately predicting their behavior. While high-fidelity models derived from first principles offer physical accuracy, they are too computationally expensive for use in real-time fire response. Low-fidelity models sacrifice some physical accuracy and generalizability via the integration of empirical measurements, but enable real-time simulations for operational use in fire response. Machine learning techniques have demonstrated the ability to bridge these objectives by learning first-principles physics while achieving computational speedups. While deep learning approaches have demonstrated the ability to predict wildfire propagation over large time periods, time-resolved fire-spread predictions are needed for active fire management. Here, in this work, we evaluate the ability of deep learning approaches in accurately modeling the time-resolved dynamics of wildfires. We use an autoregressive process in which a convolutional recurrent deep learning model makes predictions that propagate a wildfire over 15 min increments. We apply the model to four simulated datasets of increasing complexity, containing both field fires with homogeneous fuel distribution as well as real-world topologies sampled from the California region of the United States. We show that even after 100 autoregressive predictions representing more than 24 h of simulated fire spread, the resulting models generate stable and realistic propagation dynamics, achieving a Jaccard score between 0.89 and 0.94 when predicting the resulting fire scar. The inference time of the deep learning models are examined and compared, and directions for future work are discussed.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

DeepMerge: Studying Distant Merging Galaxies with Deep Neural Networks

The hierarchical merging of galaxies is a probe of the cosmos to test the canonical ?CDM cosmology paradigm. A particularly interesting period is "cosmic high noon" at redshifts z? 2?3, during which star formation rates were the highest, and significant amounts of stellar mass assembled into galaxy-scale bodies. Detecting galaxy mergers in observations by conventional automated methods (which use extracted parameters of galaxy structure - asymmetry, clumpiness, concentration etc.) or by visual inspection has proven to be quite time-consuming and prone to errors. Convolutional Neural Network (CNNs) are a primary representative of deep learning algorithms which are used in computer vision tasks by training to detect features in images. CNNs were already used for classification of low-redshift merging galaxies [1, 2]. Here we will use CNN to to learn directly from images (without the need to extract morphology parameters) of distant merging galaxies in order to distinguish between merging and non-merging objects.

79 ASTRONOMY AND ASTROPHYSICS↗