Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Frost prediction using machine learning and deep neural network models

This study describes accurate, computationally efficient models that can be implemented for practical use in predicting frost events for point-scale agricultural applications. Frost damage in agriculture is a costly burden to farmers and global food security alike. Timely prediction of frost events is important to reduce the cost of agricultural frost damage and traditional numerical weather forecasts are often inaccurate at the field-scale in complex terrain. In this paper, we developed machine learning (ML) algorithms for the prediction of such frost events near Alcalde, NM at the point-scale. ML algorithms investigated include deep neural network, convolution neural networks, and random forest models at lead-times of 6–48 h. Our results show promising accuracy (6-h prediction RMSE = 1.53–1.72°C) for use in frost and minimum temperature prediction applications. Seasonal differences in model predictions resulted in a slight negative bias during Spring and Summer months and a positive bias in Fall and Winter months. Additionally, we tested the model transferability by continuing training and testing using data from sensors at a nearby farm. We calculated the feature importance of the random forest models and were able to determine which parameters provided the models with the most useful information for predictions. We determined that soil temperature is a key parameter in longer term predictions (>24 h), while other temperature related parameters provide the majority of information for shorter term predictions. The model error compared favorable to previous ML based frost studies and outperformed the physically based High Resolution Rapid Refresh forecasting system making our ML-models attractive for deployment toward real-time monitoring of frost events and damage at commercial farming operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Theoretical Prediction of Thermal Expansion Anisotropy for Y 2 Si 2 O 7 Environmental Barrier Coatings Using a Deep Neural Network Potential and Comparison to Experiment

Environmental barrier coatings (EBCs) are an enabling technology for silicon carbide (SiC)-based ceramic matrix composites (CMCs) in extreme environments such as gas turbine engines. However, the development of new coating systems is hindered by the large design space and difficulty in predicting the properties for these materials. Density Functional Theory (DFT) has successfully been used to model and predict some thermodynamic and thermo-mechanical properties of high-temperature ceramics for EBCs, although these calculations are challenging due to their high computational costs. In this work, we use machine learning to train a deep neural network potential (DNP) for Y 2 Si 2 O 7 , which is then applied to calculate the thermodynamic and thermo-mechanical properties at near-DFT accuracy much faster and using less computational resources than DFT. We use this DNP to predict the phonon-based thermodynamic properties of Y 2 Si 2 O 7 with good agreement to DFT and experiments. We also utilize the DNP to calculate the anisotropic, lattice direction-dependent coefficients of thermal expansion (CTEs) for Y 2 Si 2 O 7 . Molecular dynamics trajectories using the DNP correctly demonstrate the accurate prediction of the anisotropy of the CTE in good agreement with the diffraction experiments. In the future, this DNP could be applied to accelerate additional property calculations for Y 2 Si 2 O 7 compared to DFT or experiments.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A Medium‐Sized Paleo‐Tsunami Reconstruction by a Deep Neural Network Processing Sedimentary Deposits

Abstract Reconstructing the magnitude and recurrence time of tsunamis, one of the most destructive and unpredictable natural hazards impacting coastal communities, is essential. While major tsunamis are the most studied due to their disastrous impact, small/medium tsunamis (SMTs) are much more frequent and can still significantly impact the coast. Therefore, SMTs potentially provide an extensive archive of information preserved in the geological record. Analyzing the deposits of small/medium paleo‐tsunamis (SMPTs) opens a window into when their direct observation was unavailable. However, deposits of SMPTs are often degraded, traditional sediment deposition inversion models might fail. Recent research has shown that Deep Neural Networks (DNN) can effectively reconstruct the flow conditions of major tsunamis from their deposits. We evaluate the effectiveness of this approach in reconstructing the characteristics of a recent medium size tsunami (2006 Java) and of a medium paleo‐tsunami (1929 Grand Banks). We successfully reconstruct the flow characteristics of the 2006 Java event and show that an inversion of comparable quality is possible for the 1929 Grand Banks tsunami, despite greater uncertainties due to the deposit degradation. Our research shows that Machine Learning has the potential to unseal the meaning of data of thousands SMPTs.

Batubo, P.↗
DOE OSTI2022

Enhanced deep neural networks with transfer learning for distribution LMP considering load and PV uncertainties

As the flexibility of generation and demand increases in distribution systems, the residential loads are emerging as a promising means to participate in demand response and the transactive energy market. Market pricing is an instrumental mechanism for the distribution system operator to exploit the full potential of the flexible resources. The distribution locational marginal price (DLMP) can be used to guide the residential load consumption. This type of market signal helps the distribution system operator to optimize the scheduling of all resources while satisfying related network constraints through a day-ahead market. However, solving the optimization problem for large-scale systems can be computationally expensive. To address the scalability and practicability limitations of the DLMP framework, a learning-based approach is proposed in this paper to complement the day-ahead distribution market framework. Here, the proposed approach combines long short-term memory and transfer learning to develop deep neural network that can capture the spatial–temporal correlation of the input data. The model can determine the optimal DLMP for each node in a distribution system without the system parameters required to formulate the optimization problem. Testing results on IEEE 33-bus and 123-bus systems show that the proposed approach can generate a comparable DLMP against the optimization solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Optimization of the deep neural network parameters for generating homogenized fuel assembly data for nodal codes

Homogenized fuel assembly (FA) data is a typical input data for nodal codes. Generating that data, however, could be time-consuming. One of promising ways to mitigate the computational burden of generating macroscopic cross-sections is to use trained artificial neural network (ANN) models for predicting nuclear data. However, there is a challenge to make the model support variable FA geometry. In this work, two most common types of FA were combined in one ANN model. Since there could be multiple ways of converting 2-dimensional FA data into 1-dimensional input vector for ANN, three different approaches of data flattening were evaluated. The input parameters included each fuel pin enrichment, fuel temperature, moderator temperature and boron concentration. The output parameters were 2-group macroscopic cross-sections (XS) and pin power distribution (HFF). A fully connected deep neural network (DNN) model was trained and tested using pre-generated data obtained with lattice physics code STREAM. The results of this study showed no statistically significant difference in the accuracy of XS and HFF generation for all 3 tested input vector orders. This means that fully connected DNN for XS generation demonstrated input sequence invariance. Results of comparing predicted XS data with reference solutions were found sufficiently close considering the reduction of computation time offered by ANN. Mean relative difference (MRD) for all output XS parameters was found below 0.7%, while HFF MRD was found higher compared to XS values, in some cases slightly exceeding 1%, mostly near guide tube locations. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Measurement of a radial flow profile with eddy current flow meters and deep neural networks

Eddy current flow meters (ECFMs) measure flows of conductive fluids. Recent interest in ECFMs has increased due to applications in advanced nuclear reactors. ECFMs are well suited for such applications, as they can provide non-invasive measurements of flow in fluids that are often difficult to measure. Traditionally, ECFMs are operated using an alternating current at a single frequency, limiting ECFMs to measure average fluid velocities, blockages, or voids. Here, we expand the capabilities of ECFMs by measuring the fluid radial velocity profile of liquid mercury. To accomplish this, we made several ECFM sensitivity measurements at a range of frequencies. Different frequencies vary the electromagnetic skin depth of the device. By adjusting frequencies, we probed the fluid velocity at various radial locations and constructed a flow-velocity profile. The relationship between the ECFM measurements and velocity profile is nonlinear and requires solving an inverse problem. Using electromagnetic finite-element simulations to train a deep neural network (DNN), we created a model that provides a stable general relationship between the sensitivity measurements of an ECFM and the fluid velocity profile. Using ECFM measurements of liquid mercury, our DNN model calculates a flow profile that agrees well with computational fluid dynamics (CFD) simulations. This technique has potential to improve flow monitoring for optimization, safe operation of conductive fluid loops, and/or validating complex CFD models.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗
DOE OSTI2020

DeepMerge: Classifying high-redshift merging galaxies with deep neural networks

In this work, we investigate and demonstrate the use of convolutional neural networks (CNNs) for the task of distinguishing between merging and non-merging galaxies in simulated images, and for the first time at high redshifts (i.e. $z=2$). We extract images of merging and non-merging galaxies from the Illustris-1 cosmological simulation and apply observational and experimental noise that mimics that from the Hubble Space Telescope; the data without noise form a "pristine" data set and that with noise form a "noisy" data set. The test set classification accuracy of the CNN is $79\%$ for pristine and $76\%$ for noisy. The CNN outperforms a Random Forest classifier, which was shown to be superior to conventional one- or two-dimensional statistical methods (Concentration, Asymmetry, the Gini, $M_{20}$ statistics etc.), which are commonly used when classifying merging galaxies. We also investigate the selection effects of the classifier with respect to merger state and star formation rate, finding no bias. Finally, we extract Grad-CAMs (Gradient-weighted Class Activation Mapping) from the results to further assess and interrogate the fidelity of the classification model.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Time‐Lapse Image Classification Using a Diffractive Neural Network

Diffractive deep neural networks (D 2 NNs), comprised of spatially engineered passive surfaces, collectively process optical input information at the speed of light propagation through a thin diffractive volume, without any external computing power. Diffractive networks were demonstrated to achieve all‐optical object classification and perform universal linear transformations. Herein, a “time‐lapse” image classification scheme using a diffractive network is demonstrated for the first time, significantly advancing its classification accuracy and generalization performance on complex input objects by using the lateral movements of the input objects and/or the diffractive network, relative to each other. In a different context, such relative movements of the objects and/or the camera are routinely being used for image super‐resolution applications; inspired by their success, a time‐lapse diffractive network is designed to benefit from the complementary information content created by controlled or random lateral shifts. The design space and performance limits of time‐lapse diffractive networks are numerically explored, revealing a blind testing accuracy of 62.03% on the optical classification of objects from the CIFAR‐10 dataset. This constitutes the highest inference accuracy achieved so far using a single diffractive network on the CIFAR‐10 dataset. Time‐lapse diffractive networks will be broadly useful for the spatiotemporal analysis of input signals using all‐optical processors.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Long-term missing value imputation for time series data using deep neural networks

We present an approach that uses a deep learning model, in particular, a MultiLayer Perceptron, for estimating the missing values of a variable in multivariate time series data. We focus on filling a long continuous gap (e.g., multiple months of missing daily observations) rather than on individual randomly missing observations. Our proposed gap filling algorithm uses an automated method for determining the optimal MLP model architecture, thus allowing for optimal prediction performance for the given time series. We tested our approach by filling gaps of various lengths (three months to three years) in three environmental datasets with different time series characteristics, namely daily groundwater levels, daily soil moisture, and hourly Net Ecosystem Exchange. We compared the accuracy of the gap-filled values obtained with our approach to the widely used R-based time series gap filling methods ImputeTS and mtsdi. The results indicate that using an MLP for filling a large gap leads to better results, especially when the data behave nonlinearly. Thus, our approach enables the use of datasets that have a large gap in one variable, which is common in many long-term environmental monitoring observations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Deep neural network uncertainty quantification for LArTPC reconstruction

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. Here we assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Deep Neural Network Algorithm for CMC Microstructure Characterization and Variability Quantification

Microstructure characterization and variability quantification are crucial for understanding ceramic matrix composites (CMCs) mechanical behavior and deformation mechanisms across length scales. Traditionally, analyses of the micrographs obtained from microscopy are labor-intensive. However, with the vast improvement in computer vision (CV) and deep learning (DL), an automated algorithm can be designed to extract essential microstructure variability from micrographs which can then be used to construct a statistically representative volume element (SRVE). The DL-based algorithm spans the taxonomy of microstructure analyses, including semantic segmentation of microstructure constituents, secondary phases, matrix/fiber interface, and defects, and quantifying the microstructure variability in terms of probability distributions. In this work, C/SiNC and SiC/SiNC CMCs microstructures are semantically segmented through a deep convolutional neural network, followed by variability quantification through the implementation of a fully connected regression layer, hence forming a deep regression network. The deep regression network operates in a feedforward regime, in which the neuron output signal traverses through the network in a unidirectional manner. The weight tensor associated with each layer is updated through a backpropagation stochastic gradient descent approach. The input gray-scale image obtained through in-house scanning electron microscope and confocal microscope micrographs is augmented through affine transformations to increase the training set size, which is then processed through four strided convolutional layers. This compresses the image resolution by half at each layer while increasing the image depth by applying different filters (image encoding). The class activation maps (CAMs) corresponding to the applied filters highlight the key architectural features and assist with the semantic segmentation of the microstructure.

Hamza, Mohamed H.↗
DOE OSTI2020

Newton versus the machine: solving the chaotic three-body problem using deep neural networks

ABSTRACT Since its formulation by Sir Isaac Newton, the problem of solving the equations of motion for three bodies under their own gravitational force has remained practically unsolved. Currently, the solution for a given initialization can only be found by performing laborious iterative calculations that have unpredictable and potentially infinite computational cost, due to the system’s chaotic nature. We show that an ensemble of converged solutions for the planar chaotic three-body problem obtained using an arbitrarily precise numerical integrator can be used to train a deep artificial neural network (ANN) that, over a bounded time interval, provides accurate solutions at a fixed computational cost and up to 100 million times faster than the numerical integrator. In addition, we demonstrate the importance of training an ANN using converged solutions from an arbitrary precise integrator, relative to solutions computed by a conventional fixed precision integrator, which can introduce errors in the training data, due to numerical round-off and time discretization, that are learned by the ANN. Our results provide evidence that, for computationally challenging regions of phase space, a trained ANN can replace existing numerical solvers, enabling fast and scalable simulations of many-body systems to shed light on outstanding phenomena such as the formation of black hole binary systems or the origin of the core collapse in dense star clusters.

Breen, Philip G.↗
DOE OSTI2023

Recurrent Convolutional Deep Neural Networks for Modeling Time-Resolved Wildfire Spread Behavior

The increasing incidence and severity of wildfires underscores the necessity of accurately predicting their behavior. While high-fidelity models derived from first principles offer physical accuracy, they are too computationally expensive for use in real-time fire response. Low-fidelity models sacrifice some physical accuracy and generalizability via the integration of empirical measurements, but enable real-time simulations for operational use in fire response. Machine learning techniques have demonstrated the ability to bridge these objectives by learning first-principles physics while achieving computational speedups. While deep learning approaches have demonstrated the ability to predict wildfire propagation over large time periods, time-resolved fire-spread predictions are needed for active fire management. Here, in this work, we evaluate the ability of deep learning approaches in accurately modeling the time-resolved dynamics of wildfires. We use an autoregressive process in which a convolutional recurrent deep learning model makes predictions that propagate a wildfire over 15 min increments. We apply the model to four simulated datasets of increasing complexity, containing both field fires with homogeneous fuel distribution as well as real-world topologies sampled from the California region of the United States. We show that even after 100 autoregressive predictions representing more than 24 h of simulated fire spread, the resulting models generate stable and realistic propagation dynamics, achieving a Jaccard score between 0.89 and 0.94 when predicting the resulting fire scar. The inference time of the deep learning models are examined and compared, and directions for future work are discussed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

DeepMerge: Studying Distant Merging Galaxies with Deep Neural Networks

The hierarchical merging of galaxies is a probe of the cosmos to test the canonical ?CDM cosmology paradigm. A particularly interesting period is "cosmic high noon" at redshifts z? 2?3, during which star formation rates were the highest, and significant amounts of stellar mass assembled into galaxy-scale bodies. Detecting galaxy mergers in observations by conventional automated methods (which use extracted parameters of galaxy structure - asymmetry, clumpiness, concentration etc.) or by visual inspection has proven to be quite time-consuming and prone to errors. Convolutional Neural Network (CNNs) are a primary representative of deep learning algorithms which are used in computer vision tasks by training to detect features in images. CNNs were already used for classification of low-redshift merging galaxies [1, 2]. Here we will use CNN to to learn directly from images (without the need to extract morphology parameters) of distant merging galaxies in order to distinguish between merging and non-merging objects.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Novel Data Driven Noise Emulation Framework using Deep Neural Network for Generating Synthetic PMU Measurements

Sensors play a critical role in supporting day-to-day grid operations and they are essential to operator’s decision-making process. Furthermore, sensors and sensor behaviors need to be emulated with grid simulations to perform modeling studies and to design cutting edge power systems applications. Ensuring the accurate behavior of these applications requires accurate emulation of sensors and pertinent signals. However, most grid simulators and modeling tools assume either zero error scenarios or simplistic noise models that may not always correlate to real-world sensors. To address the above issue, this work presents an initial study on the noise characteristics of phasor measurement units (PMUs), along with models for recreating their unique noise signatures. The proposed methods (both analytical and machine-learning-based) provide a substantial increase in a sensor’s model fidelity, a feature that can be leveraged by an end-user application to yield more accurate system representations. The proposed methods were then applied to micro PMU data from the EPFL microgrid campus to extract sensor noise profiles. This data was used to train a deep learning model, which was tested to emulate the noise characteristics present in actual signals. Based on the observed results and the employed data-driven methodology, the proposed methods may be adapted to replicate the behavior of other grid sensors and power new applications capable of detecting sensor degradation and eventual device failures in near real-time.

PMU, noise emulation, synthetic measurements, deep↗