Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,459 records · Page 81
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Evolving Larger Convolutional Layer Kernel Sizes for a Settlement Detection Deep-Learner on Summit

Deep-learner hyper-parameters, such as kernel sizes, batch sizes, and learning rates, can significantly influence the quality of trained models. The state of the art for finding optimal hyper-parameters generally uses a brute force, grid search approach, random search, or Bayesian-based optimization among other techniques. We applied an evolutionary algorithm to optimize kernel sizes for a convolutional neural network used to detect settlements in satellite imagery. Usually convolutional layer kernel sizes are small - typically one, three, or five - but we found that the system converged at, or near, kernel sizes of nine for the last convolutional layer, and that this occurred for multiple runs using two different datasets. Moreover, the larger kernel sizes had fewer false positives than the 3x3 kernel sizes found as optimal via a brute force uniform grid search. This suggests that this large kernel size may be leveraging patterns found in larger areal features in the source imagery, and that this may be generalized as possible guidance for similar remote sensing deep-learning tasks.

Coletti, Mark↗
DOE OSTI2022

Optimal Coordination of Distributed Energy Resources Using Deep Deterministic Policy Gradient

Recent studies showed that reinforcement learning (RL) is a promising approach for coordination and control of distributed energy resources (DER) under uncertainties. Many existing RL approaches, including Q-learning and approximate dynamic programming, are based on lookup table methods, which become inefficient when the problem size is large and infeasible when continuous states and actions are involved. In addition, when modeling battery energy storage system (BESS), the loss of life is not reasonably considered into the decision-making process. This paper proposes an innovative deep RL method for DER coordination considering BESS degradation. The proposed deep RL is designed based on an adaptive actor-critic architecture and employs an off-policy deterministic policy gradient method for determining the dispatch operation that minimizes the operation cost and BESS life loss. Case studies were performed to validate the proposed method and demonstrate the effects of incorporating degradation models into control design.

Das, Avijit↗
DOE OSTI2024

Entropy-Infused Deep Learning Loss Function for Capturing Extreme Values in Wind Power Forecasting

Extreme scenarios in wind power generation occur with higher frequency and larger magnitude in the recent years due to the ever-increasing extreme meteorological factors. Accurate forecasting of the occurrence of extreme values in wind power generation is of great concern to ensure reliable power system operation. Recently, deep learning models have surged in popularity for wind power forecasting, with the mean squared error (MSE) loss function being commonly used. However, the MSE loss function, being sensitive to extreme values, disproportionately penalizes larger errors, cannot adequately capture the extreme values present in wind energy data, and novel loss functions have seldom been tailored for wind power forecasting. To this end, in this paper, we introduce a novel loss function specifically crafted to capture extreme values in wind power forecasting. The experimental results with four fundamental deep learning methods on open source wind power dataset validate that the new loss function is efficient and superior in all cases compared to MSE in capturing extreme values while maintaining forecasting performance.

17 WIND ENERGY↗
DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗
DOE OSTI2019

Deep Transfer Learning With Field-Based Measurements for Large Area Classification

Classification of Electro-Optical (EO) datasets using Deep Neural Networks (DNNs) has lead to high-performance supervised learning algorithms. However, DNNs require a large amount of labeled data for training which often have limited availability in ecological studies. Up-to-date classification of vegetation in sensitive Arctic ecosystems continue to be a challenge. In an ecosystem undergoing rapid change, capturing the dynamics of vegetation requires the existing maps of vegetation (such as Alaska Existing Vegetation Type (AKEVT), circa 2000) to be updated based on frequent field based observation of vegetation. A method is needed to transfer the knowledge gained from field-collected observations to a larger area using DNNs. Transfer learning is a machine learning technique where a model trained on one task is re-purposed on another similar task. This paper seeks to train DNNs using field-collected observations and apply transfer learning to apply the knowledge gained to update existing vegetation map at larger scale. We test two DNN methods, (1) a deep multilayer perceptron (MLP) model and (2) siamese MLP network that uses a structure to rank similarity between inputs and can be used for training datasets with few samples and show good performance with limited datasets (e.g. few-shot learning). The results show ∼90% accuracy (using the field observations for evaluation) when transfer learning is applied to a siamese network, compared to ∼45% accurate when a MLP is trained on the AKEVT and evaluated on the field observations. The approach show promise for improving and update the existing vegetation maps over large areas using limited field-based observations.

Langford, Zach↗
DOE OSTI2021

Revisit the Scalability of Deep Auto-Regressive Models for Graph Generation

As a new promising approach to graph generations, deep auto-regressive graph generation has drawn increasing attention. It however has been commonly deemed as hard to scale up to work with large graphs. In existing studies, it is perceived that the consideration of the full non-local graph dependences is indispensable for this approach to work, which entails the needs for keeping the entire graph’s info in memory and hence the perceived “inherent” scalability limitation of the approach. This paper revisits the common perception. It proposes three ways to relax the dependences and conducts a series of empirical measurements. It concludes that the perceived “inherent” scalability limitation is a misperception; with the right design and implementation, deep auto-regressive graph generation can be applied to graphs much larger than the device memory. The rectified perception removes a fundamental barrier for this approach to meet practical needs.

Yang, Shuai↗
DOE OSTI2020

DeepGrid: Robust Deep Reinforcement Learning-based Contingency Management

Increasing uncertainty raised by the integration of renewable energy resources requires an enormous number of simulations to be carried out for the security assessment of the power grid. However, it is challenging to assess the steady-state and dynamic security indices for different system contingency events by doing an exhaustive analysis in real-time due to the computational and communication constraints. One promising solution is using data-driven techniques along with the system models to train an intelligent contingency management framework to better handle the contingencies in real-time. Nevertheless, implementing a data-driven technique to obtain the best remedial actions necessitates to account for the effect of the measurement noise on the performance of the contingency management. To tackle these challenges, we leverage a robust deep reinforcement learning (DRL) algorithm called Double Deep Q-Network (DDQN) to design a recommender system capable of prescribing optimal control actions with the help of the real-time digital simulator (RTDS). The use of RTDS system in combination with the advanced DRL algorithm allows to explore a wide variety of system contingencies in order to derive better remedial actions. The performance of the proposed algorithm is evaluated in IEEE 9-bus system under different loading conditions, and different network configurations in presence of noisy measurements.

Ghasemkhani, Amir↗
DOE OSTI2023

DeepGraphONet: A Deep Graph Operator Network to Learn and Zero-Shot Transfer the Dynamic Response of Networked Systems

This article develops a deep graph operator network (DeepGraphONet) framework that learns to approximate the dynamics of a complex system (e.g., the power grid or traffic) with an underlying subgraph structure. Here, we build our DeepGraphONet by fusing the ability of graph neural networks to exploit spatially correlated graph information and deep operator networks to approximate the solution operator of dynamical systems. The resulting DeepGraphONet can then predict the dynamics within a given short/medium-term time horizon by observing a finite history of the graph state information. Furthermore, we design our DeepGraphONet to be resolution independent. That is, we do not require the finite history to be collected at the exact/same resolution. In addition, to disseminate the results from a trained DeepGraphONet, we design a zero-shot learning strategy that enables using it on a different subgraph. Finally, empirical results on the transient stability prediction problem of power grids and traffic flow forecasting problem of a vehicular system illustrate the effectiveness of the proposed DeepGraphONet.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

Intelligent Buildings of the Future: Cyberaware, Deep Learning Powered, and Human Interacting

This study discuss, intelligent buildings are quickly becoming cohesive and integral entities of cyber-physical ecosystems. Modern buildings adapt to internal and external elements and thrive on ever increasing data sources such as ubiquitous smart devices and sensors while mimicking various approaches previously known in software, hardware, and bio inspired systems. This article provides an overview of intelligent buildings of the future from a range of perspectives. First, a brief overview of prospects of U.S. and world energy consumption, concepts of smart buildings as integral entities of smart grids is presented. This is followed by a discussion of the role of connectivity and interoperability as well as current issues and solutions in the areas of security, resilience, and humans in realm of intelligent buildings. The article dives into detail about the intelligence aspect of intelligent buildings of the future. The learning and predictive Artificial Intelligence (AI) techniques behind many of the concepts of intelligent buildings are reviewed, including the latest techniques such as deep learning. As a concrete application of AI in the realm of intelligent buildings, prediction of energy consumption/load/demand, which is a much researched application of AI in the realm of intelligent buildings, is elaborated. The discussion is focused on most recent advancements of AI based demand prediction and the overview consists of existing research and a case study conducted by the authors for deep learning based demand prediction. Then, the article discusses the role of humans in intelligent buildings. Different types of human-building interactions are identified. Further, the article presents how each type of human-building interaction can benefit the overall control of intelligent buildings. The section is concluded by elaborating a case study conducted by the authors to achieve effective and efficient human-building interaction. The article is concluded with compilation of future insights based on latest technological advancements in U.S. industry and government.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

High-Performance Deep Learning Toolbox for Genome-Scale Prediction of Protein Structure and Function

Computational biology is one of many scientific disciplines ripe for innovation and acceleration with the advent of high-performance computing (HPC). In recent years, the field of machine learning has also seen significant benefits from adopting HPC practices. In this work, we present a novel HPC pipeline that incorporates various machine-learning approaches for structure-based functional annotation of proteins on the scale of whole genomes. Our pipeline makes extensive use of deep learning and provides computational insights into best practices for training advanced deep-learning models for high-throughput data such as proteomics data. We showcase methodologies our pipeline currently supports and detail future tasks for our pipeline to envelop, including large-scale sequence comparison using SAdLSA and prediction of protein tertiary structures using AlphaFold2.

Gao, Mu↗
DOE OSTI2023

Geodesic-based relaxation for deep canonical correlation analysis

Deep canonical correlation analysis (DCCA) is often applied to paired data samples from diverse sources to extract meaningful common information. However, when the data sources are heterogeneous, some of the useful information may be complementary but not exactly common. In spite of this fact, existing techniques learn maximally correlated representations from multiple views and are formulated so that they aim to yield identical latent subspaces for each view. This approach is sub-optimal in estimating the true signal subspaces for heterogeneous data sources. We propose a residual relaxation for deep canonical correlation analysis (RDCCA) based on a subspace distance metric, which generalizes the existing problem formulation and extracts representations that are better estimates of the actual, non-identical subspaces. We demonstrate that when using such a relaxation, the learned representations are closer to the true ones and that RDCCA outperforms CCA and DCCA in scenarios with heterogeneous data.

Kuschel, Maurice↗
DOE OSTI2018

Deep Convolutional Neural Networks for Distribution System Fault Classification

Faults happen very frequently in distribution systems. Identifying fault types and phases are of critical importance for outage management, fault location, and service restoration. However, this task becomes very challenging due to measurement scarcity in distribution systems. This paper is among the first few that applies deep learning techniques in distribution system fault classification. Specifically, a sequential Convolutional Neural Network(CNN)-based classifier is developed to identify fault buses and phases. The input to the CNN is the steady-state voltage and current data measured at substations. The fault identification is modeled as a multi-label classification problem. Training data under various fault scenarios are obtained in OpenDSS and Gaussian noises are added to mimic measurement errors. A case study in IEEE 13-feeder test system is conducted with single and multiple bus faults scenarios. Numerical results demonstrate the high accuracy and fast computation of the proposed deep CNN-based fault classification.

Tian, Guanyu↗
DOE OSTI2021

Adaptive Load Shedding for Grid Emergency Control via Deep Reinforcement Learning

Emergency control, typically such as under-voltage load shedding (UVLS), is broadly used to grapple with low voltage and voltage instability issues in real-world power systems under contingencies. However, existing emergency control schemes are rule-based and cannot be adaptively applied to uncertain and floating operating conditions. Here, we propose an adaptive UVLS algorithm for emergency control via deep reinforcement learning (DRL) and expert systems. We first construct dynamic components for picturing the power system operation as the environment. The transient voltage recovery criteria, which poses time-varying requirements to UVLS, is integrated into the states and reward function to advise the learning of deep neural networks. The proposed method has no tuning issue of coefficients in reward functions, and this issue was regarded as a deficiency in the existing DRL-based algorithms. Case studies illustrate that the proposed method outperforms the traditional UVLS relay in both the timeliness and efficacy for emergency control.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Optimal Coordination of Electric Vehicles for Grid Services using Deep Reinforcement Learning

Recent research has shown the effectiveness of reinforcement learning (RL) in coordinating electric vehicles (EVs) with vehicle-to-grid capabilities for grid services. However, many of these studies rely on lookup table and deep Q-network techniques, which can be impractical when dealing with continuous states and actions. In addition, existing RL designs inadequately account for battery aging effects, EV user satisfaction, uncertain departure and arrival time, and trip distance, which may compromise effective coordination. This paper aims to bridge these gaps by developing an innovative deep deterministic policy gradient-based RL framework for optimal coordination of EVs. Case studies were carried out using a test system with 100 EVs, and numerical analysis results showed that the proposed RL framework can effectively coordinate EVs to maximize economic benefits and user satisfaction while ensuring the expected battery lifespan.

Das, Avijit↗
DOE OSTI2018

167-PFlops Deep Learning for Electron Microscopy: From Learning Physics to Atomic Manipulation

An artificial intelligence system called MENNDL, which used 25,200 Nvidia Volta GPUs on Oak Ridge National Laboratory’s Summit machine, automatically designed an optimal deep learning network in order to extract structural information from raw atomic-resolution microscopy data. In a few hours, MENNDL creates and evaluates millions of networks using a scalable, parallel, asynchronous genetic algorithm augmented with a support vector machine to automatically find a superior deep learning network topology and hyper-parameter set than a human expert can find in months. For the application of electron microscopy, the system furthers the goal of improving our understanding of the electron-beam-matter interactions and real-time image-based feedback, which enables a huge step beyond human capacity toward nanofabricating materials automatically. Finally, MENNDL has been scaled to the 4,200 available nodes of Summit achieving a measured 152.5 PFlops, with an estimated sustained performance of 167 PFlops when the entire machine is available.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

The Second Skin: A Wearable Sensor Suite that Enables Real-Time Human Biomechanics Tracking Through Deep Learning

Objective: Real-time determination of human kinematics and kinetics could advance biomechanics research and enable valuable applications of biofeedback and generalizable exoskeleton control. Here, this work aims to investigate a taskindependent, user-independent method for obtaining precise realtime joint state estimation across lower-body joints during a wide variety of tasks. Methods: We developed a generalizable sensing approach using a suit comprised of inertial measurement units (IMUs) and pressure insoles. With the suit, we collected a dataset of 33 tasks commonly performed during construction and hazardous waste cleanup (N = 10). We then trained deep learning user-independent, task-agnostic models to estimate joint lowerbody kinematics and dynamics using only worn sensor data. We likewise computed joint kinematics and dynamics analytically from sensor data to serve as a comparison tool for model results. Results: Our models achieved overall angle estimation root-meansquared-errors (RMSE) of 6.56±.92°, 8.60±1.01°, 7.58±.89°, and 6.00±.73° compared to 13.9±.1.3°, 15.31±1.0°, 10.76±.70°, and 7.56±.48° via analytical methods at the lower back, hip, knee, and ankle, respectively. Likewise, our models achieved overall normalized moment estimation RMSEs of .207±.069 Nm/kg, .242±.044 Nm/kg, .202±.038 Nm/kg, and .193±.034 Nm/kg compared to .306±.036 Nm/kg, .407±.021 Nm/kg, 1.18 ±.022 Nm/kg, and 1.73±.071 Nm/kg via analytical methods at the lower back, hip, knee, and ankle, respectively. Conclusion: These results are comparable to other state-of-the-art wearable sensing systems, establishing deep learning as a viable sensing approach that generalizes to new users and tasks. Significance: This work shows promise for enabling accurate real-world biomechanical data collection and enhancement of biofeedback systems and wearable robot control.

Casey, Ryan T. F. [Georgia Institute of Technology↗
DOE OSTI2024

Deep Koopman Neural Network for Analyzing High-Energy-Density Simulations of Electrical Wire Explosions

Megaampere-scale electrical wire experiments (EWEs) provide a platform for studying magnetohydrodynamic (MHD) instability growth in magneto-inertial fusion (MIF) devices. Even when nonlinear simulations of these experiments can digitally reproduce much of the experimentally observed instability growth, interpreting the results and understanding mode growth and evolution can be non-trivial. As a first step toward providing better interpretation of these simulation features, this work investigates the use of a deep neural network that uses Koopman operator theory to analyze the dynamics of pulsed-power-driven explosions of EWEs. This deep neural network is trained on 1-D resistive MHD simulations of EWEs. This neural network learns to transform the nonlinear data into a lower-dimensional representation where the time dynamics are linear. Layers of this neural network are shown to learn features of the simulations, including the locations of shock waves and different physical regimes of the simulation. Using the learned features, the network can compress a time state of the simulation consisting of 5120 data point into a 36-parameter lower-dimensional latent space embedding. Furthermore, these embeddings are shown to be clustered in the latent space by initial radius and time state.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Learning and Fast Adaptation for Grid Emergency Control via Deep Meta Reinforcement Learning

As power systems are undergoing a significant transformation with more uncertainties, less inertia and closer to operation limits, there is increasing risk of large outages. Thus, there is an imperative need to enhance grid emergency control to maintain system reliability and security. Towards this end, great progress has been made in developing deep reinforcement learning (DRL) based grid control solutions in recent years. However, existing DRL-based solutions have two main limitations: 1) they cannot handle well with a wide range of grid operation conditions, system parameters, and contingencies; 2) they generally lack the ability to fast adapt to new grid operation conditions, system parameters, and contingencies, limiting their applicability for real-world applications. Here, in this paper, we mitigate these limitations by developing a novel deep meta-reinforcement learning (DMRL) algorithm. The DMRL combines the meta strategy optimization together with DRL, and trains policies modulated by a latent space that can quickly adapt to new scenarios. We test the developed DMRL algorithm on the IEEE 300-bus system. We demonstrate fast adaptation of the meta-trained DRL polices with latent variables to new operating conditions and scenarios using the proposed method, which achieves superior performance compared to the state-of-the-art DRL and model predictive control (MPC) methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗