Results for “Deep learning” — Page 89

DOE OSTI2020

Neural-based time series forecasting of loss of coolant accidents in nuclear power plants

During the last few years, deep learning in neural networks has demonstrated impressive successes in the areas of computer vision, speech and image recognition, text generation, and many others. However, sensitive engineering areas such as nuclear engineering benefited less from these efficient techniques. In this work, deep learning expert systems are utilized to model and predict time series progression of a design-basis nuclear accident, featuring a loss of coolant accident. Two major findings are accomplished in this work. First, the ability to train expert systems with high accuracy, which could help nuclear power plant operators to figure out plant responses during the accident. Second, building fast, efficient, and accurate deep models to simulate nuclear phenomena, which could be valuable to nuclear computational science. In this work, large amount of time series data is obtained from simulation tools by simulating different conditions of the base-case/nominal accident scenario. Four critical outputs/responses are monitored during the accident (e.g. temperature, pressure, break flow rate, water level). Two approaches are adopted in this work. The first approach is to use feedforward deep neural networks (DNN) to fit all time steps and outputs in a single model. The second approach is to use long short-term memory (LSTM) to fit all time steps together for each reactor response separately. Both DNN and LSTM demonstrate very good performance in predicting the test and base-case scenarios, with accuracy as low as 92% and as high as 99%, where these test scenarios are unknown to the expert systems and are not included in the model training. In addition, both approaches demonstrate a significant reduction in computational costs, as the deep expert system is able to accurately predict the accident 100,000 times faster than the original simulation tool. Given sufficient data, the methodology adopted in this study demonstrates that DNN/LSTM expert systems can be used as a decision support system to model advanced time series phenomena within nuclear power plants with high accuracy and negligible computational costs.

42 ENGINEERING↗

NASA NTRS2022

Exploring Applications of Machine Learning for Wildfire Monitoring and Detection using Unmanned Aerial Vehicles

Wildfires are increasing in frequency and severity around the world, including the United States. The losses caused by wildfires could be mitigated if high-risk areas, hotspots, and flare-ups could be monitored continuously, such as through the use of Unmanned Aerial Vehicles (UAVs). This paper documents exploratory efforts using machine learning to determine efficient flight paths for UAVs and to detect wildfires using image classification. On path planning, three machine learning techniques—Genetic Algorithm, Simulated Annealing, and Dynamic Programming—were explored. Genetic Algorithm was found to be an effective approach for path planning for wildfire monitoring and surveillance by UAVs. For a scenario of 25 locations in a circular arrangement, the algorithm was able to return the optimal path. The accuracy and execution time was found to be sensitive to the algorithm hyperparameters selected, which was especially evident in scenarios with hundreds or thousands of locations. Simulated Annealing was also found to be an effective approach for UAV path planning, with a major benefit of avoiding getting trapped in local minima and being straightforward to implement. Like Genetic Algorithm, the performance of Simulated Annealing was also found to be sensitive to the algorithm hyperparameters selected. By comparison, Dynamic Programming guarantees optimality for any number of locations, but it was found to be less practical in terms of execution time for scenarios with more than about a couple dozen locations. On wildfire detection, image classification using deep learning with a convolutional neural network was explored. Transfer learning was found to be a useful technique to efficiently train deep learning models. Also, it was determined that GPU processing can increase training speed by an order of magnitude, which enables significantly faster development. For a validation test set of 500 images, there were only two false negatives and zero false positives. These results demonstrate that detecting wildfires in static cameras using machine learning is feasible and establish a baseline for using images captured by UAVs in flight for wildfire detection.

Wildfire management↗

DOE OSTI2022

Prediction of inter-chain distance maps of protein complexes with 2D attention-based deep neural networks

Residue-residue distance information is useful for predicting tertiary structures of protein monomers or quaternary structures of protein complexes. Many deep learning methods have been developed to predict intra-chain residue-residue distances of monomers accurately, but few methods can accurately predict inter-chain residue-residue distances of complexes. We develop a deep learning method CDPred (i.e., Complex Distance Prediction) based on the 2D attention-powered residual network to address the gap. Tested on two homodimer datasets, CDPred achieves the precision of 60.94% and 42.93% for top L/5 inter-chain contact predictions (L: length of the monomer in homodimer), respectively, substantially higher than DeepHomo’s 37.40% and 23.08% and GLINTER’s 48.09% and 36.74%. Tested on the two heterodimer datasets, the top Ls/5 inter-chain contact prediction precision (Ls: length of the shorter monomer in heterodimer) of CDPred is 47.59% and 22.87% respectively, surpassing GLINTER’s 23.24% and 13.49%. Moreover, the prediction of CDPred is complementary with that of AlphaFold2-multimer.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRSDate not supplied

Health Monitoring and Prognostics in Li-ion Batteries

Space applications need to overcome a very critical challenge of predicting remaining useful life of its critical systems/subsystems, with batteries being one of them. Batteries, power electronics conditioning system and motors and one of the most critical systems. Similarly in case of electric aircrafts, computing remaining flying time is safety-critical, since an aircraft that runs out of power (battery charge) while in the air will eventually lose control leading to catastrophe. To tackle and solve the prediction problem, it is essential to have awareness of the current state and health of the system, especially since it is necessary to perform condition-based predictions. To be able to accurately predict the future state of any system, it is required to possess knowledge of its current and future operations. Given models of the current and future system behavior, the general approach of model-based prognostics can be employed as a solution to the prior stated prediction problem. Two approaches are presented with battery prognostics application. The first approach presentation covers a physics based-modeling approach implemented for battery prognostics. Given models of the current and future system behavior, a general approach of model-based prognostics can be employed as a solution to the prediction problem and further for decision making. A second hybrid framework for fusing information from physics-based performance models along with deep learning algorithms for prognostics of complex safety critical systems i.e. batteries is presented. In this framework, physics-based performance models infer unobservable model parameters related to the system's components health solving a calibration problem in the deep learning approach.

Batteries↗

NASA NTRSDate not supplied

Health Monitoring and Prognostics in Li-ion Batteries

Space applications need to overcome a very critical challenge of predicting remaining useful life of its critical systems/subsystems, with batteries being one of them. Batteries, power electronics conditioning system and motors and one of the most critical systems. Similarly in case of electric aircrafts, computing remaining flying time is safety-critical, since an aircraft that runs out of power (battery charge) while in the air will eventually lose control leading to catastrophe. To tackle and solve the prediction problem, it is essential to have awareness of the current state and health of the system, especially since it is necessary to perform condition-based predictions. To be able to accurately predict the future state of any system, it is required to possess knowledge of its current and future operations. Given models of the current and future system behavior, the general approach of model-based prognostics can be employed as a solution to the prior stated prediction problem. Two approaches are presented with battery prognostics application. The first approach presentation covers a physics based-modeling approach implemented for battery prognostics. Given models of the current and future system behavior, a general approach of model-based prognostics can be employed as a solution to the prediction problem and further for decision making. A second hybrid framework for fusing information from physics-based performance models along with deep learning algorithms for prognostics of complex safety critical systems i.e. batteries is presented. In this framework, physics-based performance models infer unobservable model parameters related to the system's components health solving a calibration problem in the deep learning approach.

Battery Prognostics↗

NASA NTRSDate not supplied

Hybrid Model Based Approaches for Systems Health Management and Prognostics

To facilitate and solve the prediction problem, awareness of the current health state of the system is key, since it is necessary to perform condition-based predictions. To accurately predict the future state of any system, it is required to possess knowledge of its current health state and future operational conditions. Latest achievements of data-driven algorithms in regression of complex nonlinear functions and classification tasks have generated a growing interest in artificial intelligence for industrial applications. Complex multi-physics models as well as digital twins, once purely built on physics and corresponding simplified lumped parameter iterations, can now benefit from machine learning algorithms to mitigate the lack of understanding of some complex behavior. Given models of the current and future system behavior, a general approach of model-based prognostics can solve the prediction problem and further decision making. In principle, data driven approaches can replace expensive experimental test-setups as well as reduce the number of simulations needed to explore, e.g., the parametric space of a multi-parameter model. Nonetheless, the limitations of pure data-driven methods came to light rather quickly, at least for some industries. In many industrial applications, data acquisition is costly, and the volume of data that can be collected does not satisfy the requirements for an effective model training and cross-validation. Therefore, some recent works in the area of machine learning is focusing on blending physics with data-driven algorithms, thus mitigating the drawbacks of the two approaches and emphasizing respective advantages. Partial physical knowledge of the problem can aid the learning process by “guiding” the algorithm towards efficient solutions that satisfy the physics driving the system behavior. The result is a hybrid modeling approach combining physical knowledge as well data driven methods to develop a unified hybrid approach. A hybrid framework for fusing information from physics-based performance models along with deep learning algorithms for prognostics of complex safety critical systems is presented. In this framework, physics-based performance models infer unobservable model parameters related to the system's components health solving a calibration problem in the deep learning approach.

Hybrid Modeling↗

NASA NTRSDate not supplied

Hybrid Approaches to Systems Health Management and Prognostics

To facilitate and solve the prediction problem, awareness of the current health state of the system is key, since it is necessary to perform condition-based predictions. To accurately predict the future state of any system, it is required to possess knowledge of its current health state and future operational conditions. Latest achievements of data-driven algorithms in regression of complex nonlinear functions and classification tasks have generated a growing interest in artificial intelligence for industrial applications. Complex multi-physics models as well as digital twins, once purely built on physics and corresponding simplified lumped parameter iterations, can now benefit from machine learning algorithms to mitigate the lack of understanding of some complex behavior. Given models of the current and future system behavior, a general approach of model-based prognostics can solve the prediction problem and further decision making. In principle, data driven approaches can replace expensive experimental test-setups as well as reduce the number of simulations needed to explore, e.g., the parametric space of a multi-parameter model. Nonetheless, the limitations of pure data-driven methods came to light rather quickly, at least for some industries. In many industrial applications, data acquisition is costly, and the volume of data that can be collected does not satisfy the requirements for an effective model training and cross-validation. Therefore, some recent works in the area of machine learning is focusing on blending physics with data-driven algorithms, thus mitigating the drawbacks of the two approaches and emphasizing respective advantages. Partial physical knowledge of the problem can aid the learning process by “guiding” the algorithm towards efficient solutions that satisfy the physics driving the system behavior. The result is a hybrid modeling approach combining physical knowledge as well data driven methods to develop a unified hybrid approach. A hybrid framework for fusing information from physics-based performance models along with deep learning algorithms for prognostics of complex safety critical systems is presented. In this framework, physics-based performance models infer unobservable model parameters related to the system's components health solving a calibration problem in the deep learning approach.

Systems Health Management↗

DOE OSTI2021

Subaru Hyper Suprime-Cam revisits the large-scale environmental dependence on galaxy morphology over 360 deg2 at z = 0.3–0.6

This study investigates the role of large-scale environments on the fraction of spiral galaxies at z = 0.3–0.6 sliced to three redshift bins of Δz = 0.1. Here, we sample 276220 massive galaxies in a limited stellar mass of 5 × 1010 solar mass (~M*) over 360 deg2, as obtained from the Second Public Data Release of the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). By combining projected two-dimensional density information (Shimakawa et al. 2021, MNRAS, 503, 3896) and the CAMIRA cluster catalog (Oguri et al. 2018, PASJ, 70, S20), we investigate the spiral fraction across large-scale overdensities and in the vicinity of red sequence clusters. We adopt transfer learning to reduce the cost of labeling spiral galaxies significantly and then perform stacking analysis across the entire field to overcome the limitations of sample size. Here we employ a morphological classification catalog by the Galaxy Zoo Hubble (Willett et al., 2017, MNRAS, 464, 4176) to train the deep learning model. Based on 74103 sources classified as spirals, we find moderate morphology–density relations on a 10 comoving Mpc scale, thanks to the wide-field coverage of HSC-SSP. Clear deficits of spiral galaxies have also been confirmed, in and around 1136 red sequence clusters. Furthermore, we verify whether there is a large-scale environmental dependence on rest-frame u - r colors of spiral galaxies; such a tendency was not observed in our sample.

Astronomy & Astrophysics↗

DOE OSTI2022

Estimating cluster masses from SDSS multiband images with transfer learning

ABSTRACT The total masses of galaxy clusters characterize many aspects of astrophysics and the underlying cosmology. It is crucial to obtain reliable and accurate mass estimates for numerous galaxy clusters over a wide range of redshifts and mass scales. We present a transfer-learning approach to estimate cluster masses using the ugriz-band images in the SDSS Data Release 12. The target masses are derived from X-ray or SZ measurements that are only available for a small subset of the clusters. We designed a semisupervised deep learning model consisting of two convolutional neural networks. In the first network, a feature extractor is trained to classify the SDSS photometric bands. The second network takes the previously trained features as inputs to estimate their total masses. The training and testing processes in this work depend purely on real observational data. Our algorithm reaches a mean absolute error (MAE) of 0.232 dex on average and 0.214 dex for the best fold. The performance is comparable to that given by redMaPPer, 0.192 dex. We have further applied a joint integrated gradient and class activation mapping method to interpret such a two-step neural network. The performance of our algorithm is likely to improve as the size of training data set increases. This proof-of-concept experiment demonstrates the potential of deep learning in maximizing the scientific return of the current and future large cluster surveys.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Temperature outweighs light and flow as the predominant driver of dissolved oxygen in US rivers

The concentration of dissolved oxygen (DO), an important measure of water quality and river metabolism, varies tremendously in time and space. Riverine DO is commonly perceived as regulated by interacting and competing drivers (light, temperature and flow) that define rivers’ climate. Its continental-scale drivers, however, have remained elusive, partly due to the scarcity and spatio-temporal inconsistency of water quality data. Here we show, via a deep learning model (long short-term memory) trained using data from 580 rivers, that temperature predominantly drives daily DO dynamics in the contiguous United States. Light comes a close second, whereas flow imparts minimal influence. This work showcases the promise of using deep learning models for data filling that enables large-scale systematic analysis of patterns and drivers. Results show fairly accurate prediction of DO by temperature alone, and declining DO in warming rivers, which has important implications for water security and ecosystem health in the future climate.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Learning genetic perturbation effects with variational causal inference

Advances in sequencing technologies have enhanced the understanding of gene regulation in cells. In particular, Perturb-seq has enabled high-resolution profiling of the transcriptomic response to genetic perturbations at the single-cell level. This understanding has implications in functional genomics and potentially for identifying therapeutic targets. Various computational models have been developed to predict perturbational effects. While deep learning models excel at interpolating observed perturbational data, they tend to overfit in the lack of enough data and may not generalize well to unseen perturbations. In contrast, mechanistic models, such as linear causal models based on gene regulatory networks, hold greater potential for extrapolation, as they encapsulate regulatory information that can predict responses to unseen perturbations. However, their application has been limited to small studies due to overly simplistic assumptions, making them less effective in handling noisy, large-scale single-cell data. We propose a hybrid approach that combines a mechanistic causal model with variational deep learning, termed Single Cell Causal Variational Autoencoder (SCCVAE). The mechanistic model employs a learned regulatory network to represent perturbational changes as shift interventions that propagate through the learned network. SCCVAE integrates this mechanistic causal model into a variational autoencoder, generating rich, comprehensive transcriptomic responses. Our results indicate that SCCVAE exhibits superior performance over current state-of-the-art baselines for extrapolating to predict unseen perturbational responses. Additionally, for the observed perturbations, the latent space learned by SCCVAE allows for the identification of functional perturbation modules and simulation of single-gene knockdown experiments of varying penetrance, presenting a robust tool for interpreting and interpolating perturbational responses at the single-cell level.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2020

Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations

For centuries, flow visualization has been the art of making fluid motion visible in physical and biological systems. Although such flow patterns can be, in principle, described by the Navier-Stokes equations, extracting the velocity and pressure fields directly from the images is challenging. We addressed this problem by developing hidden fluid mechanics (HFM), a physics-informed deep-learning framework capable of encoding the Navier-Stokes equations into the neural networks while being agnostic to the geometry or the initial and boundary conditions. We demonstrate HFM for several physical and biomedical problems by extracting quantitative information for which direct measurements may not be possible.

Raissi, Maziar↗

NASA NTRS2021

AI4MARS: A Dataset for Terrain-Aware Autonomy on Mars

Deep learning has quickly become a necessity for selfdriving vehicles on Earth. In contrast, the self-driving vehicles on Mars, including NASA’s latest rover, Perseverance, which is planned to land on Mars in February 2021, are still driven by classical machine vision systems. Deep learning capabilities, such as semantic segmentation and object recognition, would substantially benefit the safety and productivity of ongoing and future missions to the red planet. To this end, we created the first large-scale dataset, AI4Mars, for training and validating terrain classification models for Mars, consisting of ~326K semantic segmentation full image labels on 35K images from Curiosity, Opportunity, and Spirit rovers, collected through crowdsourcing. Each image was labeled by ~10 people to ensure greater quality and agreement of the crowdsourced labels. It also includes ~1.5K validation labels annotated by the rover planners and scientists from NASA’s MSL (Mars Science Laboratory) mission, which operates the Curiosity rover, and MER (Mars Exploration Rovers) mission, which operated the Spirit and Opportunity rovers. We trained a DeepLabv3 model on the AI4Mars training dataset and achieved over 96% overall classification accuracy on the test set. The dataset is made publicly available.1

Ono, Hiro↗

DOE OSTI2023

An Online Prototype Toolset for Predicting and Optimizing P&T Performance (FY23 Status Report)

A new web-based toolset is being developed to support ongoing remediation optimization efforts and implementation of an adaptive site management strategy for the 200 West Area Pump-and-Treat (P&T) system at the Hanford Site. This toolset, comprising the well performance index tool and the well optimization pre-screening tool, will offer a user-friendly interface to predict and optimize the P&T well network’s performance at a preliminary level. Efforts in fiscal year (FY) 2023 focused on three main components: updating the existing deep learning model for predicting P&T performance, designing and developing a prototype of a web-based performance index tool, and initiating the conceptual design of the well optimization pre-screening tool. The well performance index tool is based on a pre-trained deep learning model that allows users to select a target contaminant and well screen length, then visualize the predicted performance of potential new wells across the site. The well optimization pre-screening tool includes two separate modules: the pre-computed scenario viewer, which organizes and visualizes offline optimization simulation results, and the quick analysis module, which provides real-time model prediction using user-specified well locations. In FY24, the plan is to add web-based applications to SOCRATES for both the well performance prediction tool and the optimization prescreening tool, with accompanying user and theory guides. These tools are intended to enable an accessible, easily applied, and transparent approach to remedy planning and decision-making.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Learning and Fast Adaptation for Grid Emergency Control via Deep Meta Reinforcement Learning

As power systems are undergoing a significant transformation with more uncertainties, less inertia and closer to operation limits, there is increasing risk of large outages. Thus, there is an imperative need to enhance grid emergency control to maintain system reliability and security. Towards this end, great progress has been made in developing deep reinforcement learning (DRL) based grid control solutions in recent years. However, existing DRL-based solutions have two main limitations: 1) they cannot handle well with a wide range of grid operation conditions, system parameters, and contingencies; 2) they generally lack the ability to fast adapt to new grid operation conditions, system parameters, and contingencies, limiting their applicability for real-world applications. Here, in this paper, we mitigate these limitations by developing a novel deep meta-reinforcement learning (DMRL) algorithm. The DMRL combines the meta strategy optimization together with DRL, and trains policies modulated by a latent space that can quickly adapt to new scenarios. We test the developed DMRL algorithm on the IEEE 300-bus system. We demonstrate fast adaptation of the meta-trained DRL polices with latent variables to new operating conditions and scenarios using the proposed method, which achieves superior performance compared to the state-of-the-art DRL and model predictive control (MPC) methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2023

An Explainable Machine-Learning Model for Compensatory Reserve Measurement: Methods for Feature Selection and the Effects of Subject Variability

Tracking vital signs accurately is critical for triaging a patient and ensuring timely therapeutic intervention. The patient’s status is often clouded by compensatory mechanisms that can mask injury severity. The compensatory reserve measurement (CRM) is a triaging tool derived from an arterial waveform that has been shown to allow for earlier detection of hemorrhagic shock. However, the deep-learning artificial neural networks developed for its estimation do not explain how specific arterial waveform elements lead to predicting CRM due to the large number of parameters needed to tune these models. Alternatively, we investigate how classical machine-learning models driven by specific features extracted from the arterial waveform can be used to estimate CRM. More than 50 features were extracted from human arterial blood pressure data sets collected during simulated hypovolemic shock resulting from exposure to progressive levels of lower body negative pressure. A bagged decision tree design using the ten most significant features was selected as optimal for CRM estimation. This resulted in an average root mean squared error in all test data of 0.171, similar to the error for a deep-learning CRM algorithm at 0.159. By separating the dataset into sub-groups based on the severity of simulated hypovolemic shock withstood, large subject variability was observed, and the key features identified for these sub-groups differed. This methodology could allow for the identification of unique features and machine-learning models to differentiate individuals with good compensatory mechanisms against hypovolemia from those that might be poor compensators, leading to improved triage of trauma patients and ultimately enhancing military and emergency medicine.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2020

The importance of temporal and spatial incoherence in quantitative interpretation of 4D-STEM

Recent developments in pixelated detectors, when combined with aberration correction of probe forming optics have greatly enhanced the field of scanning electron diffraction. Differential phase contrast is now routine and deep learning has been proposed as a method to extract maximum information from diffraction patterns. This study examines the effects of temporal and spatial incoherence on convergent beam electron diffraction patterns and demonstrates that simple center of mass measurements cannot be naively interpreted. The inclusion of incoherence in deep learning data sets is also discussed.

36 MATERIALS SCIENCE↗