Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep generative models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Data-driven wind turbine wake modeling via probabilistic machine learning

Wind farm design primarily depends on the variability of the wind turbine wake flows to the atmospheric wind conditions and the interaction between wakes. Physics-based models that capture the wake flow field with high-fidelity are computationally very expensive to perform layout optimization of wind farms, and, thus, data-driven reduced-order models can represent an efficient alternative for simulating wind farms. In this work, we use real-world light detection and ranging (LiDAR) measurements of wind-turbine wakes to construct predictive surrogate models using machine learning. Specifically, we first demonstrate the use of deep autoencoders to find a low-dimensional latent space that gives a computationally tractable approximation of the wake LiDAR measurements. Then, we learn the mapping between the parameter space and the (latent space) wake flow fields using a deep neural network. Additionally, we also demonstrate the use of a probabilistic machine learning technique, namely, Gaussian process modeling, to learn the parameter-space-latent-space mapping in addition to the epistemic and aleatoric uncertainty in the data. Finally, to cope with training large datasets, we demonstrate the use of variational Gaussian process models that provide a tractable alternative to the conventional Gaussian process models for large datasets. Furthermore, we introduce the use of active learning to adaptively build and improve a conventional Gaussian process model predictive capability. Overall, we find that our approach provides accurate approximations of the wind-turbine wake flow field that can be queried at an orders-of-magnitude cheaper cost than those generated with high-fidelity physics-based simulations.

Deep neural networks↗
DOE OSTI2021

Model-Free Voltage Control of Active Distribution System with PVs Using Surrogate Model-Based Deep Reinforcement Learning

Accurate knowledge of the distribution system topology and parameters is required to achieve good voltage control performance, but this is difficult to obtain in practice. This paper proposes a physical-model-free voltage control method based on a surrogate-model-enabled deep reinforcement learning approach. Specifically, a surrogate model is trained in a supervised manner using the recorded limited number of historical data to learn the relationship between the power injections and voltage fluctuations of each node. Then, the deep reinforcement learning algorithm is applied to learn an optimal control strategy from the experiences obtained by continuous interactions with the surrogate model. The proposed method can achieve physical-model-free control of unbalanced distribution network and inform real-time decisions to deal with fast voltage fluctuations caused by the rapid variation of PV generation. Simulation results on an unbalance IEEE 123-bus system show that the proposed method can achieve similar performance as that of perfect physical-model-based approaches while being advantageous over other traditional methods.

active distribution network↗
DOE OSTI2021

Process Anomaly Detection for Sparsely Labeled Events in Nuclear Power Plants

An essential aspect of online monitoring, subtle anomaly detection increases the detection lead time for equipment failure and enables a nuclear power plant (NPP) to mitigate unexpected partial or full outages, resulting in significant cost saving to the plant. Once an anomaly is detected by plant staff, its cause and severity are investigated. Because the vast majority of anomalies require some level of investigation, including some that require time-consuming examination, before they are passed over to the engineering organization for further analysis, plants are often equipped with tools to assist the staff in performing anomaly detection. Those tools operate as a black box and are often based on statistical methods that establish sensor correlations using preconfigured mathematical models and flag correlation deviations as anomalies. Due to the number of anomalies detected at a given NPP on a daily basis, a significant number of flagged anomalies usually await examination for days or weeks. A primary cause of this backlog is that the methods used by the tools generate many false positives. Though this is usually attributed to oversensitive model settings due to very narrow normal operation bands, it can also be associated with the model development being inadequate for the process being monitored, or with missing model inputs that could have explained misclassified positives. The performance of anomaly detection tools impacts their plant acceptance and utilization, especially when the effort to address false positives generated by the tool depletes the value or cost saved by using that tool. Thus, means to advance anomaly detection performance have been investigated by the Department of Energy’s Light Water Reactor Sustainability program. Previous and ongoing efforts have targeted unsupervised machine-learning (ML) methods, which do not require the labeling of any data fed into the ML model. By contrast, in supervised anomaly detection methods, every data point is labeled as either a normal or abnormal process condition, and the model is trained to replicate the classification process. Supervised methods usually outperform unsupervised methods, due to the added value in differentiating normal from anomalous states of the monitored process. An NPP’s corrective action program requires it to track and document, via a dedicated report, the resolution of any issues that occur within the plant. Once created, each report is reviewed by a plant screening committee, and several classifications and decisions are made. Recently, a collaborating NPP developed an artificial intelligence and ML-based classifier to categorize a condition report (CR) into classes that can serve to label the data as normal or anomalous. Applying CRs as labels represents a semi-supervised use case. Semi-supervised ML assumes that labels exist for some data points (i.e., labeled anomalies, in this case) but not for the rest. In this effort, semi-supervised ML methods were used to fuse data from CRs with anomaly detection methods in order to test the hypothesis that partially labeled anomalies would improve the accuracy of the anomaly detection methods. Specifically, two methods were used. The first is the deep Semi-supervised Anomaly Detection (deep SAD) method, which can handle labels ranging from fully unsupervised to fully supervised cases. The second is a newly designed ML method developed specifically for this effort and referred to as the high-order feature (HOF)-based method. To evaluate these two methods in controlled environments, synthetic data generators were developed and used. The first datasets used a spring-mass-damper (SMD) system simulator commonly found in mechanical engineering references. This was used to create two use cases: a one- and a three-mass system. Anomalies were introduced by changing the spring and damper coefficients while the system was actuated by random forces. The second datasets used the commercial Dymola-Modelica software to build a simplified nuclear reactor model. Anomalies were added in the form of corrupted sensor readings and/or control commands. The deep SAD method was tested using the SMD system, while the HOF method was tested using both datasets. Application of the deep SAD semi-supervised ML method demonstrated that labels can generate increased confidence in detecting true anomalies. This helped increase the number of true positives and decrease the number of false negatives—something that would aid in addressing the backlog of possible anomalies. Application of the HOF method demonstrated that labels can aid in down selecting from a candidate set of features to a more optimal subset in order to better differentiate between normal and anomalous conditions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Deep learning inversion of gravity data for detection of CO 2 plumes in overlying aquifers

In this work, we developed an effective U-Net based deep learning (DL) model for inversion of surface gravity data on a rectangular grid to predict 2-D high-resolution subsurface CO 2 distribution along a vertical cross-section due to CO 2 leakage through a wellbore within a deep CO 2 storage reservoir. We used synthetic data to model two types of CO 2 leakage scenarios: one CO 2 plume in a shallow aquifer (single plume case), and two plumes present at different depths (double plume case). The 3-D synthetic plume samples were created by sampling among predetermined CO 2 plume depths, saturations, and volumes. The corresponding surface gravity data on a rectangular grid were generated by a 3-D forward model. The U-Net model detected 72% of single-plume samples, and one or both plumes in 75% of double-plume samples. Most of the undetected single plumes have small gravity field strengths below the typical noise level of 5 μGal. This model generated reproducible, reliable predictions with acceptable errors and demonstrated improved spatial resolution over the conventional least-squares inversion. In contrast to the conventional least-squares inversion, which often overestimates the size of its target and underestimates its density, this U-Net model accurately delineated the boundary of a target. Furthermore, this DL inversion detected deep, small, or low saturation CO 2 plumes that are often more difficult to resolve with conventional gravity inversion methods. We note the limitations of this feasibility study, including the use of synthetic data with regular CO 2 plume shapes, and the prediction of a 2-D plume cross-section rather than the full 3-D plume, as well, we recognize the lower detection fraction for double-plume scenarios. Nevertheless, this study demonstrates that DL gravity inversion is a promising and potentially superior method to conventional least-squares inversion. Our U-Net based deep learning inversion approach may be adapted for inversion of other types of geophysical data. DL inversion can facilitate near real-time monitoring of geologic carbon sequestration to provide site operators with prompt information about subsurface CO 2 distribution for risk management and mitigation.

58 GEOSCIENCES↗
NASA NTRS2022

Bayesian Deep Learning for Segmentation for Autonomous Safe Planetary Landing

Hazard detection is critical for enabling autonomous landing on planetary surfaces. Current state-of-the-art methods leverage traditional computer vision approaches to automate the identification of safe terrain from input digital elevation models (DEMs). However, performance for these methods can degrade for input DEMs with increased sensor noise. In the last decade, deep learning techniques have been developed for various applications. Nevertheless, their applicability to safety-critical space missions has often been limited due to concerns regarding their outputs’ reliability. In response to these limitations, this paper proposes an application of the Bayesian deep learning segmentation method for hazard detection. The developed approach enables reliable, safe landing site detection by i) generating simultaneously a safety prediction map and its uncertainty map via Bayesian deep learning and semantic segmentation, and ii) using the uncertainty map to filter out the uncertain pixels in the prediction map so that the safe site identification is performed only based on the certain pixels (i.e., pixels for which the model is certain about its safety prediction). Experiments are presented with simulated data based on a Mars HiRISE digital terrain model by varying uncertainty threshold and noise levels to demonstrate the performance of the proposed approach.

Kento Tomita↗
DOE OSTI2022

Materials structure–property factorization for identification of synergistic phase interactions in complex solar fuels photoanodes

Abstract Properties can be tailored by tuning composition in high-order composition spaces. For spaces with complex phase behavior, modeling the properties as a function of composition and phase distribution remains a formidable challenge. We present materials structure–property factorization (MSPF) as an approach to automate modeling of such data and identify synergistic phase interactions. MSPF is an interpretable machine learning algorithm that couples phase mapping via Deep Reasoning Networks (DRNets) to matrix factorization-based modeling of the representative properties of each phase in a dataset. MSPF is demonstrated for Bi–Cu–V oxide photoanodes for solar fuel generation, which contains 25 different phase combinations and correspondingly exhibits complex composition-structure-photoactivity relationships. Comparing the measured photoactivity to a learned model for non-interacting phases, synergistic phase interactions are identified to guide further photoactivity optimization and understanding. MSPF identifies synergistic interactions of a BiVO 4 -like phase with both Cu 2 V 2 O 7 -like and CuV 2 O 6 -like phases, creating avenues for understanding complex photoelectrocatalysts.

36 MATERIALS SCIENCE↗
DOE OSTI2020

AI for Science: Report on the Department of Energy (DOE) Town Halls on Artificial Intelligence (AI) for Science

The report documents the DOE Town Halls held during 2019 at Argonne National Laboratory, Oak Ridge National Laboratory, Lawrence Berkeley National Laboratory, and in Washington, DC. From July to October 2019, the Argonne, Oak Ridge, and Berkeley National Laboratories hosted a series of four town hall meetings attended by more than 1,000 U.S. scientists and engineers. The goal of the town hall series was to examine scientific opportunities in the areas of artificial intelligence (AI), Big Data, and high-performance computing (HPC) in the next decade, and to capture the big ideas, grand challenges, and next steps to realizing these opportunities. In this report and in the Department of Energy (DOE) laboratory community, we use the term “AI for Science” to broadly represent the next generation of methods and scientific opportunities in computing, including the development and application of AI methods (e.g., machine learning, deep learning, statistical methods, data analytics, automated control, and related areas) to build models from data and to use these models alone or in conjunction with simulation and scalable computing to advance scientific research. The AI for Science town hall discussions focused on capturing the transformational uses of AI that employ HPC and/or data analysis, leveraging data sets from HPC simulations or instruments and user facilities, and addressing scientific challenges unique to DOE user facilities and the agency’s wide-ranging fundamental and applied science enterprise.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Implementation of Detailed Polyethylene Pyrolysis Kinetics into CFD Simulations using Machine Learning

Municipal solid waste (MSW) and waste plastics have received significant attention due to the issues of waste generation and storage, as well as their potential as an energy resource. High-density polyethylene (HDPE) makes up a large portion of plastic waste and has been the subject of several conversion studies. However, the mechanisms associated with converting HDPE through pyrolysis and gasification are extensive and complex making them difficult to implement into high-fidelity computational fluid dynamic (CFD) simulations. For this project, a primary pyrolysis mechanism containing 42 unique species and 737 heterogeneous reactions was used to generate kinetic data over a range of operating conditions. A machine learning (ML) model was developed to replicate the results of the detailed pyrolysis mechanism while significantly increasing the computational efficiency. A deep operator network (DeepONet) architecture was adopted to train the model using time steps relevant to CFD simulations. The ML used physics-based loss functions to ensure mass conservation. The ML model has been deployed in simple MFiX CFD simulations, single particle, and an experimental drop tube reactor, and has shown promising performance compared to the original scheme.

Houston, Ross↗
NASA NTRS1987

A new linear quadratic optimal controller for the 34-meter high efficiency antenna position loop

The design of a new position loop controller for the 34-meter High Efficiency Deep Space antennas using linear quadratic (LQ) optimal control techniques is discussed. The LQ optimal control theory is reviewed, and model development and verification are discussed. Families of optimal gain vectors are generated by varying weight parameters. Performance specifications were used to select a final gain vector. Estimator dynamics were selected and the corresponding gain vectors were computed. Final estimator selection was based on position, commanded rate, and estimator error responses.

Nickerson, J. A.↗
DOE OSTI2020

Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

This paper develops a model-free volt-VAR optimization (VVO) algorithm via multi-agent deep reinforcement learning (DRL) in unbalanced distribution systems. This method is novel since we cast the VVO problem in distribution networks to an intelligent deep Q-network (DQN) framework, which avoids solving a specific optimization model directly when facing time-varying operating conditions in the systems. We consider statuses/ratios of switchable capacitors, voltage regulators, and smart inverters installed at distributed generators as the action variables of the agents. A delicately designed reward function guides these agents to interact with the distribution system, in the direction of reinforcing voltage regulation and power loss reduction simultaneously. The forward-backward sweep method for radial three-phase distribution systems provides accurate power flow results within a few iterations to the DRL environment. The proposed method realizes the dual goals for VVO. We test this algorithm on the unbalanced IEEE 13-bus and 123-bus systems. Numerical simulations validate the excellent performance of this method in voltage regulation and power loss reduction.

41 EE - Solar Energy Technologies Office (EE-4S)↗
NASA NTRS2005

A Hierarchical Modeling Study of the Interactions Among Turbulence, Cloud Microphysics, and Radiative Transfer in the Evolution of Cirrus Clouds

This project used a hierarchy of cloud resolving models to address the following science issues of relevance to CRYSTAL-FACE: What ice crystal nucleation mechanisms are active in the different types of cirrus clouds in the Florida area and how do these different nucleation processes influence the evolution of the cloud system and the upper tropospheric humidity? How does the feedback between supersaturation and nucleation impact the evolution of the cloud? What is the relative importance of the large-scale vertical motion and the turbulent motions in the evolution of the crystal size spectra? How does the size spectra impact the life-cycle of the cloud, stratospheric dehydration, and cloud radiative forcing? What is the nature of the turbulence and waves in the upper troposphere generated by precipitating deep convective cloud systems? How do cirrus microphysical and optical properties vary with the small-scale dynamics? How do turbulence and waves in the upper troposphere influence the cross-tropopause mixing and stratospheric and upper tropospheric humidity? The models used in this study were: 2-D hydrostatic model with explicit microphysics that can account for 30 size bins for both the droplet and crystal size spectra. Notably, a new ice crystal nucleation scheme has been incorporated into the model. Parcel model with explicit microphysics, for developing and evaluating microphysical parameterizations. Single column model for testing bulk microphysics parameterizations

Curry, Judith↗
DOE OSTI2021

Limited-angle computed tomography with deep image and physics priors

Computed tomography is a well-established x-ray imaging technique to reconstruct the three-dimensional structure of objects. It has been used extensively in a variety of fields, from diagnostic imaging to materials and biological sciences. One major challenge in some applications, such as in electron or x-ray tomography systems, is that the projections cannot be gathered over all the angles due to the sample holder setup or shape of the sample. This results in an ill-posed problem called the limited angle reconstruction problem. Typical image reconstruction in this setup leads to distortion and artifacts, thereby hindering a quantitative evaluation of the results. To address this challenge, we use a generative model to effectively constrain the solution of a physics-based approach. Our approach is self-training that can iteratively learn the nonlinear mapping from partial projections to the scanned object. Because our approach combines the data likelihood and image prior terms into a single deep network, it is computationally tractable and improves performance through an end-to-end training. We also complement our approach with total-variation regularization to handle high-frequency noise in reconstructions and implement a solver based on alternating direction method of multipliers. We present numerical results for various degrees of missing angle range and noise levels, which demonstrate the effectiveness of the proposed approach.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits

Variational quantum computing schemes train a loss function by sending an initial state through a parametrized quantum circuit, and measuring the expectation value of some operator. Despite their promise, the trainability of these algorithms is hindered by barren plateaus (BPs) induced by the expressiveness of the circuit, the entanglement of the input data, the locality of the observable, or the presence of noise. Up to this point, these sources of BPs have been regarded as independent. In this work, we present a general Lie algebraic theory that provides an exact expression for the variance of the loss function of sufficiently deep parametrized quantum circuits, even in the presence of certain noise models. Our results allow us to understand under one framework all aforementioned sources of BPs. This theoretical leap resolves a standing conjecture about a connection between loss concentration and the dimension of the Lie algebra of the circuit’s generators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Risk-Aware Measurement Synchronization and Recovery for DSSE With Heterogeneous Data Sources

Power distribution systems are increasingly integrating heterogeneous sensors with varying data reporting rates and types, which pose challenges to achieving observability at the desired temporal resolution of distribution system state estimation (DSSE). Multisensor failures caused by extreme events exacerbate these issues, introducing substantial uncertainties into DSSE. This article proposes a novel solution to these challenges by ensuring high-resolution system observability despite heterogeneous data sources and multisensor failures. First, a deep learning architecture combining long short-term memory (LSTM) and graph convolutional network (GCN) is employed to synchronize meters with different reporting rates, aiming to achieve system observability. A random-walk-model-based approach is introduced to generate pseudo-measurements while properly characterizing their uncertainties under multisensor failures. Finally, a disaster-risk-informed observability metric (RiOM) is defined to quantify the uncertainty associated with state estimation results. The proposed framework offers deeper insights into the system observability on the fly compared with conventional analysis. The effectiveness of the framework is demonstrated on an IEEE standard test case and a large-scale real-world distribution feeder in mid-Minnesota in the U.S.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Attribute-Guided Adversarial Training for Robustness to Natural Perturbations

We report while existing work in robust deep learning has focused on small pixel-level norm-based perturbations, this may not account for perturbations encountered in several real world settings. In many such cases although test data might not be available, broad specifications about the types of perturbations (such as an unknown degree of rotation) may be known. We consider a setup where robustness is expected over an unseen test domain that is not i.i.d. but deviates from the training domain. While this deviation may not be exactly known, its broad characterization is specified a priori, in terms of attributes. We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space, without having access to the data from the test domain. Our adversarial training solves a min-max optimization problem, with the inner maximization generating adversarial perturbations, and the outer minimization finding model parameters by optimizing the loss on adversarial perturbations generated from the inner maximization. We demonstrate the applicability of our approach on three types of naturally occurring perturbations --- object-related shifts, geometric transformations, and common image corruptions. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations. We demonstrate the usefulness of the proposed approach by showing the robustness gains of deep neural networks trained using our adversarial training on MNIST, CIFAR-10, and a new variant of the CLEVR dataset.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Application of machine learning to estimate fireball characteristics and their uncertainty from infrared spectral data

Experiments or events involving high explosives (HE) can be monitored remotely by infrared (IR) sensors to gather information about the configuration or materials involved in the device. Researchers at the Air Force Institute of Technology (AFIT) developed a phenomenological model for HE fireball spectra in the IR range that allows for parameters to be extracted from Fourier transform infrared (FTIR) data. This model includes parameters tied to physical characteristics of the fireball: temperature, size, soot, and gas concentrations. Previous works have sought to recover these parameters by the fitting of either whole spectra or select wavenumber bands to this phenomenological model. Difficulties arise due to the complex relationships between the parameters to be fit. Uncertainty quantification of the estimated fireball parameters is also problematic since HE experiments do not have any ground truth information on the parameters. It is suggested that artificial neural network (ANN) based approaches may be well suited to this problem, because of their ability to capture complex and highly nonlinear relationships. As such, this work seeks to explore the efficacy of deep artificial neural networks (DNNs) for this problem of parameter recovery from spectra and to also investigate the uncertainty of recovering the fireball parameters from FTIR data. Networks are designed using the hyperparameter optimization tool Hyperopt and trained/tested on artificial data generated using the phenomenological model developed by AFIT. The results of applying the network to the artificial data set are compared to a physics-based band approach that uses a selected number of bands based on their physical properties. Information on the uncertainty of estimating parameters from remotely sensed experimental data is obtained by treating the accuracy of the DNN model on artificial data as an upper bound and by examining the impact of emissivity due to soot on parameter estimation error; the results for artificial data are likely to be optimistic as compared to recovering parameters from experimental data.

42 ENGINEERING↗
DOE OSTI2025

Incorporating geological structure into sensitivity analysis of subsurface contaminant transport

Simulating subsurface contaminant transport at the kilometer-scale often entails modeling reactive flow and transport within and through complex geologic structures. These structures are typically meshed by hand and as a result geologic structure is usually represented by one or a few deterministically generated geological models for uncertainty studies of flow and transport in the subsurface. Uncertainty in geologic structure can have a significant impact on contaminant transport. In this study, the impact of geologic structure on contaminant tracer transport in a shale formation is investigated for a simplified generic deep geologic repository for permanent disposal of spent nuclear fuel. An open-source modeling framework is used to perform a sensitivity analysis study on transport of two tracers from a generic spent nuclear fuel repository with uncertain location of the interfaces between the stratum of the geologic structure. The automated workflow uses sampled realizations of the geological structural model in addition to uncertain flow parameters in a nested sensitivity analysis. Concentration of the tracers at observation points within, in line with, and downstream of the repository are used as the quantities of interest for determining model sensitivity to input parameters and geological realization. Finally, the results of the study indicate that the location of strata interfaces in the geological structure has a first-order impact on tracer transport in the example shale formation, and that this impact may be greater than that of the uncertain flow parameters.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

Deep learning symmetries and their Lie groups, algebras, and subalgebras from first principles

Abstract We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset. We use fully connected neural networks to model the symmetry transformations and the corresponding generators. The constructed loss functions ensure that the applied transformations are symmetries and the corresponding set of generators forms a closed (sub)algebra. Our procedure is validated with several examples illustrating different types of conserved quantities preserved by symmetry. In the process of deriving the full set of symmetries, we analyze the complete subgroup structure of the rotation groups SO (2), SO (3), and SO (4), and of the Lorentz group S O ( 1 , 3 ) . Other examples include squeeze mapping, piecewise discontinuous labels, and SO (10), demonstrating that our method is completely general, with many possible applications in physics and data science. Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.

97 MATHEMATICS AND COMPUTING↗