Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,279 records · Page 71
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Safety Assessment of a Machine Learning-Based Aircraft Emergency Braking System: A Case Study

Machine Learning (ML) is revolutionizing many technological fields, but its use in aviation remains restricted due to stringent certification requirements. Efforts by the aviation community to establish standards for certifying ML-based systems are progressing, yet challenges persist, particularly with safety assessment methods for ML-based systems. This research addresses these challenges through a case study of an autonomous emergency braking system utilizing a computer vision deep neural network (DNN). We demonstrate a safety assessment process tailored to ML-specific concerns, such as low integrity and performance variability in quantitative safety analysis. This study can serve as an illustrative example to facilitate the discussion and convergence on certification aspects for ML-based systems within the aviation community.

Safety certification↗
DOE OSTI2024

Unpaired image translation to mitigate domain shift in liquid argon time projection chamber detector responses

Deep learning algorithms often are developed and trained on a training dataset and deployed on test datasets. Any systematic difference between the training and a test dataset may severely degrade the final algorithm performance on the test dataset—what is known as the domain shift problem . This issue is prevalent in many scientific domains where algorithms are trained on simulated data but applied to real-world datasets. Typically, the domain shift problem is solved through various domain adaptation (DA) methods. However, these methods are often tailored for a specific downstream task, such as classification or semantic segmentation, and may not easily generalize to different tasks. This work explores the feasibility of using an alternative way to solve the domain shift problem that is not specific to any downstream algorithm. The proposed approach relies on modern Unpaired Image-to-Image (UI2I) translation techniques, designed to find translations between different image domains in a fully unsupervised fashion. In this study, the approach is applied to a domain shift problem commonly encountered in Liquid Argon Time Projection Chamber (LArTPC) detector research when seeking a way to translate samples between two differently distributed LArTPC detector datasets deterministically. This translation allows for mapping real-world data into the simulated data domain where the downstream algorithms can be run with much less domain-shift-related performance degradation. Conversely, using the translation from the simulated data to a real-world domain can increase the realism of the simulated dataset and reduce the magnitude of any systematic uncertainties. To evaluate the quality of the translations, we use both pixel-wise metrics and a downstream task to measure the effectiveness of UI2I methods for mitigating the domain shift problem. We adapted several popular UI2I translation algorithms to work on scientific data and demonstrated the viability of these techniques for solving the domain shift problem with LArTPC detector data. To facilitate further development of DA techniques for scientific datasets, the ‘Simple Liquid-Argon Track Samples’ dataset used in this study is also published.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Newton-Raphson AC Power Flow Convergence Based on Deep Learning Initialization and Homotopy Continuation

Power flow forms the basis of many power system studies. With the increased penetration of renewable energy, grid planners tend to perform multiple power flow simulations under various operating conditions and not just selected snapshots at peak or light load conditions. Getting a converged AC power flow (ACPF) case remains a significant challenge for grid planners especially in large power grid networks. This paper proposes a two-stage approach to improve Newton-Raphson ACPF convergence and was applied to a 6102 bus Electric Reliability Council of Texas (ERCOT) system. The first stage utilizes a deep learning-based initializer with data re-training. Here a deep neural network (DNN) initializer is developed to provide better initial voltage magnitude and angle guesses to aid in power flow convergence. This is because Newton-Raphson ACPF is quite sensitive to the initial conditions and bad initialization could lead to divergence. The DNN initializer includes a data re-training framework that improves the initializer's performance when faced with limited training data. The DNN initializer successfully solved 3,285 cases out of 3,899 non-converging dispatch and performed better than random forest and DC power flow initialization methods. ACPF cases not solved in this first stage are then passed through a hot-starting algorithm based on homotopy continuation with switched shunt control. The hot-starting algorithm successfully converged 416 cases out of the remaining 614 non-converging ACPF dispatch. In conclusion, the combined two-stage approach achieved a 94.9% success rate, by converging a total of 3,701 cases out of the initial 3,899 unsolved cases.

Deep learning↗
DOE OSTI2024

Buyers Collusion in Incentivized Forwarding Networks: A Multi-Agent Reinforcement Learning Study

We present the issue of monetarily incentivized forwarding in a multi-hop mesh network architecture from an economic perspective. It is anticipated that credit-incentivized forwarding and relaying will be a simple method of exchanging transmission power and spectrum for connectivity. However, gateways and forwarding nodes, like any other free market, may create an oligopolistic market for the users they serve. In this study, a coalition scheme between buyers aims to address price control by gateways or nodes closer to gateways. In a Stackelberg competition game, buyer agents (users) and sellers (gateways) make decisions using reinforcement learning (RL), with decentralized Deep Q-Networks to buy and sell forwarding resources. We allow communication links between the buyers with a limited messaging space, without defining a collusion mechanism. The idea is to demonstrate that through messaging, and RL tacit collusion can emerge between agents in a decentralized setup. The multi-agent reinforcement learning (MARL) system is presented and analyzed from a machine-learning perspective. Moreover, MARL dynamics are discussed via mean field analysis to better understand divergence causes and make implementation recommendations for such systems. Finally, the simulation results show the results of coordination among the users.

42 ENGINEERING↗
DOE OSTI2022

Uncertainty Aware Deep Learning for Particle Accelerators

Standard deep learning models for classification and regression applications are ideal for capturing complex system dynamics. However, their predictions can be arbitrarily inaccurate when the input samples are not similar to the training data. Implementation of distance aware uncertainty estimation can be used to detect these scenarios and provide a level of confidence associated with their predictions. In this paper, we present results from using Deep Gaussian Process Approximation (DGPA) methods for errant beam prediction at Spallation Neutron Source (SNS) accelerator (classification) and we provide an uncertainty aware surrogate model for the Fermi National Accelerator Lab (FNAL) Booster Accelerator Complex (regression).

Rajput, Kishansingh↗
DOE OSTI2022

On the Verification of Deep Reinforcement Learning Solution for Intelligent Operation of Distribution Grids

Capabilities of deep reinforcement learning (DRL) in obtaining fast decision policies in high dimensional and stochastic environments have led to its extensive use in operational research, including the operation of distribution grids with high penetration of distributed energy resources (DER). However, the feasibility and robustness of DRL solutions are not guaranteed for the system operator, and hence, those solutions may be of limited practical value. This paper proposes an analytical method to find feasibility ellipsoids that represent the range of multi-dimensional system states in which the DRL solution is guaranteed to be feasible. Empirical studies and stochastic sampling determine the ratio of the discovered to the actual feasible space as a function of the sample size. In addition, the performance of logarithmic, linear, and exponential penalization of infeasibility during the DRL training are studied and compared in order to reduce the number of infeasible solutions

Hosseini, Mohammad Mehdi↗
DOE OSTI2024

Yet Another Discriminant Analysis (YADA): A Probabilistic Model for Machine Learning Applications

This paper presents a probabilistic model for various machine learning (ML) applications. While deep learning (DL) has produced state-of-the-art results in many domains, DL models are complex and over-parameterized, which leads to high uncertainty about what the model has learned, as well as its decision process. Further, DL models are not probabilistic, making reasoning about their output challenging. In contrast, the proposed model, referred to as Yet Another Discriminate Analysis(YADA), is less complex than other methods, is based on a mathematically rigorous foundation, and can be utilized for a wide variety of ML tasks including classification, explainability, and uncertainty quantification. YADA is thus competitive in most cases with many state-of-the-art DL models. Ideally, a probabilistic model would represent the full joint probability distribution of its features, but doing so is often computationally expensive and intractable. Hence, many probabilistic models assume that the features are either normally distributed, mutually independent, or both, which can severely limit their performance. YADA is an intermediate model that (1) captures the marginal distributions of each variable and the pairwise correlations between variables and (2) explicitly maps features to the space of multivariate Gaussian variables. Numerous mathematical properties of the YADA model can be derived, thereby improving the theoretic underpinnings of ML. Validation of the model can be statistically verified on new or held-out data using native properties of YADA. However, there are some engineering and practical challenges that we enumerate to make YADA more useful.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2018

GEONEX: Progressive Conditional Generative Adversarial Training Using Transfer learning

Obtaining accurate segmentation on large scale images is an open problem in deep learning. The main problem is the amount of labeled data that exists for large scale images. Traditionally, the common solution to this problem is to crop the large images into smaller images to increase the amount of available data and train a Conditional Generative Adversarial Network (CGAN). CGANs are currently the state of the art in image to image translation and provide better accuracy than the traditional method of training a encoder based conv-net architecture to minimize the loss at each pixel. This method can produce noisy and discontinuous images with inaccurate results. We seek to solve this problem by utilizing the concepts of transfer learning and progressive training to create a CGAN that can segment large scale images with a limited amount of labeled data. In transfer learning we recognize that many learned features are applicable to many classes from multiple domains. This introduces the concept of feature reusability, which is the basis for finetuning. Progressive training got its start in training models on the same images at different resolutions. In this work we instead train a GAN on increasing image scales by transferring the weights from the smaller scales to the larger scales. The learned features at the smaller scales are continually reused and applied to larger scales to create a CGAN that can perform accurate segmentation on large scale images. We apply this method to detect building footprints on very high-resolution overhead imagery (e.g Digital Globe and high resolution airborne platforms).

GEONEX↗
DOE OSTI2025

Data Selection Improvement For MicroBooNE

Data selection is an extremely important part of data analysis for any experiment. Finding a physics result is often the result of sifting through a massive amount of data, keeping data that we believe to be signal, and throwing out data we do not. This process is called data selection. Creating a selection algorithm is an intensive process that must balance keeping enough data to have statistics and maximizing the signal purity of that data. We also need to choose the right reconstruction method, a tool to take raw data from the detector and convert it into physics results. In this study, we used three different reconstruction tools, Pandora, WireCell, and LANTERN, for the MicroBooNE experiment in conjunction to improve the selection algorithm for analysis. For the case of this study, we look into the charged current N proton 0 pions (CCNp0$\pi$) interaction channel. This is the dominant channel for the Short Baseline Neutrino (SBN) program and is expected to be a large contributor to the Deep Underground Neutrino Experiment (DUNE). We first investigated each of the three tools to find out more about their strengths and weaknesses as reconstructions, and compared them to the truth information directly from the MicroBooNE simulation pipeline. We then put together a direct comparison of the three methods to find which method or combination of methods would return the best result for us. While the study is ongoing, we have learned a lot about data selection for the experiment and the differences between the reconstruction tools.

Dillon, Brayden [Fermilab]↗
DOE OSTI2023

Systems and methods for customizing kernel machines with deep neural networks

A method including receiving an input data set. The input data set can include one of a feature domain set or a kernel matrix. The method also can include constructing dense embeddings using: (i) Nyström approximations on the input data set when the input data set comprises the kernel matrix, and (ii) clustered Nyström approximations on the input data set when the input data set comprises the feature domain set. The method additionally can include performing representation learning on each of the dense embeddings using a multi-layer fully-connected network for each of the dense embeddings to generate latent representations corresponding to each of the dense embeddings. The method further can include applying a fusion layer to the latent representations corresponding to the dense embeddings to generate a combined representation. The method additionally can include performing classification on the combined representation. Other embodiments of related systems and methods are also disclosed.

Song, Huan↗
DOE OSTI2021

Evaluating Automated Face Identity-Masking Methods with Human Perception and a Deep Convolutional Neural Network

Face de-identification (or “masking”) algorithms have been developed in response to the prevalent use of video recordings in public places. Here, we evaluated the success of face identity masking for human perceivers and a deep convolutional neural network (DCNN). Eight de-identification algorithms were applied to videos of drivers’ faces, while they actively operated a motor vehicle. These masks were pre-selected to be applicable to low-quality video and to maintain coarse information about facial actions. Humans studied high-resolution images to learn driver identities and were tested on their recognition of active drivers in low-resolution videos. Faces in the videos were either unmasked or were masked by one of the eight algorithms. When participants were tested immediately after learning (Experiment 1), all masks reduced identification, with six of eight masks reducing identification to extremely poor performance. In a second experiment, two of the most effective masks were tested after a delay of 7 or 28 days. The delay did not further reduce identification of the masked faces. In all masked conditions, participants maintained stringent decision criteria, with low confidence in recognition, further indicating the effectiveness of the masks. Next, the DCNN performed an identity-matching task between high-resolution images and masked videos—a task analogous to that done by humans. The pattern of accuracy for the DCNN mirrored some, but not all, aspects of human performance, highlighting the need to test the effectiveness of identity masking for both humans and machines. The DCNN was also tested on its ability to match identity between masked and unmasked versions of the same video, based only on the face. DCNN performance for the eight masks offers insight into the nature of the information in faces that is coded in these networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Improving River Routing Using a Differentiable Muskingum‐Cunge Model and Physics‐Informed Machine Learning

Recently, rainfall-runoff simulations in small headwater basins have been improved by methodological advances such as deep neural networks (NNs) and hybrid physics-NN models—particularly, a genre called differentiable modeling that intermingles NNs with physics to learn relationships between variables. However, hydrologic routing simulations, necessary for simulating floods in stem rivers downstream of large heterogeneous basins, had not yet benefited from these advances and it was unclear if the routing process could be improved via coupled NNs. We present a novel differentiable routing method (δMC-Juniata-hydroDL2) that mimics the classical Muskingum-Cunge routing model over a river network but embeds an NN to infer parameterizations for Manning's roughness (n) and channel geometries from raw reach-scale attributes like catchment areas and sinuosity. The NN was trained solely on downstream hydrographs. Synthetic experiments show that while the channel geometry parameter was unidentifiable, n can be identified with moderate precision. With real-world data, the trained differentiable routing model produced more accurate long-term routing results for both the training gage and untrained inner gages for larger subbasins (>2,000 km2) than either a machine learning model assuming homogeneity, or simply using the sum of runoff from subbasins. The n parameterization trained on short periods gave high performance in other periods, despite significant errors in runoff inputs. The learned n pattern was consistent with literature expectations, demonstrating the framework's potential for knowledge discovery, but the absolute values can vary depending on training periods. The trained n parameterization can be coupled with traditional models to improve national-scale hydrologic flood simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Transition metal impurities in silicon: computational search for a semiconductor qubit

Abstract Semiconductors offer a promising platform for physical implementation of qubits, but their broad adoption is presently hindered by limited scalability and/or very low operating temperatures. Learning from the nitrogen-vacancy centers in diamond, our goal is to find equivalent optically active point defect centers in crystalline silicon, which could be advantageous for their scalability and integration with classical devices. Transition metal (TM) impurities in silicon are common paramagnetic deep defects, but a comprehensive theoretical study of the whole 3 d series that considers generalized Koopmans’ condition is missing. We apply the HSE06(+U) method to examine their potential as optically active spin qubits and identify seven TM impurities that have optically allowed triplet–triplet transitions within the silicon band gap. These results provide the first step toward silicon-based qubits with higher operating temperatures for quantum sensing. Additionally, these point defects could lead to spin-photon interfaces in silicon-based qubits and devices for mid-infrared free-space communications.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Deblending galaxies with variational autoencoders: A joint multiband, multi-instrument approach

ABSTRACT Blending of galaxies has a major contribution in the systematic error budget of weak-lensing studies, affecting photometric and shape measurements, particularly for ground-based, deep, photometric galaxy surveys, such as the Rubin Observatory Legacy Survey of Space and Time (LSST). Existing deblenders mostly rely on analytic modelling of galaxy profiles and suffer from the lack of flexible yet accurate models. We propose to use generative models based on deep neural networks, namely variational autoencoders (VAE), to learn probabilistic models directly from data. We train a VAE on images of centred, isolated galaxies, which we reuse, as a prior, in a second VAE-like neural network in charge of deblending galaxies. We train our networks on simulated images including six LSST bandpass filters and the visible and near-infrared bands of the Euclid satellite, as our method naturally generalizes to multiple bands and can incorporate data from multiple instruments. We obtain median reconstruction errors on ellipticities and r-band magnitude between ±0.01 and ±0.05, respectively, in most cases, and ellipticity multiplicative bias of 1.6 per cent for blended objects in the optimal configuration. We also study the impact of decentring and prove the method to be robust. This method only requires the approximate centre of each target galaxy, but no assumptions about the number of surrounding objects, pointing to an iterative detection/deblending procedure we leave for future work. Finally, we discuss future challenges about training on real data and obtain encouraging results when applying transfer learning.

Arcelin, Bastien↗
DOE OSTI2024

Stochastic representation and conditioning of process-based geological model by deep generative and recognition networks

Accurate and realistic geological modeling is the core of oil and gas development and production. In recent years, process-based methods are developed to produce highly realistic geological models by simulating the physical processes that reproduce the sedimentary events and develop the geometry. However, the complex dynamic processes are extremely expensive to simulate, making process-based models difficult to be conditioned to field data. In this work, we propose a comprehensive generative adversarial network framework as a machine-learning-assisted approach for mimicking the outputs of process-based geological models with fast generation. The main objective of our work is to obtain a continuous parametrization of the highly realistic process-based geological models which enables us to calibrate the models and condition the models to data. Numerical results are presented to illustrate the capability of our proposed methodology.

58 GEOSCIENCES↗
NASA NTRS2021

Applying the Cognitive Space Gateway to Swarm Topologies

NASA's future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize routing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, CubeSat swarm topologies, and cloud services. To test the algorithm in a realistic scenario, the emulated network topology is based on a CubeSat swarm. The swarm may function as a mesh of nodes or as a hub-and-spoke network. An emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. The cloud environment may enable a flexible, lower maintenance approach versus a multi-hop network based in a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent↗
NASA NTRSDate not supplied

Applying the Cognitive Space Gateway to Swarm Topologies

NASA’s future vision for interplanetary networking includes a lunar network, Cube Satellite (CubeSat) constellations, and deep space robotic missions, comprising what could be viewed as a network of networks. Delay-tolerant networking (DTN) architecture and protocols provide a standard network layer among these varying scenarios and mitigate many challenges of the space environment, such as long delays, unplanned service interruptions, and asymmetric links. The Cognitive Space Gateway (CSG) is a routing method in a DTN architecture that uses spiking neural networks as the learning element to optimize outing decisions in a complex environment. This work aims to further develop cognitive networking technologies in several critical areas, including DTN, the CSG algorithm, SmallSat swarm topologies, and cloud services. The CSG algorithm is tested in a realistic scenario in which the emulated network topology is based on a SmallSat swarm. The emulation environment will be built upon a commercial cloud service, such as Amazon Web Services (AWS) Elastic Compute Cloud. This work investigates the ability of such a platform to enable a flexible, lower maintenance approach to creating a multihop network outside of a physical laboratory. The cloud platform will provide a secure environment allowing for collaboration among government and academic entities.

Ricardo Lent↗
DOE OSTI2022

Deep compressed seismic learning for fast location and moment tensor inferences with natural and induced seismicity

Fast detection and characterization of seismic sources is crucial for decision-making and warning systems that monitor natural and induced seismicity. However, besides the laying out of ever denser monitoring networks of seismic instruments, the incorporation of new sensor technologies such as Distributed Acoustic Sensing (DAS) further challenges our processing capabilities to deliver short turnaround answers from seismic monitoring. In response, this work describes a methodology for the learning of the seismological parameters: location and moment tensor from compressed seismic records. In this method, data dimensionality is reduced by applying a general encoding protocol derived from the principles of compressive sensing. The data in compressed form is then fed directly to a convolutional neural network that outputs fast predictions of the seismic source parameters. Thus, the proposed methodology can not only expedite data transmission from the field to the processing center, but also remove the decompression overhead that would be required for the application of traditional processing methods. An autoencoder is also explored as an equivalent alternative to perform the same job. We observe that the CS-based compression requires only a fraction of the computing power, time, data and expertise required to design and train an autoencoder to perform the same task. Implementation of the CS-method with a continuous flow of data together with generalization of the principles to other applications such as classification are also discussed.

54 ENVIRONMENTAL SCIENCES↗