SEARCH · Engineering Papers

Results for “deep learning methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,513 records · Page 84

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Exploration with Scalable Gaussian Process Reinforcement Learning

Exploration is a challenging problem in reinforcement learning (RL), especially in environments with sparse rewards. Quantifying and utilizing the parametric uncertainty has been shown to be paramount for successful exploration [Osband et al., 2018]. Bayesian, or approximately Bayesian, methods present a principled means of estimating the parametric uncertainty in RL problems. Gaussian processes, nonparametric Bayesian models, are often impractical due to poor scalability and computational bottlenecks. We introduce a scalable Gaussian process RL (GPRL) method which directly induces sparsity in the covariance matrix to facilitate faster computation. This is a departure from previous GPRL methods which instead rely on data reduction and subsampling. We compare various covariance-based exploration techniques (Thompson sampling, upper confidence bound, and probabilistic maximum variance) which leverage our scalable GP framework in sparse reward environments. Finally, we show favorable comparison against the bootstrapped deep Q-Network.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Computer-aided Abnormality Detection in Chest Radiographs in a Clinical Setting via Domain-adaptation

Deep learning (DL) models are being deployed at medical centers to aid radiologists for diagnosis of lung conditions from chest radiographs. Such models are often trained on a large volume of publicly available labeled radiographs. These pre-trained DL models’ ability to generalize in clinical settings is poor because of the changes in data distributions between publicly available and privately held radiographs. In chest radiographs, the heterogeneity in distributions arises from the diverse conditions in X-ray equipment and their configurations used for generating the images. In the machine learning community, the challenges posed by the heterogeneity in the data generation source is known as domain shift, which is a mode shift in the generative model. In this work, we introduce a domain-shift detection and removal method to overcome this problem. Our experimental results show the proposed method’s effectiveness in deploying a pre-trained DL model for abnormality detection in chest radiographs in a clinical setting.

Dubey, Abhishek↗

NASA NTRS2010

The Heliophysics Data Environment, Virtual Observatories, NSSDC, and SPASE

Heliophysics (the study of the Sun and its effects on the Solar System, especially the Earth) has an interesting data environment in that the data are often to be found in relatively small data sets widely scattered in archives around the world. Within the last decade there have been more concentrated efforts to organize the data access methods and create a Heliophysics Data and Model Consortium (HDMC). To provide data search and access capability a number of Virtual Observatories (VO's) have been established both via funding from the U.S. National Aeronautics and Space Administration (NASA) and through other funding agencies in the U.S. and worldwide. At least 15 systems can be labeled as Heliophysics Virtual Observatories, 9 of them funded by NASA. Other parts of this data environment include Resident Archives, and the final, or "deep" archive at the National Space Science Data Center (NSSDC). The problem is that different data search and access approaches are used by all of these elements of the HDMC and a search for data relevant to a particular research question can involve consulting with multiple VO's - needing to learn a different approach for finding and acquiring data for each. The Space Physics Archive Search and Extract (SPASE) project is intended to provide a common data model for Heliophysics data and therefore a common set of metadata for searches of the VO's and other data environment elements. The SPASE Data Model has been developed through the common efforts of the HDMC representatives over a number of years. We currently have released Version 2.1. of the Data Model. The advantages and disadvantages of the Data Model will be discussed along with the plans for the future. Recent changes requested by new members of the SPASE community indicate some of the directions for further development.

Thieman, James↗

DOE OSTI2022

Anderson Acceleration for Distributed Training of Deep Learning Models

Anderson acceleration (AA) is an extrapolation technique that has recently gained interest in the deep learning (DL) community to speed-up the sequential training of DL models. However, when performed at large scale, the DL training is exposed to a higher risk of getting trapped into steep local minima of the training loss function, and standard AA does not provide sufficient acceleration to escape from these steep local minima. This results in poor generalizability and makes AA ineffective. To restore AA’s advantage to speed-up the training of DL models on large scale computing platforms, we combine AA with an adaptive moving average procedure that boosts the training to escape from steep local minima. By monitoring the relative standard deviation between consecutive iterations, we also introduce a criterion to automatically assess whether the moving average is needed. We applied the method to the following DL instantiations for image classification: (i) ResNet50 trained on the open-source CIFAR100 dataset and (ii) ResNet50 trained on the open-source ImageNet1k dataset. Numerical results obtained using up to 1,536 NVIDIA V100 GPUs on the OLCF supercomputer Summit showed the stabilizing effect of the moving average on AA for all the problems above.

Lupo Pasini, Massimiliano↗

DOE OSTI2021

The deep-DRT: A deep neural network approach to deconvolve the distribution of relaxation times from multidimensional electrochemical impedance spectroscopy data

Electrochemical impedance spectroscopy (EIS) is an experimental technique ubiquitously used to study electrochemical systems. However, conventional EIS data interpretation through physical and equivalent circuit models is challenging because physical models are problem-specific, and equivalent circuits are often just lumped-element analogs lacking physical meaning. The distribution of relaxation times (DRT) has emerged as a complementary approach to resolve these issues. One drawback of conventional DRT deconvolution is that the EIS data is understood to be (only) a function of frequency (i.e. 1D data) and deconvolved accordingly. This work proposes a novel deconvolution method based on deep neural networks (DNNs), allowing the analysis of multidimensional EIS spectra to bridge data dependency on both frequencies and experimental conditions. Two particularly appealing traits of the deep-DRT method developed in this article are that neither regularization nor specific spacing on the state variables defining the experiment are required. Finally, leveraging DNN to examine complex EIS spectra and their dependence on experimental conditions, this work opens a new research direction in the area of EIS analysis and DRT deconvolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

DCTRGAN: improving the precision of generative models with reweighting

Significant advances in deep learning have led to more widely used and precise neural network-based generative models such as Generative Adversarial Networks (Gans). We introduce a post-hoc correction to deep generative models to further improve their fidelity, based on the Deep neural networks using the Classification for Tuning and Reweighting (Dctr) protocol. The correction takes the form of a reweighting function that can be applied to generated examples when making predictions from the simulation. We illustrate this approach using Gans trained on standard multimodal probability densities as well as calorimeter simulations from high energy physics. We show that the weighted Gan examples significantly improve the accuracy of the generated samples without a large loss in statistical power. This approach could be applied to any generative model and is a promising refinement method for high energy physics applications and beyond.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Machine Learning Assisted HPC Workload Trace Generation for Leadership Scale Storage Systems

Monitoring and analyzing a wide range of I/O activities in an HPC cluster is important in maintaining mission-critical performance in a large-scale, multi-user, parallel storage system. Center-wide I/O traces can provide high-level information and fine-grained activities per application or per user running in the system. Studying such large-scale traces can provide helpful insights into the system. It can be used to develop predictive methods for making predictive decisions, adjusting scheduling policies, or providing decisions for the design of next-generation systems. However, sharing real-world I/O traces to expedite such research efforts leaves a few concerns; i) the cost of sharing the large traces is expensive due to this large size, and ii) privacy concern is an issue.We address such issues by building an end-to-end machine learn- ing (ML) workflow that can generate I/O traces for large-scale HPC applications. We leverage ML based feature selection and gener- ative models for I/O trace generation. The generative models are trained on I/O traces collected by the darshan I/O characterization tool over a period of one year. We present a two-step generation process consisting of two deep-learning models, called the feature generator and the trace generator. The combination of two-step generative models provides robustness by reducing the bias of the model and accounting for the stochastic nature of the I/O traces across different runs of an application. We evaluate the performance of the generative models and show that the two-step model can generate time-series I/O traces with less than 20% root mean square error.

Paul, Arnab↗

DOE OSTI2025

Deep learning of structural morphology imaged by scanning X-ray diffraction microscopy

Scanning X-ray nanodiffraction microscopy is a powerful technique for spatially resolving nanoscale structural morphologies by diffraction contrast. One of the critical challenges in experimental nanodiffraction data analysis is posed by the convergence angle of nanoscale focusing optics which creates simultaneous dependency of the far-field scattering data on three independent components of the local strain tensor-corresponding to dilation and two potential rigid body rotations of the unit cell. All three components are in principle resolvable through a spatially mapped sample tilt series; however, traditional data analysis is computationally expensive and prone to artifacts. In this study, we implement NanobeamNN, a convolutional neural network specifically tailored to the analysis of scanning probe X-ray microscopy data. NanobeamNN learns lattice strain and rotation angles from simulated diffraction of a focused X-ray nanobeam by an epitaxial thin film and can directly make reasonable predictions on experimental data without the need for additional fine-tuning. We demonstrate that this approach represents a significant advancement in computational speed over conventional methods, as well as a potential improvement in accuracy over the current standard.

Luo, Aileen [Cornell Univ., Ithaca, NY (United Sta↗

NASA NTRS2021

Unsupervised Deep Persistent Monocular Visual Odometry and Depth Estimation in Extreme Environments

In recent years, unsupervised deep learning ap-proaches have received a significant attention to estimate depthand visual odometry (VO) from unlabelled monocular imagesequences. However, their performance is limited in challengingenvironments due to perceptual degradation, occlusions andrapid motions. Moreover, the existing unsupervised methodssuffer from the lack of scale-consistency constraints acrossframes, which causes that the VO estimators fail to providepersistent trajectories over long sequences. In this study, wepropose a unsupervised monocular deep VO framework thatpredicts 6 degrees-of-freedom pose camera motion and depthmap of the scene from unlabelled RGB image sequences.We provide detailed quantitative and qualitative evaluationsof the proposed framework on a) a challenging dataset col-lected during the DARPA Subterranean challenge1; and b)the benchmark KITTI and Cityscapes datasets. The proposedapproach outperforms both traditional and state-of-the-artunsupervised deep VO methods providing better results for bothpose estimation and depth recovery. The presented approach ispart of the solution used by the COSTAR team participatingat the DARPA Subterranean Challenge

Agha-mohammadi, Ali-akbar↗

DOE OSTI2022

A Hybrid Reinforcement Learning-MPC Approach for Distribution System Critical Load Restoration: Preprint

This paper proposes a hybrid control approach for distribution system critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while MPC models grid operations incorporating RL policy actions, i.e., the reserve requirement, renewable (wind and solar) power predictions, and load demand forecasts. We formulate the reserve requirement determination problem as a sequential decision making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The RL algorithm is trained off-line using historical forecast of renewable generation and load demand. The method is tested using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine and battery. Case studies demonstrated that the proposed method outperforms other operating reserve determination methods.

distribution system↗

DOE OSTI2022

A Hybrid Reinforcement Learning-MPC Approach for Distribution System Critical Load Restoration

This paper proposes a hybrid control approach for distribution system critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while MPC models grid operations incorporating RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. We formulate the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The RL algorithm is trained offline using a historical forecast of renewable generation and load demand. The method is tested using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery. Case studies demonstrated that the proposed method outperforms other policies with static operating reserves.

distribution system↗

NASA NTRSDate not supplied

Towards Autonomous Lunar Resource Excavation via Deep Reinforcement Learning

To support sustainable infrastructure on the Moon, NASA needs to leverage lunar resources for in-situ processing and construction. NASA’s Regolith Advanced Surface Systems Operations Robot (RASSOR) is principally designed to mine and deliver regolith for these tasks. To reliably perform these operations on the lunar surface, RASSOR's sensors and control systems need to be robust and maximize information extracted from a reduced sensor payload. Herein, we present our findings from the Intelligent Capabilities Enhanced RASSOR project. We created reduced-order simulation environments in which we applied reinforcement learning algorithms to learn autonomous trenching controllers and produced state estimation architectures. We developed two simulations: a 2D excavation simulation used to facilitate parameter selection, and a 3D simulation developed using a game physics engine to simulate simplified soil interactions and incorporate robotic agents parameterized by dynamic models. Within these simulations, we learned autonomous excavation routines that exceed excavation efficiency measures as compared against RASSOR's existing control and teleoperation-based methods.

RASSOR↗

NASA NTRSDate not supplied

Towards Autonomous Lunar Resource Excavation via Deep Reinforcement Learning

RASSOR↗

DOE OSTI2024

Scalar bounded-from-below conditions from Bayesian active learning

We present a procedure leveraging Bayesian deep active learning to rapidly produce highly accurate approximate bounded-from-below conditions for arbitrary renormalizable scalar potentials, in the form of a neural network which may be saved and exported for use in arbitrary parameter space scans. We explore the performance of our procedure on three different scalar potentials with either highly nontrivial or unknown symbolic bounded-from-below conditions (the most general two-Higgs doublet model, the three-Higgs doublet model, and a version of the Georgi-Machacek model without custodial symmetry). We find that we can produce fast and highly accurate binary classifiers for all three potentials. Furthermore, for the potentials for which no known symbolic necessary and sufficient conditions on boundedness-from-below exist, our classifiers substantially outperform some common approximate analytical methods, such as producing tractable sufficient but not necessary conditions or evaluating boundedness-from-below conditions for scenarios in which only a subset of the theory’s fields achieve vacuum expectation values. Our methodology can be readily adapted to any renormalizable scalar field theory. For the community’s use, we have developed a package, BFBrain, which allows for the rapid implementation of our analysis procedure on user-specified scalar potentials with a high degree of customizability. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

DOE OSTI2025

Real-time inference and extrapolation with Time-Conditioned UNet: Applications in hypersonic flows, incompressible flows, and global temperature forecasting

Neural Operators are fast and accurate surrogates for nonlinear mappings between functional spaces within training domains. Extrapolation beyond the training domain remains a grand challenge across all application areas. We present Time-Conditioned UNet (TC-UNet) as an operator learning method to solve time-dependent PDEs continuously in time without any temporal discretization, including in extrapolation scenarios. TC-UNet incorporates the temporal evolution of the PDE into its architecture by combining a parameter conditioning approach with the attention mechanism from the Transformer architecture. After training, TC-UNet makes real-time inferences on an arbitrary temporal grid. We demonstrate its extrapolation capability on a climate problem by estimating the global temperature for several years and also for inviscid hypersonic flow around a double cone. We propose different training strategies involving temporal bundling and sub-sampling. We demonstrate performance improvements for several benchmarks, performing extrapolation for long time intervals and zero-shot super-resolution time.

Deep learning↗

DOE OSTI2023

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

Variational Neural-Network Ansatz for Continuum Quantum Field Theory

Physicists dating back to Feynman have lamented the difficulties of applying the variational principle to quantum field theories. In nonrelativistic quantum field theories, the challenge is to parametrize and optimize over the infinitely many n-particle wave functions comprising the state’s Fock-space representation. Here we approach this problem by introducing neural-network quantum field states, a deep learning ansatz that enables application of the variational principle to nonrelativistic quantum field theories in the continuum. Our ansatz uses the Deep Sets neural network architecture to simultaneously parametrize all of the n-particle wave functions comprising a quantum field state. We employ our ansatz to approximate ground states of various field theories, including an inhomogeneous system and a system with long-range interactions, thus demonstrating a powerful new tool for probing quantum field theories.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2020

Beam Hardening Artifact Reduction in X-Ray CT Reconstruction of 3D Printed Metal Parts Leveraging Deep Learning and CAD Models

Nondestructive evaluation (NDE) of additively manufactured (AM) parts is important for understanding the impacts of various process parameters and qualifying the built part. X-ray computed tomography (XCT) has played a critical role in rapid NDE and characterization of AM parts. However, XCT of metal AM parts can be challenging because of artifacts produced by standard reconstruction algorithms as a result of a confounding effect called “beam hardening.” Beam hardening artifacts complicate the analysis of XCT images and adversely impact the process of detecting defects, such as pores and cracks, which is key to ensuring the quality of the parts being printed. In this work, we propose a novel framework based on using available computer-aided design (CAD) models for parts to be manufactured, accurate XCT simulations, and a deep-neural network to produce high-quality XCT reconstructions from data that are affected by noise and beam hardening. Using extensive experiments with simulated data sets, we demonstrate that our method can significantly improve the reconstruction quality, thereby enabling better detection of defects compared with the state of the art. We also present promising preliminary results of applying the deep networks trained using CAD models to experimental data obtained from XCT of an AM jet-engine turbine blade.

Ziabari, Amir↗