Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Practical galaxy morphology tools from deep supervised representation learning

Astronomers have typically set out to solve supervised machine learning problems by creating their own representations from scratch. We show that deep learning models trained to answer every Galaxy Zoo DECaLS question learn meaningful semantic representations of galaxies that are useful for new tasks on which the models were never trained. We exploit these representations to outperform several recent approaches at practical tasks crucial for investigating large galaxy samples. The first task is identifying galaxies of similar morphology to a query galaxy. Given a single galaxy assigned a free text tag by humans (e.g. ‘#diffuse’), we can find galaxies matching that tag for most tags. The second task is identifying the most interesting anomalies to a particular researcher. Our approach is 100 per cent accurate at identifying the most interesting 100 anomalies (as judged by Galaxy Zoo 2 volunteers). The third task is adapting a model to solve a new task using only a small number of newly labelled galaxies. Models fine-tuned from our representation are better able to identify ring galaxies than models fine-tuned from terrestrial images (ImageNet) or trained from scratch. We solve each task with very few new labels; either one (for the similarity search) or several hundred (for anomaly detection or fine-tuning). This challenges the longstanding view that deep supervised methods require new large labelled data sets for practical use in astronomy. To help the community benefit from our pretrained models, we release our fine-tuning code zoobot. Zoobot is accessible to researchers with no prior experience in deep learning.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Deep Learning and Likelihood Approaches for Viral Phylogeography Converge on the Same Answers Whether the Inference Model Is Right or Wrong

Abstract Analysis of phylogenetic trees has become an essential tool in epidemiology. Likelihood-based methods fit models to phylogenies to draw inferences about the phylodynamics and history of viral transmission. However, these methods are often computationally expensive, which limits the complexity and realism of phylodynamic models and makes them ill-suited for informing policy decisions in real-time during rapidly developing outbreaks. Likelihood-free methods using deep learning are pushing the boundaries of inference beyond these constraints. In this paper, we extend, compare, and contrast a recently developed deep learning method for likelihood-free inference from trees. We trained multiple deep neural networks using phylogenies from simulated outbreaks that spread among 5 locations and found they achieve close to the same levels of accuracy as Bayesian inference under the true simulation model. We compared robustness to model misspecification of a trained neural network to that of a Bayesian method. We found that both models had comparable performance, converging on similar biases. We also implemented a method of uncertainty quantification called conformalized quantile regression that we demonstrate has similar patterns of sensitivity to model misspecification as Bayesian highest posterior density (HPD) and greatly overlap with HPDs, but have lower precision (more conservative). Finally, we trained and tested a neural network against phylogeographic data from a recent study of the SARS-Cov-2 pandemic in Europe and obtained similar estimates of region-specific epidemiological parameters and the location of the common ancestor in Europe. Along with being as accurate and robust as likelihood-based methods, our trained neural networks are on average over 3 orders of magnitude faster after training. Our results support the notion that neural networks can be trained with simulated data to accurately mimic the good and bad statistical properties of the likelihood functions of generative phylogenetic models.

Evolutionary Biology↗
DOE OSTI2026

Dark Energy Survey Year 3 results: Simulation-based 𝑤CDM inference from weak lensing and galaxy clustering maps with deep learning: Analysis design

Data-driven approaches using deep learning are emerging as powerful techniques to extract non-Gaussian information from cosmological large-scale structure. Here, this work presents the first simulation-based inference (SBI) pipeline that combines weak lensing and galaxy clustering maps in a realistic Dark Energy Survey Year 3 (DES Y3) configuration and serves as preparation for a forthcoming analysis of the survey data. We develop a scalable forward model based on the CosmoGridV1 suite of N-body simulations to generate over one million self-consistent mock realizations of DES Y3 at the map level. Leveraging this large dataset, we train deep graph convolutional neural networks on the full survey footprint in spherical geometry to learn low-dimensional features that approximately maximize mutual information with target parameters. These learned compressions enable neural density estimation of the implicit likelihood via normalizing flows in a ten-dimensional parameter space spanning cosmological 𝑤CDM, intrinsic alignment, and linear galaxy bias parameters, while marginalizing over baryonic, photometric redshift, and shear bias nuisances. To ensure robustness, we extensively validate our inference pipeline using synthetic observations derived from both systematic contaminations in our forward model and independent Buzzard galaxy catalogs. Our forecasts yield significant improvements in cosmological parameter constraints, achieving 2−3× higher figures of merit in the 𝛺 𝑚 − 𝑆 8 plane relative to our implementation of baseline two-point statistics and effectively breaking parameter degeneracies through probe combination. These results demonstrate the potential of SBI analyses powered by deep learning for upcoming Stage-IV wide-field imaging surveys.

Thomsen, A. [Zurich, ETH] (ORCID:0000000203099021)↗
DOE OSTI2026

Semi-inclusive deep-inelastic scattering on a polarized spin-1 target. I. Cross section and spin observables

We develop the theoretical framework for semi-inclusive deep-inelastic scattering on a polarized spin-1 target and apply it to scattering on the polarized deuteron with spectator nucleon tagging. In Part I (this article) we present the general form of the semi-inclusive cross section and polarization observables for the spin-1 target. A relativistically covariant formulation in terms of four-vectors and invariant polarization parameters is employed. The target polarization is described by a spin density matrix with vector and tensor polarization. The spin and azimuthal angle dependence of the semi-inclusive cross section is derived and parametrized in terms of invariant structure functions. To validate the result, the structure functions are expressed as photon-target helicity amplitudes with known symmetry properties. The expressions presented here are kinematic (no assumptions about particle production dynamics) and valid in all regions of the deep-inelastic final state (current and target fragmentation regions). In Part II (following article), we consider deep-inelastic scattering on the polarized deuteron with spectator nucleon tagging as a special case of target fragmentation. The semi-inclusive structure functions are computed by separating nuclear and hadronic structure, and the polarization observables are explored as functions of the tagged nucleon momentum.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Coherence-Induced Deep Thermalization Transition in Random Permutation Quantum Dynamics

We report a phase transition in the projected ensemble—the collection of postmeasurement wave functions of a local subsystem obtained by measuring its complement. The transition emerges in systems undergoing random permutation dynamics, a type of quantum time evolution wherein computational basis states are shuffled without creating superpositions. It separates a phase exhibiting deep thermalization, where the projected ensemble is distributed over Hilbert space in a maximally entropic fashion (Haar random), from a phase where it is minimally entropic (“classical bit-string ensemble”). Crucially, this deep thermalization transition is invisible to the subsystem’s density matrix, which always exhibits thermalization to infinite temperature across the phase diagram. Through a combination of analytical arguments and numerical simulations, we show that the transition is tuned by the total amount of injected by the input state and the measurement basis, and is exhibited robustly across different microscopic models. Our findings represent a novel form of ergodicity-breaking universality in quantum many-body dynamics, characterized not by a failure of regular thermalization, but rather by a failure of deep thermalization.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Isotope effects in molecular structures and electronic properties of liquid water via deep potential molecular dynamics based on the SCAN functional

Here, Feynman path-integral deep potential molecular dynamics (PI-DPMD) calculations have been employed to study both light (H 2 O) and heavy water (D 2 O) within the isothermalisobaric ensemble. In particular, the deep neural network is trained based on ab initio data obtained from the strongly constrained and appropriately normed (SCAN) exchange-correlation functional. Because of the lighter mass of hydrogen than deuteron, the properties of light water are more influenced by nuclear quantum effect than those of heavy water. Clear isotope effects are observed and analyzed in terms of hydrogen-bond structure and electronic properties of water that are closely associated with experimental observables. The molecular structures of both liquid H 2 O and D 2 O agree well with the data extracted from scattering experiments. The delicate isotope effects on radial distribution functions and angular distribution functions are well reproduced as well. Our approach demonstrates that deep neural network combined with SCAN functional based ab initio molecular dynamics provides an accurate theoretical tool for modeling water and its isotope effects.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Effects of deep superconducting gap minima and disorder on residual thermal transport in Sr 2 RuO 4

Recent thermal conductivity measurements on Sr 2 RuO 4 were interpreted as favoring a pairing gap function with vertical line nodes while conflicting with chiral p-wave pairing. Motivated by this work we study the effects of deep superconducting gap minima on impurity induced quasiparticle thermal transport in chiral p-wave models of Sr 2 RuO 4 . Combining a self-consistent T-matrix analysis and self-consistent Bogoliubov–de Gennes calculations, we show that the dependence of the residual thermal conductivity on the normal state impurity scattering rate can be quite similar to the d-wave pairing state that was shown to fit the thermal conductivity measurements, provided the normal state impurity scattering rate is large compared with the deep gap minima. Furthermore, thermal conductivity measurements on Sr 2 RuO 4 can be reconciled with a chiral p-wave pairing state with deep gap minima. However, the data impose serious constraints on such models and these constraints are examined in the context of several different chiral p-wave models.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2015

Interplay of threshold resummation and hadron mass corrections in deep inelastic processes

We discuss hadron mass corrections and threshold resummation for deep-inelastic scattering lN-->l'X and semi-inclusive annihilation e + e - → hX processes, and provide a prescription how to consistently combine these two corrections respecting all kinematic thresholds. We find an interesting interplay between threshold resummation and target mass corrections for deep-inelastic scattering at large values of Bjorken x B . In semi-inclusive annihilation, on the contrary, the two considered corrections are relevant in different kinematic regions and do not affect each other. A detailed analysis is nonetheless of interest in the light of recent high precision data from BaBar and Belle on pion and kaon production, with which we compare our calculations. For both deep inelastic scattering and single inclusive annihilation, the size of the combined corrections compared to the precision of world data is shown to be large. Therefore, we conclude that these theoretical corrections are relevant for global QCD fits in order to extract precise parton distributions at large Bjorken x B , and fragmentation functions over the whole kinematic range.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Adaptive Neuron Apoptosis for Accelerating Deep Learning on Large Scale Systems

Machine Learning and Data Mining (MLDM) algorithms are becoming ubiquitous in {\em model learning} from the large volume of data generated using simulations, experiments and handheld devices. Deep Learning algorithms -- a class of MLDM algorithms -- are applied for automatic feature extraction, and learning non-linear models for unsupervised and supervised algorithms. Naturally, several libraries which support large scale Deep Learning -- such as TensorFlow and Caffe -- have become popular. In this paper, we present novel techniques to accelerate the convergence of Deep Learning algorithms by conducting low overhead removal of redundant neurons -- {\em apoptosis} of neurons -- which do not contribute to model learning, during the training phase itself. We provide in-depth theoretical underpinnings of our heuristics (bounding accuracy loss and handling apoptosis of several neuron types), and present the methods to conduct adaptive neuron apoptosis. We implement our proposed heuristics with the recently introduced TensorFlow and using its recently proposed extension with MPI. Our performance evaluation on two difference clusters -- one connected with Intel Haswell multi-core systems, and other with nVIDIA GPUs -- using InfiniBand, indicates the efficacy of the proposed heuristics and implementations. Specifically, we are able to improve the training time for several datasets by 2-3x, while reducing the number of parameters by 30x (4-5x on average) on datasets such as ImageNet classification. For the Higgs Boson dataset, our implementation improves the accuracy (measured by Area Under Curve (AUC)) for classification from 0.88/1 to 0.94/1, while reducing the number of parameters by 3x in comparison to existing literature, while achieving a 2.44x speedup in comparison to the default (no apoptosis) algorithm.

Siegel, Charles M.↗
DOE OSTI2019

Visualization System for Evolutionary Neural Networks for Deep Learning

Deep learning is actively used in a wide range of fields for scientific discovery. To effectively apply deep learning to a particular problem, it is important to select an appropriate network architecture and other hyper-parameters (at each layer). Evolving architectures and hyper-parameters using a genetic algorithm is one current approach to search the huge space of all possible configurations to find those more optimal for the problem. However, examining an evolutionary process and tuning the genetic algorithm are challenging, pushing most users to treat the process as a black box. To address this challenge, we propose a visualization system for evolutionary neural networks for deep learning. The key feature of our visualization system is to provide a visual analytics environment for evaluating a genetic algorithm in order to improve the underlying operations to reduce time to find good solutions. Our system is able to not only visualize how a genetic algorithm traverses its search space but also allows users to examine evolving networks in-depth to get insights to improve performance through interactive visualization components.

Chae, Junghoon↗
DOE OSTI2021

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗
DOE OSTI2024

Deep Multi-Agent Reinforcement Learning for Real-World Signalized Traffic Corridor Control

Signalized traffic control problem has been addressed recently with deep Reinforcement Learning (RL) approaches involving diverse state, action, and reward structures. While significant progress has been noted in the literature, open challenges still remain in the areas of adaptive signal phase timing, coordination in a multi-intersection corridor setting, and consideration of real-world traffic conditions. In the context of deep RL-based problem framing, extensions are needed that enable adaptive signal phase timings in an intersection agent's action space, computationally efficient information sharing among neighboring signalized intersection agents along a corridor, and experimentation in realistic simulation environments. In this paper, we develop a deep Advantage Actor Critic (A2C) multi-agent RL (MARL) approach capturing the research extensions above and apply it within a real-world calibrated Aimsun Next traffic corridor simulation model based on traffic data from the City of Coral Gables, Florida. For a multi-intersection corridor control setting, our numerical simulation experiments with a decentralized A2C MARL algorithm applied at different time periods led to a total average corridor travel delay reduction (expressed in seconds/mile averaged over vehicles) from 4.9% to 19.9% compared to state-of-the-art actuated control.

Shuvo, Salman S. [BATTELLE (PACIFIC NW LAB)]↗
DOE OSTI2017

Vertex reconstruction of neutrino interactions using deep learning

Deep learning offers new tools to improve our understanding of many important scientific problems. Neutrinos are the most abundant particles in existence and are hypothesized to explain the matter-antimatter asymmetry that dominates our universe. Definitive tests of this conjecture require a detailed understanding of neutrino interactions with a variety of nuclei. Many measurements of interest depend on vertex reconstruction -- finding the origin of a neutrino interaction using data from the detector, which can be represented as images. Traditionally, this has been accomplished by utilizing methods that identify the tracks coming from the interaction. However, these methods are not ideal for interactions where an abundance of tracks and cascades occlude the vertex region. Manual algorithm engineering to handle these challenges is complicated and error prone. Deep learning extracts rich, semantic features directly from raw data, making it a promising solution to this problem. In this work, deep learning models are presented that classify the vertex location in regions meaningful to the domain scientists improving their ability to explore more complex interactions.

Terwilliger, Adam↗
DOE OSTI2022

Strategies for Integrating Deep Learning Surrogate Models with HPC Simulation Applications

The emerging trend of the convergence of high performance computing (HPC), machine learning/deep learning (ML/DL), and big data analytics presents a host of challenges for large-scale computing campaigns that seek best practices to interleave traditional scientific simulation-based workloads with ML/DL models. A portfolio of systematic approaches to incorporate deep learning into modeling and simulation serves a vital need when we support AI for science at a computing facility. In this paper, we evaluate several strategies for deploying deep learning surrogate models in a representative physics application on supercomputers at the Oak Ridge Leadership Computing Facility (OLCF). We discuss a set of recommended deployment architectures and implementation approaches. We analyze and evaluate these alternatives and show their performance and scalability up to 1000 GPUs on two mainstream platforms equipped with different deep learning hardware and software stacks.

Yin, Junqi↗
DOE OSTI2022

Volt-VAR Optimization in Distribution Networks Using Twin Delayed Deep Reinforcement Learning

Modern distribution grids are undergoing new challenges due to the stochastic nature of distributed energy resources (DERs). High penetration of DERs has a significant impact on Volt-VAR profile and system power losses. This work proposes a deep reinforcement learning (DRL)-based Volt-VAR optimization approach for improving voltage profile and reducing system power loss under high penetration of distributed energy resources, such as battery energy storage and solar photovoltaic units in distribution grids. The twin delayed deep deterministic policy gradient (TD3) method-based DRL agent is proposed to configure optimal set-points of reactive power outputs of fast responding smart inverters. The agent schedules the reactive power of inverters according to their physical capabilities, such as minimum allowed power factor, e.g., 0.9 leading/lagging. The reward function of the proposed DRL scheme is designed carefully to ensure a proper voltage profile of the grids with effective scheduling of reactive power outputs from inverters. The performance of the proposed model is verified on modified IEEE 34- and 123-bus systems and compared with base case with no reactive supply by inverters, and local droop Volt-VAR control approach. The results show that the proposed method performs better than the local droop control and deep deterministic policy gradient (DDPG)-based DRL method for reducing voltage fluctuation and minimizing power loss.

Hossain, Rakib↗
DOE OSTI2019

Transmission Line Fault Location Using Deep Learning Techniques

This letter presents a novel single-ended fault location approach for transmission lines that uses modern deep learning methods. A mixed convolutional neural network and long short-term memory structure are proposed to preserve the translation invariance and capture the temporal correlation of the time-series input data. Advanced deep learning techniques such as adaptive moment estimation and dropout are used to efficiently train the neural network and prevent over-fitting. Our extensive studies have demonstrated the accuracy and effectiveness of the deep-learning-based, singled-ended fault location approach.

Fan, Rui↗
DOE OSTI2024

Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs

The future of grid control requires a hybrid approach combining centralized and decentralized methods to fully utilize the potential of smart edge devices with artificial intelligence (AI) capabilities. This paper aims to develop and evaluate a federated deep reinforcement learning (FDRL) framework for decentralized adaptive volt-var optimization (VVO) of behind-the-meter (BTM) distributed energy resources (DERs). First, this paper models a single deep reinforcement learning (DRL) agent using the Markov Decision Process (MDP) framework for decentralized adaptive VVO of BTM DERs. Two DRL algorithms, soft actor-critic (SAC) and twin-delayed deep deterministic policy gradient (TD3), are compared for their effectiveness in optimizing VVO. Results show that TD3 outperforms SAC, achieving a 71.3% improvement in mean reward. Finally, the DRL agent is deployed within the FDRL framework, using the Flower platform, to enhance learning, provide adaptive control, and ensure data privacy for BTM DERs.

Ravi, Abhijith↗
DOE OSTI2019

Performance Analysis of Deep Learning Workloads on Leading-edge Systems

This work examines the performance of leading-edge systems designed for machine learning computing, including the NVIDIA DGX-2, Amazon Web Services (AWS) P3, IBM Power System Accelerated Compute Server AC922, and a consumer-grade Exxact TensorEX TS4 GPU server. Representative deep learning workloads from the fields of computer vision and natural language processing are the focus of the analysis. Performance analysis is performed along with a number of important dimensions. Performance of the communication interconnects and large and high-throughput deep learning models are considered. Different potential use models for the systems as standalone and in the cloud also are examined. The effect of various optimization of the deep learning models and system configurations is included in the analysis.

97 MATHEMATICS AND COMPUTING↗