Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

Isotope effects in molecular structures and electronic properties of liquid water via deep potential molecular dynamics based on the SCAN functional

Here, Feynman path-integral deep potential molecular dynamics (PI-DPMD) calculations have been employed to study both light (H 2 O) and heavy water (D 2 O) within the isothermalisobaric ensemble. In particular, the deep neural network is trained based on ab initio data obtained from the strongly constrained and appropriately normed (SCAN) exchange-correlation functional. Because of the lighter mass of hydrogen than deuteron, the properties of light water are more influenced by nuclear quantum effect than those of heavy water. Clear isotope effects are observed and analyzed in terms of hydrogen-bond structure and electronic properties of water that are closely associated with experimental observables. The molecular structures of both liquid H 2 O and D 2 O agree well with the data extracted from scattering experiments. The delicate isotope effects on radial distribution functions and angular distribution functions are well reproduced as well. Our approach demonstrates that deep neural network combined with SCAN functional based ab initio molecular dynamics provides an accurate theoretical tool for modeling water and its isotope effects.

36 MATERIALS SCIENCE↗

Physisorption of an electron in deep surface potentials off a dielectric surface

We study phonon-mediated adsorption and desorption of an electron at dielectric surfaces with deep polarization-induced surface potentials where multiphonon transitions are responsible for electron energy relaxation. Focusing on multiphonon processes due to the nonlinearity of the coupling between the external electron and the acoustic bulk phonon triggering the transitions between surface states, we calculate electron desorption times for graphite, MgO, CaO, Al{sub 2}O{sub 3}, and SiO{sub 2} and electron sticking coefficients for Al{sub 2}O{sub 3}, CaO, and SiO{sub 2}. To reveal the kinetic stages of electron physisorption, we moreover study the time evolution of the image-state occupancy and the energy-resolved desorption flux. Depending on the potential depth and the surface temperature, we identify two generic scenarios: (i) adsorption via trapping in shallow image states followed by relaxation to the lowest image state and desorption from that state via a cascade through the second strongly bound image state in not too deep potentials, and (ii) adsorption via trapping in shallow image states but followed by a relaxation bottleneck retarding the transition to the lowest image state and desorption from that state via a one-step process to the continuum in deep potentials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Effects of deep superconducting gap minima and disorder on residual thermal transport in Sr 2 RuO 4

Recent thermal conductivity measurements on Sr 2 RuO 4 were interpreted as favoring a pairing gap function with vertical line nodes while conflicting with chiral p-wave pairing. Motivated by this work we study the effects of deep superconducting gap minima on impurity induced quasiparticle thermal transport in chiral p-wave models of Sr 2 RuO 4 . Combining a self-consistent T-matrix analysis and self-consistent Bogoliubov–de Gennes calculations, we show that the dependence of the residual thermal conductivity on the normal state impurity scattering rate can be quite similar to the d-wave pairing state that was shown to fit the thermal conductivity measurements, provided the normal state impurity scattering rate is large compared with the deep gap minima. Furthermore, thermal conductivity measurements on Sr 2 RuO 4 can be reconciled with a chiral p-wave pairing state with deep gap minima. However, the data impose serious constraints on such models and these constraints are examined in the context of several different chiral p-wave models.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Interplay of threshold resummation and hadron mass corrections in deep inelastic processes

We discuss hadron mass corrections and threshold resummation for deep-inelastic scattering lN-->l'X and semi-inclusive annihilation e + e - → hX processes, and provide a prescription how to consistently combine these two corrections respecting all kinematic thresholds. We find an interesting interplay between threshold resummation and target mass corrections for deep-inelastic scattering at large values of Bjorken x B . In semi-inclusive annihilation, on the contrary, the two considered corrections are relevant in different kinematic regions and do not affect each other. A detailed analysis is nonetheless of interest in the light of recent high precision data from BaBar and Belle on pion and kaon production, with which we compare our calculations. For both deep inelastic scattering and single inclusive annihilation, the size of the combined corrections compared to the precision of world data is shown to be large. Therefore, we conclude that these theoretical corrections are relevant for global QCD fits in order to extract precise parton distributions at large Bjorken x B , and fragmentation functions over the whole kinematic range.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Adaptive Neuron Apoptosis for Accelerating Deep Learning on Large Scale Systems

Machine Learning and Data Mining (MLDM) algorithms are becoming ubiquitous in {\em model learning} from the large volume of data generated using simulations, experiments and handheld devices. Deep Learning algorithms -- a class of MLDM algorithms -- are applied for automatic feature extraction, and learning non-linear models for unsupervised and supervised algorithms. Naturally, several libraries which support large scale Deep Learning -- such as TensorFlow and Caffe -- have become popular. In this paper, we present novel techniques to accelerate the convergence of Deep Learning algorithms by conducting low overhead removal of redundant neurons -- {\em apoptosis} of neurons -- which do not contribute to model learning, during the training phase itself. We provide in-depth theoretical underpinnings of our heuristics (bounding accuracy loss and handling apoptosis of several neuron types), and present the methods to conduct adaptive neuron apoptosis. We implement our proposed heuristics with the recently introduced TensorFlow and using its recently proposed extension with MPI. Our performance evaluation on two difference clusters -- one connected with Intel Haswell multi-core systems, and other with nVIDIA GPUs -- using InfiniBand, indicates the efficacy of the proposed heuristics and implementations. Specifically, we are able to improve the training time for several datasets by 2-3x, while reducing the number of parameters by 30x (4-5x on average) on datasets such as ImageNet classification. For the Higgs Boson dataset, our implementation improves the accuracy (measured by Area Under Curve (AUC)) for classification from 0.88/1 to 0.94/1, while reducing the number of parameters by 3x in comparison to existing literature, while achieving a 2.44x speedup in comparison to the default (no apoptosis) algorithm.

Siegel, Charles M.↗

Visualization System for Evolutionary Neural Networks for Deep Learning

Deep learning is actively used in a wide range of fields for scientific discovery. To effectively apply deep learning to a particular problem, it is important to select an appropriate network architecture and other hyper-parameters (at each layer). Evolving architectures and hyper-parameters using a genetic algorithm is one current approach to search the huge space of all possible configurations to find those more optimal for the problem. However, examining an evolutionary process and tuning the genetic algorithm are challenging, pushing most users to treat the process as a black box. To address this challenge, we propose a visualization system for evolutionary neural networks for deep learning. The key feature of our visualization system is to provide a visual analytics environment for evaluating a genetic algorithm in order to improve the underlying operations to reduce time to find good solutions. Our system is able to not only visualize how a genetic algorithm traverses its search space but also allows users to examine evolving networks in-depth to get insights to improve performance through interactive visualization components.

Chae, Junghoon↗

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗

Deep Multi-Agent Reinforcement Learning for Real-World Signalized Traffic Corridor Control

Signalized traffic control problem has been addressed recently with deep Reinforcement Learning (RL) approaches involving diverse state, action, and reward structures. While significant progress has been noted in the literature, open challenges still remain in the areas of adaptive signal phase timing, coordination in a multi-intersection corridor setting, and consideration of real-world traffic conditions. In the context of deep RL-based problem framing, extensions are needed that enable adaptive signal phase timings in an intersection agent's action space, computationally efficient information sharing among neighboring signalized intersection agents along a corridor, and experimentation in realistic simulation environments. In this paper, we develop a deep Advantage Actor Critic (A2C) multi-agent RL (MARL) approach capturing the research extensions above and apply it within a real-world calibrated Aimsun Next traffic corridor simulation model based on traffic data from the City of Coral Gables, Florida. For a multi-intersection corridor control setting, our numerical simulation experiments with a decentralized A2C MARL algorithm applied at different time periods led to a total average corridor travel delay reduction (expressed in seconds/mile averaged over vehicles) from 4.9% to 19.9% compared to state-of-the-art actuated control.

Shuvo, Salman S. [BATTELLE (PACIFIC NW LAB)]↗

Vertex reconstruction of neutrino interactions using deep learning

Deep learning offers new tools to improve our understanding of many important scientific problems. Neutrinos are the most abundant particles in existence and are hypothesized to explain the matter-antimatter asymmetry that dominates our universe. Definitive tests of this conjecture require a detailed understanding of neutrino interactions with a variety of nuclei. Many measurements of interest depend on vertex reconstruction -- finding the origin of a neutrino interaction using data from the detector, which can be represented as images. Traditionally, this has been accomplished by utilizing methods that identify the tracks coming from the interaction. However, these methods are not ideal for interactions where an abundance of tracks and cascades occlude the vertex region. Manual algorithm engineering to handle these challenges is complicated and error prone. Deep learning extracts rich, semantic features directly from raw data, making it a promising solution to this problem. In this work, deep learning models are presented that classify the vertex location in regions meaningful to the domain scientists improving their ability to explore more complex interactions.

Terwilliger, Adam↗

Strategies for Integrating Deep Learning Surrogate Models with HPC Simulation Applications

The emerging trend of the convergence of high performance computing (HPC), machine learning/deep learning (ML/DL), and big data analytics presents a host of challenges for large-scale computing campaigns that seek best practices to interleave traditional scientific simulation-based workloads with ML/DL models. A portfolio of systematic approaches to incorporate deep learning into modeling and simulation serves a vital need when we support AI for science at a computing facility. In this paper, we evaluate several strategies for deploying deep learning surrogate models in a representative physics application on supercomputers at the Oak Ridge Leadership Computing Facility (OLCF). We discuss a set of recommended deployment architectures and implementation approaches. We analyze and evaluate these alternatives and show their performance and scalability up to 1000 GPUs on two mainstream platforms equipped with different deep learning hardware and software stacks.

Yin, Junqi↗

Volt-VAR Optimization in Distribution Networks Using Twin Delayed Deep Reinforcement Learning

Modern distribution grids are undergoing new challenges due to the stochastic nature of distributed energy resources (DERs). High penetration of DERs has a significant impact on Volt-VAR profile and system power losses. This work proposes a deep reinforcement learning (DRL)-based Volt-VAR optimization approach for improving voltage profile and reducing system power loss under high penetration of distributed energy resources, such as battery energy storage and solar photovoltaic units in distribution grids. The twin delayed deep deterministic policy gradient (TD3) method-based DRL agent is proposed to configure optimal set-points of reactive power outputs of fast responding smart inverters. The agent schedules the reactive power of inverters according to their physical capabilities, such as minimum allowed power factor, e.g., 0.9 leading/lagging. The reward function of the proposed DRL scheme is designed carefully to ensure a proper voltage profile of the grids with effective scheduling of reactive power outputs from inverters. The performance of the proposed model is verified on modified IEEE 34- and 123-bus systems and compared with base case with no reactive supply by inverters, and local droop Volt-VAR control approach. The results show that the proposed method performs better than the local droop control and deep deterministic policy gradient (DDPG)-based DRL method for reducing voltage fluctuation and minimizing power loss.

Hossain, Rakib↗

Transmission Line Fault Location Using Deep Learning Techniques

This letter presents a novel single-ended fault location approach for transmission lines that uses modern deep learning methods. A mixed convolutional neural network and long short-term memory structure are proposed to preserve the translation invariance and capture the temporal correlation of the time-series input data. Advanced deep learning techniques such as adaptive moment estimation and dropout are used to efficiently train the neural network and prevent over-fitting. Our extensive studies have demonstrated the accuracy and effectiveness of the deep-learning-based, singled-ended fault location approach.

Fan, Rui↗

Federated Deep Reinforcement Learning for Decentralized VVO of BTM DERs

The future of grid control requires a hybrid approach combining centralized and decentralized methods to fully utilize the potential of smart edge devices with artificial intelligence (AI) capabilities. This paper aims to develop and evaluate a federated deep reinforcement learning (FDRL) framework for decentralized adaptive volt-var optimization (VVO) of behind-the-meter (BTM) distributed energy resources (DERs). First, this paper models a single deep reinforcement learning (DRL) agent using the Markov Decision Process (MDP) framework for decentralized adaptive VVO of BTM DERs. Two DRL algorithms, soft actor-critic (SAC) and twin-delayed deep deterministic policy gradient (TD3), are compared for their effectiveness in optimizing VVO. Results show that TD3 outperforms SAC, achieving a 71.3% improvement in mean reward. Finally, the DRL agent is deployed within the FDRL framework, using the Flower platform, to enhance learning, provide adaptive control, and ensure data privacy for BTM DERs.

Ravi, Abhijith↗

Performance Analysis of Deep Learning Workloads on Leading-edge Systems

This work examines the performance of leading-edge systems designed for machine learning computing, including the NVIDIA DGX-2, Amazon Web Services (AWS) P3, IBM Power System Accelerated Compute Server AC922, and a consumer-grade Exxact TensorEX TS4 GPU server. Representative deep learning workloads from the fields of computer vision and natural language processing are the focus of the analysis. Performance analysis is performed along with a number of important dimensions. Performance of the communication interconnects and large and high-throughput deep learning models are considered. Different potential use models for the systems as standalone and in the cloud also are examined. The effect of various optimization of the deep learning models and system configurations is included in the analysis.

97 MATHEMATICS AND COMPUTING↗

Tellurium Secondary-Phase Defects in CdZnTe and their Association With the 1.1-eV Deep Trap

Defects located at the EC - 1.1 (eV) level, which are electro-optically active deep traps, generally have been overlooked since their presence cannot be detected except for those in highly resistive CdTe compounds. The origin of this trap is still debated on whether it is from Te vacancies or from dislocations induced by secondary phase defects in Te. We have grown high-resistivity Te-rich CZT ingots to clarify the origin of the 1.1 eV defects and to analyze the defect levels in the CZT samples by current deep level transient spectroscopy (I-DLTS)and photoluminescence (PL). From the analysis, defect levels such as shallow acceptor/donor, A-centers, Cd vacancies and Te antisite appeared to be in both high and low concentrations of Te inclusions, but the level of 1.1-eV defects exhibited dependence on the density of Te inclusion in the CZT samples. We also evaluated the effect of the 1.1-eV deep-level defects on the detector's performance in point view of carrier trapping and de-trapping.

36 MATERIALS SCIENCE↗

Microbial potential for carbon and nutrient cycling in a geogenic supercritical carbon dioxide reservoir: Microbial life in the deep carbonated biosphere

Microorganisms catalyze carbon cycling and biogeochemical reactions in the deep subsurface and thus may be expected to influence the fate of injected super-critical (sc) CO 2 following geological carbon sequestration (GCS). We hypothesized that natural subsurface scCO 2 reservoirs, which serve as analogs for the long-term fate of sequestered scCO 2 harbor a ‘deep carbonated biosphere’ with carbon cycling potential. We sampled subsurface fluids from scCO 2 - water separators at a natural scCO 2 reservoir at McElmo Dome, Colorado for analysis of 16S rRNA gene diversity and metagenome content. Sequence annotations indicated dominance of Sulfurospirillum, Rhizobium, Desulfovibrio and four members of the Clostridiales family. Genomes extracted from metagenomes using homology and compositional approaches revealed diverse mechanisms for growth and nutrient cycling, including pathways for CO 2 and N 2 fixation, anaerobic respiration, sulfur oxidation, fermentation and potential for metabolic syntrophy. Differences in biogeochemical potential between two production well communities were consistent with differences in fluid chemical profiles, suggesting a potential link between microbial activity and geochemistry. In conclusion, the existence of a microbial ecosystem associated with the McElmo Dome scCO 2 reservoir indicates that potential impacts of the deep biosphere on CO 2 fate and transport should be taken into consideration as a component of GCS planning and modelling.

54 ENVIRONMENTAL SCIENCES↗

Pressure Transducer Measurement Variability in Deep Wells Screened Across the Water Table

Abstract Automated water level measurements collected using vented pressure transducers in deep wells screened across the water table may exhibit a greater response to barometric pressure changes than the true water level. The cause was hypothesized to be disequilibrium in barometric pressure between the wellbores and land surface due to air exchange with the deep vadose zone. In this study, vented and nonvented pressure transducers were installed and operated simultaneously in two deep wells screened across the water table. A vent tube open to the atmosphere at land surface allowed for barometric compensation of the vented transducers. Two nonvented transducers were installed in each well, one submerged in the water and one above the water surface. The difference in readings allowed for barometric compensation. Manual measurements were also collected. It was confirmed that measurements from the vented transducers exhibited greater variability in response to barometric pressure changes than the nonvented transducers and manual measurements. Comparison of the downhole barometric pressure measurements to values from a nearby meteorology station showed the response in the wells to changes in barometric pressure was time‐lagged and attenuated. Thus, the reference pressure from land surface supplied to the vented transducers was not representative of the air pressure within the wells. This caused fluctuations of the transducer readings in response to barometric pressure changes to be greater than the true water level change. This issue can be resolved by the use of nonvented pressure transducers.

McDonald, John P.↗

Phylogenomic analysis of the hemp family (Cannabaceae) reveals deep cyto‐nuclear discordance and provides new insights into generic relationships

Abstract Cannabaceae are a relatively small family of angiosperms, but they include several species of huge economic and cultural significance: marijuana or hemp ( Cannabis sativa ) and hops ( Humulus lupulus ). Previous phylogenetic studies have clarified the most deep relationships in Cannabaceae, but relationships remain ambiguous among several major lineages. Here, we sampled 82 species representing all genera of Cannabaceae and utilized a new dataset of 90 nuclear genes and 82 chloroplast loci from Hyb‐Seq to investigate the phylogenomics of Cannabaceae. Nuclear phylogenetic analyses revealed a robust and consistent backbone for Cannabaceae. We observed nuclear gene‐tree conflict at several deep nodes in inferred species trees, also cyto‐nuclear discordance concerning the relationship between Gironniera and Lozanella and the relationships among Trema s.l. (including Parasponia ), Cannabis + Humulus , and Chaetachme + Pteroceltis . Coalescent simulations and network analyses suggest that observed deep cyto‐nuclear discordances were most likely to stem from incomplete lineage sorting (ILS); nuclear gene‐tree conflict might be caused by both ILS and gene flow between species. All genera of Cannabaceae were recovered as monophyletic, except for Celtis , which consisted of two distinct clades: Celtis I (including most Celtis species) and Celtis II (including Celtis gomphophylla and Celtis schippii ). We suggest that Celtis II should be recognized as the independent genus Sparrea based on both molecular and morphological evidence. Our work provides the most comprehensive and reliable phylogeny to date for Cannabaceae, enabling further exploration of evolutionary patterns across this family and highlighting the necessity of comparing nuclear with chloroplast data to examine the evolutionary history of plant groups.

Plant Sciences↗