Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

Deep Learning for Unsupervised Insider Threat Detection in Structured Cyber Security Data Streams

Analysis of an organization's computer network activity is a key component of early detection and mitigation of insider threat, a growing concern for many organizations. Raw system logs are a prototypical example of streaming data that can quickly scale beyond the cognitive power of a human analyst. As a prospective filter for the human analyst, we present an online unsupervised deep learning approach to detect anomalous network activity from system logs in real time. Our models decompose anomaly scores into the contributions of individual user behavior features for increased interpretability to aid analysts reviewing potential cases of insider threat. Using the CERT Insider Threat Dataset v6.2 and threat detection recall as our performance metric, our novel deep and recurrent neural network models outperform Principal Component Analysis, Support Vector Machine and Isolation Forest based anomaly detection baselines. For our best model, the events labeled as insider threat activity in our dataset had an average anomaly score in the 95.53 percentile, demonstrating our approach's potential to greatly reduce analyst workloads.

Deep Learning, Insider Threat↗

Distributed Power Allocation for 6-GHz Unlicensed Spectrum Sharing via Multi-agent Deep Reinforcement Learning

We consider the problem of power allocation over the 6 GHz Unlicensed National Information Infrastructure (UNII)- 5 spectrum. We propose a novel deep Reinforcement Learning (DRL)-based distributed power allocation scheme which utilizes the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. In particular, we model the base stations (BSs) as DRL agents that simultaneously determine the transmit powers to their scheduled user equipment (UE) in a synchronized manner. The power decision of each BS is based on its own observation of the radio environment, which consists of several local interference measurements and a limited amount of information obtained from other BSs. One advantage of the proposed scheme is that it addresses the single-agent non-stationarity problem of RL in the multi-agent scenario by incorporating the actions and observations of other BSs into each BS’s own critic which helps it to gain a more accurate perception of the overall radio environment. A centralized-training-distributed execution framework is used to train the policies where the critics are trained over the joint actions and observations of all BSs while the actor of each BS only takes the local observation as input in order to produce the transmit power. Simulation shows that the proposed power allocation scheme can achieve better throughput performance than several state-of-the-art approaches.

99 GENERAL AND MISCELLANEOUS↗

A Novel Multi-Agent Deep Reinforcement Learning-enabled Distributed Power Allocation Scheme for mmWave Cellular Networks

We consider the power allocation problem over shared spectrum for millimeter-Wave (mmWave) cellular downlink. Existing approaches usually find sub-optimal solutions by solving a non-convex optimization which leads to scalability issues due to centralized control. Therefore, distributed and adaptive approaches are desirable. Recently, model-free Deep Reinforcement Learning (DRL) has achieved success in such wireless resource management tasks. By modeling the radio environment as a Markov Decision Process (MDP) with the base stations (BSs) being the agents, power allocation can be automated at the agent level with comparable throughput performance to conventional centralized schemes. The multi-agent setting presents new challenges as the radio environment is impacted by the joint actions of the agents and is no longer stationary from any individual agent’s perspective. Existing literature bypasses this non-stationarity violation by ignoring it which may cause performance degradation. To tackle this issue, we propose a distributed continuous power allocation scheme based on a modified version of multi-agent Deep Deterministic Policy Gradient (MADDPG) that is tailored for the distributed multiple-agent setting. The proposed scheme employs a centralized-training distributed-execution framework where Q-functions are trained over subsets of BSs while each BS determines its transmit power based only on its own local observation. It admits constant per-BS communication and computation complexity and is thus scalable to large networks. Numerical evaluation shows that the proposed scheme adapts well to a wide range of interference conditions and can achieve comparable or better performance than several state-of-the-art non-learning approaches.

99 GENERAL AND MISCELLANEOUS↗

Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities

Although deep learning has made great progress in recent years, the exploding economic and environmental costs of training neural networks are becoming unsustainable. To address this problem, there has been a great deal of research on algorithmically-efficient deep learning, which seeks to reduce training costs not at the hardware or implementation level, but through changes in the semantics of the training program. In this paper, we present a structured and comprehensive overview of the research in this field. First, we formalize the algorithmic speedup problem, then we use fundamental building blocks of algorithmically efficient training to develop a taxonomy. Our taxonomy highlights commonalities of seemingly disparate methods and reveals current research gaps. Next, we present evaluation best practices to enable comprehensive, fair, and reliable comparisons of speedup techniques. To further aid research and applications, we discuss common bottlenecks in the training pipeline (illustrated via experiments) and offer taxonomic mitigation strategies for them. Finally, we highlight some unsolved research challenges and present promising future directions.

97 MATHEMATICS AND COMPUTING↗

Stress Analysis of Coated Particle Fuel in the Deep-Burn Pebble Bed Reactor Design

High fuel temperatures and resulting fuel particle coating stresses can be expected in a Pu and minor actinide fueled Pebble Bed Modular Reactor (400 MWth) design as compared to the ’standard’ UO2 fueled core. The high discharge burnup aimed for in this Deep-Burn design results in increased power and temperature peaking in the pebble bed near the inner and outer reflector. Furthermore, the pebble power in a multi-pass in-core pebble recycling scheme is relatively high for pebbles that make their first core pass. This might result in an increase of the mechanical failure of the coatings, which serve as the containment of radioactive fission products in the PBMR design. To investigate the integrity of the particle fuel coatings as a function of the irradiation time (i.e. burnup), core position and during a Loss Of Forced Cooling (LOFC) incident the PArticle STress Analysis code (PASTA) has been coupled to the PEBBED code for neutronics, thermal-hydraulics and depletion analysis of the core. Two deep burn fuel types (Pu with or without initial MA fuel content) have been investigated with the new code system for normal and transient conditions including the effect of the statistical variation of thickness of the coating layers.

99 GENERAL AND MISCELLANEOUS↗

Core Optimization of a Deep-Burn Pebble Bed Reactor

Achieving a high fuel burnup in the Deep-Burn (DB) pebble bed reactor design, while remaining within the limits for fuel temperature, power peaking and temperature reactivity feedback, is challenging. The high content of Pu and Minor Actinides in the Deep-Burn fuel significantly impacts the thermal neutron energy spectrum as compared to a ’standard’ UO2 fueled core. This can result in power and temperature peaking in the pebble bed core in locally thermalized regions near the graphite reflectors. Furthermore, the interplay of the Pu resonances of the neutron absorption cross sections at low-lying energies can lead to a positive temperature reactivity coefficient for the graphite moderator at certain operating conditions. The DB concept focuses on the destruction of spent fuel transuranics in TRISO coated particle fueled gas-cooled reactors with the aim of a fractional fuel burnup of 60-70% in fissions per initial metal atom (FIMA), using a single-pass, multi in-core fuel (re)cycling scheme. In principle, the DB pebble bed concept employs the same reactor designs as the present low enriched uranium core designs, i.e. the 400 MWth Pebble Bed Modular Reactor (PBMR-400). A Pu and Minor Actinide fueled PBMR-400 design serves as the starting point for a core optimization study. The fuel temperature, power peak, temperature reactivity coefficients, and burnup capabilities of the modified designs are analyzed with the PEBBED code. A code-to-code coupling with the PASTA code allows for the analysis of the TRISO fuel performance for both normal and Loss Of Forced Cooling conditions. An improved core design is sought, maximizing the fuel discharge burnup, while retaining negative temperature reactivity feedback coefficients for the entire temperature range and avoiding high fuel temperatures (fuel failure probabilities).

99 GENERAL AND MISCELLANEOUS↗

Deep and shallow sources for the Lusi mud eruption revealed by surface deformation

The Lusi mud eruption, in East Java, Indonesia, began in May 2006 and continues to the present. Previous analyses of surface deformation data suggested an exponential decay of the pressure in the mud source but did not constrain the location, geometry, and evolution of the possible source(s) of the erupting mud and fluids. To map the surface deformation, we employ multitemporal interferometric synthetic aperture radar and analyze a well-populated L-band data set acquired by the Advanced Land Observing Satellite (ALOS) between May 2006 and April 2011. We then apply a time-dependent inverse modeling scheme. Volume changes occur in two regions beneath Lusi, at 0.3-2.0km and 3.5-4.75km depth. The cumulative volume change within the shallow source is ~2-3 times larger than that of the deep source. The observation and model suggest that a shallow source plays a key role by supplying the erupting mud, but that additional fluids do ascend from depths >4km on eruptive timescales. Key Points Spatiotemporal map of the surface deformation associated with Lusi mud eruption Time-dependent source model of the Lusi and nearby gas field Revealing the deep and shallow sources of the Lusi and characterizing their relationship.

58 GEOSCIENCES↗

Reflection imaging of the Moon's interior using deep-moonquake seismic interferometry

The internal structure of the Moon has been investigated over many years using a variety of seismic methods, such as travel time analysis, receiver functions, and tomography. Here we propose to apply body-wave seismic interferometry to deep moonquakes in order to retrieve zero-offset reflection responses (and thus images) beneath the Apollo stations on the nearside of the Moon from virtual sources colocated with the stations. This method is called deep-moonquake seismic interferometry (DMSI). Our results show a laterally coherent acoustic boundary around 50 km depth beneath all four Apollo stations. We interpret this boundary as the lunar seismic Moho. This depth agrees with Japan Aerospace Exploration Agency's (JAXA) SELenological and Engineering Explorer (SELENE) result and previous travel time analysis at the Apollo 12/14 sites. The deeper part of the image we obtain from DMSI shows laterally incoherent structures. Finally, such lateral inhomogeneity we interpret as representing a zone characterized by strong scattering and constant apparent seismic velocity at our resolution scale (0.2–2.0 Hz).

58 GEOSCIENCES↗

New High-Pressure Phase of CaCO 3 at the Topmost Lower Mantle: Implication for the Deep-Mantle Carbon Transportation

In this study, we have investigated the stability of CaCO 3 at high pressures and temperatures using synchrotron X-ray diffraction in laser-heated diamond anvil cells. Our experimental results have shown that CaCO 3 in the aragonite structure transforms into CaCO 3 -VII (P2 1 /c) at 27 GPa and 1,500 K with a negative Clapeyron slope of –4.3(9) MPa/K. CaCO 3 -VII is stable between 23 and 38 GPa at 2,300 K and transforms into post-aragonite at 42 GPa and 1,400 K. Furthermore, it reacts with stishovite, an abundant form of SiO 2 in subducted oceanic crust, forming CaSiO 3 -perovskite. The occurrence of CaSiO 3 -perovskite via the reaction of CaCO 3 -VII and stishovite provides an explanation for the observation of the high concentrations of CaSiO 3 -perovskite and some amount of CaCO 3 in deep-mantle inclusions. We conclude, CaCO 3 -VII is thus an important carbon-bearing phase at the topmost lower mantle and may provide necessary carbon to produce deep-mantle diamonds.

58 GEOSCIENCES↗

Control of Host-Matrix Morphology Enables Efficient Deep-Blue Organic Light-Emitting Devices

Mixing a sterically bulky, electron-transporting host material into a conventional single host–guest emissive layer is demonstrated to suppress phase separation of the host matrix while increasing the efficiency and operational lifetime of deep-blue phosphorescent organic light-emitting diodes (PHOLEDs) with chromaticity coordinates of (0.14, 0.15). Here, the bulky host enables homogenous mixing of the molecules comprising the emissive layer while suppressing single host aggregation; a significant loss channel of nonradiative recombination. By controlling the amorphous phase of the host-matrix morphology, the mixed-host device achieves a significant reduction in nonradiative exciton decay, resulting in 120 ± 6% increase in external quantum efficiency relative to an analogous, single-host device. In contrast to single host PHOLEDs where electrons are transported by the host and holes by the dopants, both charge carriers are conducted by the mixed host, reducing the probability of exciton annihilation, thereby doubling of the deep-blue PHOLED operational lifetime. These findings demonstrate that the host matrix morphology affects almost every aspect of PHOLED performance.

36 MATERIALS SCIENCE↗

Prediction of Carbon Dioxide Adsorption via Deep Learning

Abstract Porous carbons with different textural properties exhibit great differences in CO 2 adsorption capacity. It is generally known that narrow micropores contribute to higher CO 2 adsorption capacity. However, it is still unclear what role each variable in the textural properties plays in CO 2 adsorption. Herein, a deep neural network is trained as a generative model to direct the relationship between CO 2 adsorption of porous carbons and corresponding textural properties. The trained neural network is further employed as an implicit model to estimate its ability to predict the CO 2 adsorption capacity of unknown porous carbons. Interestingly, the practical CO 2 adsorption amounts are in good agreement with predicted values using surface area, micropore and mesopore volumes as the input values simultaneously. This unprecedented deep learning neural network (DNN) approach, a type of machine learning algorithm, exhibits great potential to predict gas adsorption and guide the development of next‐generation carbons.

Zhang, Zihao↗

Prediction of Carbon Dioxide Adsorption via Deep Learning

Porous carbons with different textural properties exhibit great differences in CO 2 adsorption capacity. It is generally known that narrow micropores contribute to higher CO 2 adsorption capacity. However, it is still unclear what role each variable in the textural properties plays in CO 2 adsorption. Herein, a deep neural network is trained as a generative model to direct the relationship between CO 2 adsorption of porous carbons and corresponding textural properties. The trained neural network is further employed as an implicit model to estimate its ability to predict the CO 2 adsorption capacity of unknown porous carbons. Interestingly, the practical CO 2 adsorption amounts are in good agreement with predicted values using surface area, micropore and mesopore volumes as the input values simultaneously. This unprecedented deep learning neural network (DNN) approach, a type of machine learning algorithm, exhibits great potential to predict gas adsorption and guide the development of next-generation carbons.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Utilizing ensemble learning for performance and power modeling and improvement of parallel cancer deep learning CANDLE benchmarks

Abstract Machine learning (ML) continues to grow in importance across nearly all domains in modeling to learn from data. Often a tradeoff exists between a model's ability to minimize bias and variance. In this article, we utilize ensemble learning to combine linear, nonlinear, and tree‐/rule‐based ML methods to cope with the bias‐variance tradeoff and result in more accurate models. We use the datasets collected for two parallel cancer deep learning CANDLE benchmarks, NT3 and P1B2, to build performance and power models based on hardware performance counters using single‐object and multiple‐objects ensemble learning to identify the most important counters for improvement on the Cray XC40 Theta at Argonne National Laboratory. Based on the insights from these models, we improve the performance and energy of P1B2 and NT3 by optimizing the deep learning environments TensorFlow, Keras, Horovod, and Python under the huge page size of 8 MB. Experimental results show that ensemble learning not only produces more accurate models but also provides more robust performance counter ranking. We achieve up to 61.15% performance improvement and up to 62.58% energy saving for P1B2 and up to 55.81% performance improvement and up to 52.60% energy saving for NT3 on up to 24,576 cores.

Wu, Xingfu↗

Implications of Arctic sea ice changes for North Atlantic deep convection and the meridional overturning circulation in CCSM4-CMIP5 simulations

Using CCSM4 climate simulations for 1850–2300 with four different future forcing scenarios, we show that the maximum strength of the Atlantic meridional overturning circulation (MOC) decreases proportionally to the applied CO 2 forcing. This weakening of the overturning is caused by a reduction or shut down of North Atlantic (NA) deep convection due to a surface freshening. In the Labrador Sea, the surface freshening is caused by strongly increased liquid freshwater exports from the Arctic, which are largely due to the decrease in the Arctic sea ice cover. In the strongest forcing scenario (RCP8.5), the Arctic becomes summer ice-free by the end of the 21st century and year-round ice-free by the end of the 23rd century. As a result of the associated freshening, all NA deep convection ceases by 2145, which leads to a 72% (18 Sv) decrease of the MOC strength by the end of the simulation in 2300.

54 ENVIRONMENTAL SCIENCES↗

Use of Refractory-Volatile Element Deep Eutectic Regions to Grow Single Crystalline Intermetallic Compounds

Compounds containing both refractory and volatile elements present a unique challenge for crystal growth, given the conflicting realities of high melting temperatures and high vapor pressures. Nevertheless, the discovery of superconductivity in FeAs and FeSe based materials and a Weyl semimetal state in TaP are motivations to explore compounds containing such pairs. Here, we discuss use of the low-melting single phase liquid regions above deep M-X eutectics (M=transition metal, X=P, S) as the basis for high temperature solutions for growing intermetallic compounds containing volatile-refractory pairs. We show that Ni-P, Pd-P, Pt-P, and Pd-S compositions form single phase melts at moderate temperatures below 1000 °C and with minimal vapor pressure. We first present the simple case of growing Ni 2 P from Ni-P and next discuss the more complicated growth of RPd 3 S 4 (R=La, Ce, Nd, Eu) from a Pd-S melt. We show how frit-disc alumina crucible sets allow for contamination-free capture of decanted liquid and its reuse in subsequent experiments, demonstrating a fractionation of the CePd 3 S 4 growth to determine the optimal conditions for crystal growth. We conclude by using the single phase liquid regions above the Pt-P and Pd-P eutectics to grow single crystals of MPt 5 P (M=Mn, Fe) and MnPd 5 P. As these materials have primarily been studied in polycrystalline form, we give an overview of the magnetic and transport properties of our single crystals. The examples outlined here illustrate the utility of using the single phase liquid above deep metal-X eutectics for solution growth and materials discovery.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Automated Programmable Logic Controller Memory Forensics Using RGB Image Analysis and Deep Learning

The introduction of Industry 4.0 and Internet-based technologies has enhanced industrial control system operations but have inadvertently increased their vulnerabilities to cyber attacks. When an industrial control system is compromised, security analysts need to identify the root cause quickly to start the recovery process and develop mitigation strategies. Memory forensics is critical in the incident analysis process to ascertain what occurred. Approaches for analyzing the persistent memory in industrial control devices are limited and almost nonexistent for volatile memory. This chapter proposes an automated methodology for programmable logic controller memory dump analysis using computer vision and deep learning techniques. The methodology converts the sequences of bytes in a programmable logic controller memory dump to red-green-blue pixels and employs a deep learning model that learns the underlying patterns and features of pre-labeled forensic artifacts in images and segments them into distinct regions. The trained model is employed to automatically segment new memory images and identify forensic artifacts. Evaluation of the methodology on a Schneider Electric Modicon M221 programmable logic controller under code injection and code modification attacks demonstrates its ability to detect attack artifacts in memory dumps.

Asmar Awad, Rima [ORNL] (ORCID:0000000233407742)↗

Deep Reinforcement Learning Based Smart Water Heater Control for Reducing Electricity Consumption and Carbon Emission

Water heating is the third largest electricity consumer in U.S. households, after space heating and cooling. Thus, water heaters represent a significant potential for reducing electricity consumption and associated CO2 emissions of residential buildings. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity consumption and the CO2 emissions of a heat pump water heater without affecting user comfort. In this approach, a set of RL agents focusing on either electricity saving or emission reduction, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on different hot water usage and Marginal Operating Emissions Rate (MOER) profiles. The testing results showed that the RL agents that focus on electricity saving can save electricity in the range of 12–22% by operating the water heater with maximum heat pump efficiency and minimum electric element utilization. On the other hand, the RL agents that focus on emission reduction reduced emissions in the range of 18–37% by making use of the variable MOER values. These RL agents used the heat pump and/or an element when the MOER values are low due to the availability of renewable energy sources (e.g., solar and wind) and mostly avoided the periods of carbon-intensive periods. Overall, these results showed that the proposed RL approach can help minimize the electricity consumption and the CO2 emissions of a heat pump water heater without having any prior knowledge about the device.

Amasyali, Kadir↗

Deep learning in color: towards automated quark/gluon jet discrimination

Artificial intelligence offers the potential to automate challenging data-processing tasks in collider physics. Here, to establish its prospects, we explore to what extent deep learning with convolutional neural networks can discriminate quark and gluon jets better than observables designed by physicists. Our approach builds upon the paradigm that a jet can be treated as an image, with intensity given by the local calorimeter deposits. We supplement this construction by adding color to the images, with red, green and blue intensities given by the transverse momentum in charged particles, transverse momentum in neutral particles, and pixel-level charged particle counts. Overall, the deep networks match or outperform traditional jet variables. We also find that, while various simulations produce different quark and gluon jets, the neural networks are surprisingly insensitive to these differences, similar to traditional observables. This suggests that the networks can extract robust physical information from imperfect simulations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗