Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,243 records · Page 69
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities

Although deep learning has made great progress in recent years, the exploding economic and environmental costs of training neural networks are becoming unsustainable. To address this problem, there has been a great deal of research on algorithmically-efficient deep learning, which seeks to reduce training costs not at the hardware or implementation level, but through changes in the semantics of the training program. In this paper, we present a structured and comprehensive overview of the research in this field. First, we formalize the algorithmic speedup problem, then we use fundamental building blocks of algorithmically efficient training to develop a taxonomy. Our taxonomy highlights commonalities of seemingly disparate methods and reveals current research gaps. Next, we present evaluation best practices to enable comprehensive, fair, and reliable comparisons of speedup techniques. To further aid research and applications, we discuss common bottlenecks in the training pipeline (illustrated via experiments) and offer taxonomic mitigation strategies for them. Finally, we highlight some unsolved research challenges and present promising future directions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Deep and shallow sources for the Lusi mud eruption revealed by surface deformation

The Lusi mud eruption, in East Java, Indonesia, began in May 2006 and continues to the present. Previous analyses of surface deformation data suggested an exponential decay of the pressure in the mud source but did not constrain the location, geometry, and evolution of the possible source(s) of the erupting mud and fluids. To map the surface deformation, we employ multitemporal interferometric synthetic aperture radar and analyze a well-populated L-band data set acquired by the Advanced Land Observing Satellite (ALOS) between May 2006 and April 2011. We then apply a time-dependent inverse modeling scheme. Volume changes occur in two regions beneath Lusi, at 0.3-2.0km and 3.5-4.75km depth. The cumulative volume change within the shallow source is ~2-3 times larger than that of the deep source. The observation and model suggest that a shallow source plays a key role by supplying the erupting mud, but that additional fluids do ascend from depths >4km on eruptive timescales. Key Points Spatiotemporal map of the surface deformation associated with Lusi mud eruption Time-dependent source model of the Lusi and nearby gas field Revealing the deep and shallow sources of the Lusi and characterizing their relationship.

58 GEOSCIENCES↗
DOE OSTI2016

Reflection imaging of the Moon's interior using deep-moonquake seismic interferometry

The internal structure of the Moon has been investigated over many years using a variety of seismic methods, such as travel time analysis, receiver functions, and tomography. Here we propose to apply body-wave seismic interferometry to deep moonquakes in order to retrieve zero-offset reflection responses (and thus images) beneath the Apollo stations on the nearside of the Moon from virtual sources colocated with the stations. This method is called deep-moonquake seismic interferometry (DMSI). Our results show a laterally coherent acoustic boundary around 50 km depth beneath all four Apollo stations. We interpret this boundary as the lunar seismic Moho. This depth agrees with Japan Aerospace Exploration Agency's (JAXA) SELenological and Engineering Explorer (SELENE) result and previous travel time analysis at the Apollo 12/14 sites. The deeper part of the image we obtain from DMSI shows laterally incoherent structures. Finally, such lateral inhomogeneity we interpret as representing a zone characterized by strong scattering and constant apparent seismic velocity at our resolution scale (0.2–2.0 Hz).

58 GEOSCIENCES↗
DOE OSTI2018

New High-Pressure Phase of CaCO 3 at the Topmost Lower Mantle: Implication for the Deep-Mantle Carbon Transportation

In this study, we have investigated the stability of CaCO 3 at high pressures and temperatures using synchrotron X-ray diffraction in laser-heated diamond anvil cells. Our experimental results have shown that CaCO 3 in the aragonite structure transforms into CaCO 3 -VII (P2 1 /c) at 27 GPa and 1,500 K with a negative Clapeyron slope of –4.3(9) MPa/K. CaCO 3 -VII is stable between 23 and 38 GPa at 2,300 K and transforms into post-aragonite at 42 GPa and 1,400 K. Furthermore, it reacts with stishovite, an abundant form of SiO 2 in subducted oceanic crust, forming CaSiO 3 -perovskite. The occurrence of CaSiO 3 -perovskite via the reaction of CaCO 3 -VII and stishovite provides an explanation for the observation of the high concentrations of CaSiO 3 -perovskite and some amount of CaCO 3 in deep-mantle inclusions. We conclude, CaCO 3 -VII is thus an important carbon-bearing phase at the topmost lower mantle and may provide necessary carbon to produce deep-mantle diamonds.

58 GEOSCIENCES↗
DOE OSTI2023

Control of Host-Matrix Morphology Enables Efficient Deep-Blue Organic Light-Emitting Devices

Mixing a sterically bulky, electron-transporting host material into a conventional single host–guest emissive layer is demonstrated to suppress phase separation of the host matrix while increasing the efficiency and operational lifetime of deep-blue phosphorescent organic light-emitting diodes (PHOLEDs) with chromaticity coordinates of (0.14, 0.15). Here, the bulky host enables homogenous mixing of the molecules comprising the emissive layer while suppressing single host aggregation; a significant loss channel of nonradiative recombination. By controlling the amorphous phase of the host-matrix morphology, the mixed-host device achieves a significant reduction in nonradiative exciton decay, resulting in 120 ± 6% increase in external quantum efficiency relative to an analogous, single-host device. In contrast to single host PHOLEDs where electrons are transported by the host and holes by the dopants, both charge carriers are conducted by the mixed host, reducing the probability of exciton annihilation, thereby doubling of the deep-blue PHOLED operational lifetime. These findings demonstrate that the host matrix morphology affects almost every aspect of PHOLED performance.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Prediction of Carbon Dioxide Adsorption via Deep Learning

Abstract Porous carbons with different textural properties exhibit great differences in CO 2 adsorption capacity. It is generally known that narrow micropores contribute to higher CO 2 adsorption capacity. However, it is still unclear what role each variable in the textural properties plays in CO 2 adsorption. Herein, a deep neural network is trained as a generative model to direct the relationship between CO 2 adsorption of porous carbons and corresponding textural properties. The trained neural network is further employed as an implicit model to estimate its ability to predict the CO 2 adsorption capacity of unknown porous carbons. Interestingly, the practical CO 2 adsorption amounts are in good agreement with predicted values using surface area, micropore and mesopore volumes as the input values simultaneously. This unprecedented deep learning neural network (DNN) approach, a type of machine learning algorithm, exhibits great potential to predict gas adsorption and guide the development of next‐generation carbons.

Zhang, Zihao↗
DOE OSTI2018

Prediction of Carbon Dioxide Adsorption via Deep Learning

Porous carbons with different textural properties exhibit great differences in CO 2 adsorption capacity. It is generally known that narrow micropores contribute to higher CO 2 adsorption capacity. However, it is still unclear what role each variable in the textural properties plays in CO 2 adsorption. Herein, a deep neural network is trained as a generative model to direct the relationship between CO 2 adsorption of porous carbons and corresponding textural properties. The trained neural network is further employed as an implicit model to estimate its ability to predict the CO 2 adsorption capacity of unknown porous carbons. Interestingly, the practical CO 2 adsorption amounts are in good agreement with predicted values using surface area, micropore and mesopore volumes as the input values simultaneously. This unprecedented deep learning neural network (DNN) approach, a type of machine learning algorithm, exhibits great potential to predict gas adsorption and guide the development of next-generation carbons.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Utilizing ensemble learning for performance and power modeling and improvement of parallel cancer deep learning CANDLE benchmarks

Abstract Machine learning (ML) continues to grow in importance across nearly all domains in modeling to learn from data. Often a tradeoff exists between a model's ability to minimize bias and variance. In this article, we utilize ensemble learning to combine linear, nonlinear, and tree‐/rule‐based ML methods to cope with the bias‐variance tradeoff and result in more accurate models. We use the datasets collected for two parallel cancer deep learning CANDLE benchmarks, NT3 and P1B2, to build performance and power models based on hardware performance counters using single‐object and multiple‐objects ensemble learning to identify the most important counters for improvement on the Cray XC40 Theta at Argonne National Laboratory. Based on the insights from these models, we improve the performance and energy of P1B2 and NT3 by optimizing the deep learning environments TensorFlow, Keras, Horovod, and Python under the huge page size of 8 MB. Experimental results show that ensemble learning not only produces more accurate models but also provides more robust performance counter ranking. We achieve up to 61.15% performance improvement and up to 62.58% energy saving for P1B2 and up to 55.81% performance improvement and up to 52.60% energy saving for NT3 on up to 24,576 cores.

Wu, Xingfu↗
DOE OSTI2013

Implications of Arctic sea ice changes for North Atlantic deep convection and the meridional overturning circulation in CCSM4-CMIP5 simulations

Using CCSM4 climate simulations for 1850–2300 with four different future forcing scenarios, we show that the maximum strength of the Atlantic meridional overturning circulation (MOC) decreases proportionally to the applied CO 2 forcing. This weakening of the overturning is caused by a reduction or shut down of North Atlantic (NA) deep convection due to a surface freshening. In the Labrador Sea, the surface freshening is caused by strongly increased liquid freshwater exports from the Arctic, which are largely due to the decrease in the Arctic sea ice cover. In the strongest forcing scenario (RCP8.5), the Arctic becomes summer ice-free by the end of the 21st century and year-round ice-free by the end of the 23rd century. As a result of the associated freshening, all NA deep convection ceases by 2145, which leads to a 72% (18 Sv) decrease of the MOC strength by the end of the simulation in 2300.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Use of Refractory-Volatile Element Deep Eutectic Regions to Grow Single Crystalline Intermetallic Compounds

Compounds containing both refractory and volatile elements present a unique challenge for crystal growth, given the conflicting realities of high melting temperatures and high vapor pressures. Nevertheless, the discovery of superconductivity in FeAs and FeSe based materials and a Weyl semimetal state in TaP are motivations to explore compounds containing such pairs. Here, we discuss use of the low-melting single phase liquid regions above deep M-X eutectics (M=transition metal, X=P, S) as the basis for high temperature solutions for growing intermetallic compounds containing volatile-refractory pairs. We show that Ni-P, Pd-P, Pt-P, and Pd-S compositions form single phase melts at moderate temperatures below 1000 °C and with minimal vapor pressure. We first present the simple case of growing Ni 2 P from Ni-P and next discuss the more complicated growth of RPd 3 S 4 (R=La, Ce, Nd, Eu) from a Pd-S melt. We show how frit-disc alumina crucible sets allow for contamination-free capture of decanted liquid and its reuse in subsequent experiments, demonstrating a fractionation of the CePd 3 S 4 growth to determine the optimal conditions for crystal growth. We conclude by using the single phase liquid regions above the Pt-P and Pd-P eutectics to grow single crystals of MPt 5 P (M=Mn, Fe) and MnPd 5 P. As these materials have primarily been studied in polycrystalline form, we give an overview of the magnetic and transport properties of our single crystals. The examples outlined here illustrate the utility of using the single phase liquid above deep metal-X eutectics for solution growth and materials discovery.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Automated Programmable Logic Controller Memory Forensics Using RGB Image Analysis and Deep Learning

The introduction of Industry 4.0 and Internet-based technologies has enhanced industrial control system operations but have inadvertently increased their vulnerabilities to cyber attacks. When an industrial control system is compromised, security analysts need to identify the root cause quickly to start the recovery process and develop mitigation strategies. Memory forensics is critical in the incident analysis process to ascertain what occurred. Approaches for analyzing the persistent memory in industrial control devices are limited and almost nonexistent for volatile memory. This chapter proposes an automated methodology for programmable logic controller memory dump analysis using computer vision and deep learning techniques. The methodology converts the sequences of bytes in a programmable logic controller memory dump to red-green-blue pixels and employs a deep learning model that learns the underlying patterns and features of pre-labeled forensic artifacts in images and segments them into distinct regions. The trained model is employed to automatically segment new memory images and identify forensic artifacts. Evaluation of the methodology on a Schneider Electric Modicon M221 programmable logic controller under code injection and code modification attacks demonstrates its ability to detect attack artifacts in memory dumps.

Asmar Awad, Rima [ORNL] (ORCID:0000000233407742)↗
DOE OSTI2023

Deep Reinforcement Learning Based Smart Water Heater Control for Reducing Electricity Consumption and Carbon Emission

Water heating is the third largest electricity consumer in U.S. households, after space heating and cooling. Thus, water heaters represent a significant potential for reducing electricity consumption and associated CO2 emissions of residential buildings. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity consumption and the CO2 emissions of a heat pump water heater without affecting user comfort. In this approach, a set of RL agents focusing on either electricity saving or emission reduction, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on different hot water usage and Marginal Operating Emissions Rate (MOER) profiles. The testing results showed that the RL agents that focus on electricity saving can save electricity in the range of 12–22% by operating the water heater with maximum heat pump efficiency and minimum electric element utilization. On the other hand, the RL agents that focus on emission reduction reduced emissions in the range of 18–37% by making use of the variable MOER values. These RL agents used the heat pump and/or an element when the MOER values are low due to the availability of renewable energy sources (e.g., solar and wind) and mostly avoided the periods of carbon-intensive periods. Overall, these results showed that the proposed RL approach can help minimize the electricity consumption and the CO2 emissions of a heat pump water heater without having any prior knowledge about the device.

Amasyali, Kadir↗
DOE OSTI2017

Deep learning in color: towards automated quark/gluon jet discrimination

Artificial intelligence offers the potential to automate challenging data-processing tasks in collider physics. Here, to establish its prospects, we explore to what extent deep learning with convolutional neural networks can discriminate quark and gluon jets better than observables designed by physicists. Our approach builds upon the paradigm that a jet can be treated as an image, with intensity given by the local calorimeter deposits. We supplement this construction by adding color to the images, with red, green and blue intensities given by the transverse momentum in charged particles, transverse momentum in neutral particles, and pixel-level charged particle counts. Overall, the deep networks match or outperform traditional jet variables. We also find that, while various simulations produce different quark and gluon jets, the neural networks are surprisingly insensitive to these differences, similar to traditional observables. This suggests that the networks can extract robust physical information from imperfect simulations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2016

Jet-images — deep learning edition

Building on the notion of a particle physics detector as a camera and the collimated streams of high energy particles, or jets, it measures as an image, we investigate the potential of machine learning techniques based on deep learning architectures to identify highly boosted W bosons. Modern deep learning algorithms trained on jet images can out-perform standard physically-motivated feature driven approaches to jet tagging. We develop techniques for visualizing how these features are learned by the network and what additional information is used to improve performance. Finally, this interplay between physically-motivated feature driven tools and supervised learning algorithms is general and can be used to significantly increase the sensitivity to discover new particles and new forces, and gain a deeper understanding of the physics within jets.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Fitting a deep generative hadronization model

Hadronization is a critical step in the simulation of high-energy particle and nuclear physics experiments. As there is no first principles understanding of this process, physically-inspired hadronization models have a large number of parameters that are fit to data. Deep generative models are a natural replacement for classical techniques, since they are more flexible and may be able to improve the overall precision. Proof of principle studies have shown how to use neural networks to emulate specific hadronization when trained using the inputs and outputs of classical methods. However, these approaches will not work with data, where we do not have a matching between observed hadrons and partons. In this paper, we develop a protocol for fitting a deep generative hadronization model in a realistic setting, where we only have access to a set of hadrons in data. Our approach uses a variation of a Generative Adversarial Network with a permutation invariant discriminator. We find that this setup is able to match the hadronization model in Herwig with multiple sets of parameters. This work represents a significant step forward in a longer term program to develop, train, and integrate machine learning-based hadronization models into parton shower Monte Carlo programs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2018

Pulling out all the tops with computer vision and deep learning

We apply computer vision with deep learning — in the form of a convolutional neural network (CNN) — to build a highly effective boosted top tagger. Previous work (the “DeepTop” tagger of Kasieczka et al) has shown that a CNN-based top tagger can achieve comparable performance to state-of-the-art conventional top taggers based on high-level inputs. Here, we introduce a number of improvements to the DeepTop tagger, including architecture, training, image preprocessing, sample size and color pixels. Our final CNN top tagger outperforms BDTs based on high-level inputs by a factor of ~ 2–3 or more in background rejection, over a wide range of tagging efficiencies and fiducial jet selections. As reference points, we achieve a QCD background rejection factor of 500 (60) at 50%top tagging efficiency for fully-merged (non-merged) top jets with pT in the 800–900 GeV (350–450 GeV) range. Our CNN can also be straightforwardly extended to the classification of other types of jets, and the lessons learned here may be useful to others designing their own deep NNs for LHC applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Deep learning and radiomics: the utility of Google TensorFlow™ Inception in classifying clear cell renal cell carcinoma and oncocytoma on multiphasic CT

Purpose: Currently, all solid enhancing renal masses without microscopic fat are considered malignant until proven otherwise and there is substantial overlap in the imaging findings of benign and malignant renal masses, particularly between clear cell RCC (ccRCC) and benign oncocytoma (ONC). Radiomics has attracted increased attention for its utility in pre-operative work-up on routine clinical images. Radiomics based approaches have converted medical images into mineable data and identified prognostic imaging signatures that machine learning algorithms can use to construct predictive models by learning the decision boundaries of the underlying data distribution. The TensorFlow™ framework from Google is a state-of-the-art open-source software library that can be used for training deep learning neural networks for performing machine learning tasks. The purpose of this study was to investigate the diagnostic value and feasibility of a deep learning-based renal lesion classifier using open-source Google TensorFlow™ Inception in differentiating ccRCC from ONC on routine four-phase MDCT in patients with pathologically confirmed renal masses. Methods: With institutional review board approval for this 1996 Health Insurance Portability and Accountability Act compliant retrospective study and a waiver of informed consent, we queried our institution’s pathology, clinical, and radiology databases for histologically proven cases of ccRCC and ONC obtained between January 2000 and January 2016 scanned with a an intravenous contrast-enhanced four-phase renal mass protocol (unenhanced (UN), corticomedullary (CM), nephrographic (NP), and excretory (EX) phases). To extract features to be used for the machine learning model, the entire renal mass was contoured in the axial plane in each of the four phases, resulting in a 3D volume of interest (VOI) representative of the entire renal mass. We investigated thirteen different approaches to convert the acquired VOI data into a set of images that adequately represented each tumor which was used to train the final layer of the neural network model. Training was performed over 4000 iterations. In each iteration, 90% of the data were designated as training data and the remaining 10% served as validation data and a leave-one-out cross-validation scheme was implemented. Accuracy, sensitivity, specificity, positive (PPV) and negative predictive (NPV) values, and CIs were calculated for the classification of the thirteen processing modes. Results: We analyzed 179 consecutive patients with 179 lesions (128 ccRCC and 51 ONC). The ccRCC cohort had a mean size of 3.8 cm (range 0.8–14.6 cm) and the ONC cohort had a mean lesion size of 3.9 cm (range 1.0–13.1 cm). The highest specificity and PPV (52.9% and 80.3%, respectively) were achieved in the EX phase when we analyzed the single mid-slice of the tumor in the axial, coronal and sagittal plane, and when we increased the number of mid-slices of the tumor to three, with an accuracy of 75.4%, which also increased the sensitivity to 88.3% and the PPV to 79.6%. Using the entire tumor volume also showed that classification performance was best in the EX phase with an accuracy of 74.4%, a sensitivity of 85.8% and a PPV of 80.1%. When the entire tumor volume, plus mid-slices from all phases and all planes presented as tiled images, were submitted to the final layer of the neural network we achieved a PPV of 82.5%. Conclusions: The best classification result was obtained in the EX phase among the thirteen classification methods tested. Our proof of concept study is the first step towards understanding the utility of machine learning in the differentiation of ccRCC from ONC on routine CT images. We hope this could lead to future investigation into the development of a multivariate machine learning model which may augment our ability to accurately predict renal lesion histology on imaging.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2018

Understanding Clinical Mammographic Breast Density Assessment: a Deep Learning Perspective

Mammographic breast density has been established as an independent risk marker for developing breast cancer. Breast density assessment is a routine clinical need in breast cancer screening and current standard is using the Breast Imaging and Reporting Data System (BI-RADS) criteria including four qualitative categories (i.e., fatty, scattered density, heterogeneously dense, or extremely dense). In each mammogram examination, a breast is typically imaged with two different views, i.e., the mediolateral oblique (MLO) view and cranial caudal (CC) view. The BI-RADS-based breast density assessment is a qualitative process made by visual observation of both the MLO and CC views by radiologists, where there is a notable inter- and intra-reader variability. In order to maintain consistency and accuracy in BI-RADS-based breast density assessment, gaining understanding on radiologists’ reading behaviors will be educational. In this study, we proposed to leverage the newly emerged deep learning approach to investigate how the MLO and CC view images of a mammogram examination may have been clinically used by radiologists in coming up with a BI-RADS density category. We implemented a convolutional neural network (CNN)-based deep learning model, aimed at distinguishing the breast density categories using a large (15,415 images) set of real-world clinical mammogram images. Our results showed that the classification of density categories (in terms of area under the receiver operating characteristic curve) using MLO view images is significantly higher than that using the CC view. This indicates that most likely it is the MLO view that the radiologists have predominately used to determine the breast density BI-RADS categories. Our study holds a potential to further interpret radiologists’ reading characteristics, enhance personalized clinical training to radiologists, and ultimately reduce reader variations in breast density assessment.

62 RADIOLOGY AND NUCLEAR MEDICINE↗