Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Influence of deep levels on the electrical transport properties of CdZnTeSe detectors

Here, we investigated the influence of deep levels on the electrical transport properties of CdZnTeSe radiation detectors by comparing experimental data with numerical simulations based on the simultaneous solution of drift-diffusion and Posisson equations, including the Shockley-Read-Hall model of the carrier trapping. We determined the Schottky barrier heights and the Fermi level position from I-V measurements. We measured the time evolution of the electric field and the electrical current after the application of a voltage bias. We observed that the electrical properties of CZTS are fundamentally governed by two deep levels close to the mid-bandgap - one recombination and one hole trap. We show that the hole trap indirectly increases the mobility-lifetime product of electrons. We conclude that the structure of deep levels in CZTS are favorable for high electrical charge transport.

36 MATERIALS SCIENCE↗

Digital twins and deep learning segmentation of defects in monolayer MX 2 phases

Developing methods to understand and control defect formation in nanomaterials offers a promising route for materials discovery. Monolayer MX 2 phases represent a particularly compelling case for defect engineering of nanomaterials due to the large variability in their physical properties as different defects are introduced into their structure. However, effective identification and quantification of defects remain a challenge even as high-throughput scanning transmission electron microscopy methods improve. This study highlights the benefits of employing first principles calculations to produce digital twins for training deep learning segmentation models for defect identification in monolayer MX 2 phases. Around 600 defect structures were obtained using density functional theory calculations, with each monolayer MX 2 structure being subjected to multislice simulations for the purpose of generating the digital twins. Several deep learning segmentation architectures were trained on this dataset, and their performances evaluated under a variety of conditions such as recognizing defects in the presence of unidentified impurities, beam damage, grain boundaries, and with reduced image quality from low electron doses. Further, this digital twin approach allows benchmarking different deep learning architectures on a theory dataset, which enables the study of defect classification under a broad array of finely controlled conditions. It thus opens the door to resolving the underpinning physical reasons for model shortcomings and potentially chart paths forward for automated discovery of materials defect phases in experiments.

36 MATERIALS SCIENCE↗

Deep learning to represent subgrid processes in climate models

The representation of nonlinear subgrid processes, especially clouds, has been a major source of uncertainty in climate models for decades. Cloud-resolving models better represent many of these processes and can now be run globally but only for short-term simulations of at most a few years because of computational limitations. Here we demonstrate that deep learning can be used to capture many advantages of cloud-resolving modeling at a fraction of the computational cost. We train a deep neural network to represent all atmospheric subgrid processes in a climate model by learning from a multiscale model in which convection is treated explicitly. The trained neural network then replaces the traditional subgrid parameterizations in a global general circulation model in which it freely interacts with the resolved dynamics and the surface-flux scheme. The prognostic multiyear simulations are stable and closely reproduce not only the mean climate of the cloud-resolving simulation but also key aspects of variability, including precipitation extremes and the equatorial wave spectrum. Furthermore, the neural network approximately conserves energy despite not being explicitly instructed to. Finally, we show that the neural network parameterization generalizes to new surface forcing patterns but struggles to cope with temperatures far outside its training manifold. Our results show the feasibility of using deep learning for climate model parameterization. In a broader context, we anticipate that data-driven Earth system model development could play a key role in reducing climate prediction uncertainty in the coming decade.

54 ENVIRONMENTAL SCIENCES↗

Geological activity shapes the microbiome in deep-subsurface aquifers by advection

Subsurface environments host diverse microorganisms in fluid-filled fractures; however, little is known about how geological and hydrological processes shape the subterranean biosphere. Here, we sampled three flowing boreholes weekly for 10 mo in a 1478-m-deep fractured rock aquifer to study the role of fracture activity (defined as seismically or aseismically induced fracture aperture change) and advection on fluid-associated microbial community composition. We found that despite a largely stable deep-subsurface fluid microbiome, drastic community-level shifts occurred after events signifying physical changes in the permeable fracture network. The community-level shifts include the emergence of microbial families from undetected to over 50% relative abundance, as well as the replacement of the community in one borehole by the earlier community from a different borehole. Null-model analysis indicates that the observed spatial and temporal community turnover was primarily driven by stochastic processes (as opposed to deterministic processes). We, therefore, conclude that the observed community-level shifts resulted from the physical transport of distinct microbial communities from other fracture(s) that outpaced environmental selection. Given that geological activity is a major cause of fracture activity and that geological activity is ubiquitous across space and time on Earth, our findings suggest that advection induced by geological activity is a general mechanism shaping the microbial biogeography and diversity in deep-subsurface habitats across the globe.

59 BASIC BIOLOGICAL SCIENCES↗

Inferring safety critical events from vehicle kinematics in naturalistic driving environment: Application of deep learning Algorithms

Advances in sensing technology has enabled the collection of countless terabytes of second-by-second kinematics data. Such data provides opportunities for real-time monitoring of driving behavior and identification of safety critical events (SCEs) including crashes and near crashes. The concept of volatility is relevant in this context, which identifies instability and erratic variations in driving behavior prior to involvement in SCEs. This study utilized vehicle kinematics from a large-scale naturalistic driving data to develop a deep learning approach based on 1D convolutional neural networks (CNN) for inferring SCEs. The data are unique in the sense that such accurate pre-crash data at high fidelity are not available in traditional crash repositories. This study contributes to the literature by providing a first attempt at predicting responses to SCEs by developing deep learning-based CNN architectures using novel driving volatility based kinematic thresholds for a sample of 9553 events. The key contribution lies in developing a volatility-based CNN input layout that is acceptable to CNN schemes and represents the motion kinematics such as speed, acceleration and volatility measures. Several 1D-CNN architectures were developed using layers, numbers of convolutions, layer patterns, and kernels. Shallow and deep architectures were tested, revealing higher accuracy of shallow architectures in detecting SCEs. The optimal number of epochs were identified using an early stopping method while the CNN performance was improved by increasing the number of epochs. The ensemble CNN had the highest predictive accuracy of 95.6% for detection of crashes and near crashes, which was 2.5% higher than the optimal CNN using 20% hold out test data. The ensemble CNN also outperformed classical machine learning models and model performance reported in past studies on detection of SCEs. Finally, these results have implications for identification of safety hotspots and providing real-time alerts and warnings in connected and highly automated vehicle environment including society of automotive engineers levels 3–5.

42 ENGINEERING↗

Deep level defects in dilute GaAsBi alloys grown under intense UV illumination

Dilute GaAs1-xBix alloys exhibiting narrow band edge photoluminescence (PL) were recently grown by molecular beam epitaxy (MBE) with the growth surface illuminated by intense UV radiation. To investigate whether the improved optical quality of these films results from a reduction in the concentration of deep level defects, p+/n and n+/p junction diodes were fabricated on both the illuminated and dark areas of several samples. Deep Level Transient Spectroscopy (DLTS) measurements show that the illuminated and dark areas of both the n- and p-type GaAs1-xBix epi-layers have similar concentrations of near mid-gap electron and hole traps, in the 1015 cm-3 range. Thus the improved PL spectra cannot be explained by a reduction in non-radiative recombination at deep level defects. We note that carrier freeze-out above 35 K is significantly reduced in the illuminated areas of the p-type GaAs1-xBix layers compared to the dark areas, allowing the first DLTS measurements of defect energy levels close to the valence band edge. These defect levels may account for differences in the PL spectra from the illuminated and dark areas of un-doped layers with a similar Bi fraction.

14 SOLAR ENERGY↗

DeepAdversaries: examining the robustness of deep learning models for galaxy morphology classification

With increased adoption of supervised deep learning methods for work with cosmological survey data, the assessment of data perturbation effects (that can naturally occur in the data processing and analysis pipelines) and the development of methods that increase model robustness are increasingly important. In the context of morphological classification of galaxies, we study the effects of perturbations in imaging data. In particular, we examine the consequences of using neural networks when training on baseline data and testing on perturbed data. We consider perturbations associated with two primary sources: (a) increased observational noise as represented by higher levels of Poisson noise and (b) data processing noise incurred by steps such as image compression or telescope errors as represented by one-pixel adversarial attacks. We also test the efficacy of domain adaptation techniques in mitigating the perturbation-driven errors. We use classification accuracy, latent space visualizations, and latent space distance to assess model robustness in the face of these perturbations. For deep learning models without domain adaptation, we find that processing pixel-level errors easily flip the classification into an incorrect class and that higher observational noise makes the model trained on low-noise data unable to classify galaxy morphologies. On the other hand, we show that training with domain adaptation improves model robustness and mitigates the effects of these perturbations, improving the classification accuracy up to 23% on data with higher observational noise. Domain adaptation also increases up to a factor of ${\approx}2.3$ the latent space distance between the baseline and the incorrectly classified one-pixel perturbed image, making the model more robust to inadvertent perturbations. Successful development and implementation of methods that increase model robustness in astronomical survey pipelines will help pave the way for many more uses of deep learning for astronomy.

79 ASTRONOMY AND ASTROPHYSICS↗

Copper-related deep-level centers in irradiated p-type silicon

Deep-level centers are investigated in the p-type Si on copper-contaminated samples which were also electron irradiated. Standard and Laplace-transform deep-level transient spectroscopy techniques were employed to characterize the samples. Several Cu-related centers are observed to form either as a result of the low-temperature Cu diffusion into the irradiated crystals or due to irradiation of the Cu-contaminated samples and subsequent annealing up to 400 deg. C. In all crystals, two Cu-related defects are found to be the most abundant; each of them possesses a pair of levels in the lower half of the gap. The Arrhenius signatures for one pair are measured to be practically identical to those for the donor and acceptor levels of substitutional copper Cu{sub s}, respectively, the levels of other defect being only barely different from the Cu{sub s} levels. Analysis of the introduction rates and depth profiles of the deep-level centers points to the vacancy-oxygen complex (VO, the A center) as the precursor of the most abundant Cu-related defects. It is inferred that Cu{sub s} is formed in irradiated silicon due to interaction with the VO centers via the rather stable intermediate CuVO complex.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Aligned breakup of heavy nuclear systems as a new type of deep inelastic collisions at small impact parameters

An interesting process of violent reseparation of a heavy nuclear system into three or four fragments of comparable size was recently observed in {sup 197}Au+{sup 197}Au collisions at 15 MeV/nucleon. Combined analysis of the binary deep inelastic events and the ternary and quaternary breakup events demonstrates that the newly observed ternary and quaternary reactions belong to the same wide class of deep inelastic collisions as the conventional (binary) damped reactions. It is shown that the ternary and quaternary breakup reactions occur at extremely inelastic collisions corresponding to small impact parameters, while more peripheral collisions lead to well-known binary deep inelastic reactions.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Inferring effective electrostatic interaction of charge-stabilized colloids from scattering using deep learning

In this article, an innovative strategy is presented that incorporates deep auto-encoder networks into a least-squares fitting framework to address the potential inversion problem in small-angle scattering. To evaluate the performance of the proposed approach, a detailed case study focusing on charged colloidal suspensions was carried out. The results clearly indicate that a deep learning solution offers a reliable and quantitative method for studying molecular interactions. The approach surpasses existing deterministic approaches with respect to both numerical accuracy and computational efficiency. Overall, this work demonstrates the potential of deep learning techniques in tackling complex problems in soft-matter structures and beyond.

36 MATERIALS SCIENCE↗

Survey of Deep Learning and Physics-Based Approaches in Computational Wave Imaging

Computational wave imaging (CWI) extracts hidden structure and physical properties of a volume of material by analyzing wave signals that traverse that volume. Applications include seismic exploration of the Earth’s subsurface, acoustic imaging and nondestructive testing (NDT) in material science, and ultrasound computed tomography (USCT) in medicine. Current approaches for solving CWI problems can be divided into two categories: those rooted in traditional physics and those based on deep learning. Physics-based methods stand out for their ability to provide high-resolution and quantitatively accurate estimates of acoustic properties within the medium. However, they can be computationally intensive and are susceptible to ill-posedness and nonconvexity typical of CWI problems. Machine learning (ML)-based computational methods have recently emerged, offering a different perspective to address these challenges. Diverse scientific communities have independently pursued the integration of deep learning in CWI. This review discusses how contemporary scientific ML techniques, and deep neural networks in particular, have been developed to enhance and integrate with traditional physics-based methods for solving CWI problems. We present a structured framework that consolidates existing research spanning multiple domains, including computational imaging, wave physics, and data science. This study concludes with important lessons learned from existing ML-based methods and identifies technical hurdles and emerging trends through a systematic analysis of the extensive literature on this topic.

42 ENGINEERING↗

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗

Deep Learning-Based Weather-Related Power Outage Prediction with Socio-Economic and Power Infrastructure Data

This paper presents a deep learning-based approach for hourly power outage probability prediction within census tracts encompassing a utility company's service territory. Two distinct deep learning models, conditional Multi-Layer Perceptron (MLP) and unconditional MLP, were developed to forecast power outage probabilities, leveraging a rich array of input features gathered from publicly available sources including weather data, weather station locations, power infrastructure maps, socio-economic and demographic statistics, and power outage records. Given a one-hour-ahead weather forecast, the models predict the power outage probability for each census tract, taking into account both the weather prediction and the location's characteristics. The deep learning models employed different loss functions to optimize prediction performance. Our experimental results underscore the significance of socio-economic factors in enhancing the accuracy of power outage predictions at the census tract level.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DeepWelding: a Deep Learning Enhanced Approach to GTAW Using Multi-source Sensing Images

Deep learning has great potential to reshape manufacturing industries. In this paper, we present DeepWelding, a novel framework that applies deep learning techniques to improve gas tungsten arc welding (GTAW) process monitoring and penetration detection using multi-source sensing images. The framework is capable of analyzing multiple types of optical sensing images synchronously and consists of three deep learning enhanced consecutive phases: image preprocessing, image selection, and weld penetration classification. Specifically, we adopted generative adversarial networks (pix2pix) for image denoising and classic convolutional neural networks (AlexNet) for image selection. Both pix2pix and AlexNet delivered satisfactory performance. However, five individual neural networks with heterogeneous architectures demonstrated inconsistent generalization capabilities in the classification phase when holding out multi-source images generated with specific experimental settings. Therefore, two ensemble methods combining multiple neural networks were designed to improve the model performance on unseen data collected from different experimental settings. In conclusion, we also found that the quality of model prediction was heavily influenced by the data stream collection environment. We think these findings are beneficial for the broad intelligent welding community.

42 ENGINEERING↗

Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data With Spatial Information

Efficient processing of large-scale time series data is an intricate problem in machine learning. Conventional sensor signal processing pipelines with hand engineered feature extraction often involve huge computational cost with high dimensional data. Deep recurrent neural networks have shown promise in automated feature learning for improved time-series processing. However, generic deep recurrent models grow in scale and depth with increased complexity of the data. This is particularly challenging in presence of high dimensional data with temporal and spatial characteristics. Consequently, this work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to efficiently process complex multi-dimensional time series data with spatial information. Here, the cellular recurrent architecture in the proposed model allows for location-aware synchronous processing of time series data from spatially distributed sensor signal sources. Extensive trainable parameter sharing due to cellularity in the proposed architecture ensures efficiency in the use of recurrent processing units with high-dimensional inputs. This study also investigates the versatility of the proposed DCRNN model for classification of multi-class time series data from different application domains. Consequently, the proposed DCRNN architecture is evaluated using two time-series datasets: a multichannel scalp EEG dataset for seizure detection, and a machine fault detection dataset obtained in-house. The results suggest that the proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.

60 APPLIED LIFE SCIENCES↗

A Study of Complex Deep Learning Networks on High-Performance, Neuromorphic, and Quantum Computers

Current deep learning approaches have been very successful using convolutional neural networks trained on large graphical-processing-unit-based computers. Three limitations of this approach are that (1) they are based on a simple layered network topology, i.e., highly connected layers, without intra-layer connections; (2) the networks are manually configured to achieve optimal results, and (3) the implementation of the network model is expensive in both cost and power. In this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. We use the MNIST dataset for our experiment, due to input size limitations of current quantum computers. Our results show the feasibility of using the three architectures in tandem to address the above deep learning limitations. Finally, we show that a quantum computer can find high quality values of intra-layer connection weights in a tractable time as the complexity of the network increases, a high performance computer can find optimal layer-based topologies, and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware.

97 MATHEMATICS AND COMPUTING↗

Indications of a Decrease in the Depth of Deep Convective Cores with Increasing Aerosol Concentration during the CACTI Campaign

Abstract An aerosol indirect effect on deep convective cores (DCCs), by which increasing aerosol concentration increases cloud-top height via enhanced latent heating and updraft velocity, has been proposed in many studies. However, the magnitude of this effect remains uncertain due to aerosol measurement limitations, modulation of the effect by meteorological conditions, and difficulties untangling meteorological and aerosol effects on DCCs. The Cloud, Aerosol, and Complex Terrain Interactions (CACTI) campaign in 2018–19 produced concentrated aerosol and cloud observations in a location with frequent DCCs, providing an opportunity to examine the proposed aerosol indirect effect on DCC depth in a rigorous and robust manner. For periods throughout the campaign with well-mixed boundary layers, we analyze relationships that exist between aerosol variables (condensation nuclei concentration > 10 nm, 0.4% cloud condensation nuclei concentration, 55–1000-nm aerosol concentration, and aerosol optical depth) and meteorological variables [level of neutral buoyancy (LNB), convective available potential energy, midlevel relative humidity, and deep-layer vertical wind shear] with the maximum radar-echo-top height and cloud-top temperature (CTT) of DCCs. Meteorological variables such as LNB and deep-layer shear are strongly correlated with DCC depth. LNB is also highly correlated with three of the aerosol variables. After accounting for meteorological correlations, increasing values of the aerosol variables [with the exception of one formulation of aerosol optical depth (AOD)] are generally correlated at a statistically significant level with a warmer CTT of DCCs. Therefore, for the study region and period considered, increasing aerosol concentration is mostly associated with a decrease in DCC depth.

54 ENVIRONMENTAL SCIENCES↗

Combining multitask and transfer learning with deep Gaussian processes for autotuning-based performance engineering

We combine deep Gaussian processes (DGPs) with multitask and transfer learning for the performance modeling and optimization of HPC applications. Deep Gaussian processes merge the uncertainty quantification advantage of Gaussian processes (GPs) with the predictive power of deep learning. Multitask and transfer learning allow for improved learning efficiency when several similar tasks are to be learned simultaneously and when previous learned models are sought to help in the learning of new tasks, respectively. A comparison with state-of-the-art autotuners shows the advantage of our approach on two application problems. In this article, we combine DGPs with multitask and transfer learning to allow for both an improved tuning of an application parameters on problems of interest but also the prediction of parameters on any potential problem the application might encounter.

97 MATHEMATICS AND COMPUTING↗