Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “dl”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Unconventional spin-orbit torques from sputtered MoTe 2 films

Materials with strong spin-orbit coupling and low crystalline symmetry are promising for generating large unconventional spin-orbit torques (SOTs), such as in-plane fieldlike (FL) torques and out-of-plane dampinglike (DL) torques, which can effectively manipulate and deterministically switch an out-of-plane magnetization without the need for additional external in-plane magnetic fields. Here, we report SOTs generated by magnetron-sputtered 1⁢T' MoTe 2 /Permalloy (Py; Ni 80 ⁢Fe 20 )/MgO heterostructures using both spin-torque ferromagnetic resonance (ST-FMR) and second harmonic Hall measurements. We observed unconventional FL and DL torques in our samples due to spins polarized normal to the interface of MoTe 2 and Py layers, and studied the influence of crystallographic order and MoTe 2 layer thickness on the SOTs. By comparing the Raman spectra of 1⁢T' MoTe 2 samples prepared in different ways, we found a tensile strain in sputtered MoTe 2 films, which might further enhance the generation of unconventional torques by reducing the symmetry of 1⁢T' MoTe 2 .

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Deep learning simulations of the microwave sky

Here we present 500 high-resolution, full-sky millimeter-wave deep learning (DL) simulations that include lensed CMB maps and correlated foreground components. We find that these MillimeterDL simulations can reproduce a wide range of non-Gaussian summary statistics matching the input training simulations, while only being optimized to match the power spectra. The procedure we develop in this work enables the capability to mass produce independent full-sky realizations from a single expensive full-sky simulation, when ordinarily the latter would not provide enough training data. We also circumvent a common limitation of high-resolution DL simulations that they be confined to small sky areas, often due to memory or GPU issues; we do this by developing a “stitching” procedure that can recover the large-scale, high-order statistics and avoid discontinuities or repeated features in the maps. In addition, since our network takes as input a full-sky lensing convergence map, it can in principle take a full-sky lensing convergence map from any large-scale structure (LSS) simulation and generate the corresponding lensed CMB and correlated foreground components at millimeter wavelengths; this is especially useful in the current era of combining results from both CMB and LSS surveys, which require a common set of simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

Measurement of the decay B → D ℓ ν ℓ in fully reconstructed events and determination of the Cabibbo-Kobayashi-Maskawa matrix element | V c b |

We present a determination of the magnitude of the Cabibbo-Kobayashi-Maskawa matrix element |Vcb| using the decay B → Dℓν ℓ (ℓ = e,μ) based on 711 fb –1 of e + e – → Υ(4S) data recorded by the Belle detector and containing 772 × 10 6 BB¯ pairs. Here, one B meson in the event is fully reconstructed in a hadronic decay mode, while the other, on the signal side, is partially reconstructed from a charged lepton and either a D + or D 0 meson in a total of 23 hadronic decay modes. The isospin-averaged branching fraction of the decay B → Dℓν ℓ is found to be B(B 0 → D – ℓ + ν ℓ ) = (2.31 ± 0.03(stat) ± 0.11(syst))%. Analyzing the differential decay rate as a function of the hadronic recoil with the parametrization of Caprini, Lellouch, and Neubert and using the form-factor prediction G(1) = 1.0541 ± 0.0083 calculated by FNAL/MILC, we obtain η EW |V cb | = (40.12 ± 1.34) × 10 –3 , where ηEW is the electroweak correction factor. Alternatively, assuming the model-independent form-factor parametrization of Boyd, Grinstein, and Lebed and using lattice QCD data from the FNAL/MILC and HPQCD collaborations, we find η EW |V cb | = (41.10 ± 1.14) × 10 –3 .

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Machine learning approaches for crystallographic classification from synthetic 2D X-ray diffraction data

Crystallographic structure identification is crucial for understanding material properties; however, current methodologies often depend on labor-intensive and time-consuming analyses of 2D X-ray diffraction (XRD) patterns. To address these limitations, this study employs synthetic 2D XRD patterns combined with deep learning (DL) techniques to enable automated and high-throughput classification of the seven crystal systems and 230 space groups. We introduce the novel Auto Diffraction Pipeline, designed to generate synthetic 2D XRD spot patterns from crystallographic information files under diverse conditions, including varying zone axes, atomic substitution, atomic depletion and mechanical loading. These conditions enhance the realism of synthetic data, mitigating the scarcity of experimental datasets and enabling the creation of large representative training sets. Convolutional neural networks were trained and validated on these synthetic datasets to classify crystallographic structures across multiple scenarios. Our results demonstrate that integrating synthetic 2D XRD patterns with DL facilitates rapid, accurate and automated crystallographic classification, promoting the wider adoption of data-driven approaches in materials science.

Shahnazari, Ayoub [Univ. of Rochester, NY (United ↗
DOE OSTI2023

Toward a fully coherent tender and hard X-ray free-electron laser via cascaded EEHG in fourth-generation synchrotron light sources

Free-electron-laser-based beamlines utilize fully coherent laser pulses with extremely narrow bandwidth allowing direct use of X-rays without monochromators. This could be very beneficial for all users of current and future fourth-generation diffraction-limited synchrotron light sources (DL-SLSs) who need narrowband full-coherence high-brightness X-ray pulses. Based on our previous finding, i.e. that separating the two stages of echo-enabled harmonic generation (EEHG) with a few extra bending-magnet sections provides an effective way to increase the momentum compaction of chicane 1, one can simultaneously achieve adequate prebunching at extremely high harmonics as well as keep the energy modulation to the ideal minimum. This could open the door for cascaded EEHG, toward fully coherent tender and hard X-ray wavelengths. Built on our compact design of a twin-pulse seeding electron beam with an adjustable delay and timing jitter at the level of a few femtoseconds, a cascaded EEHG can be implemented, which includes two EEHG beamlines, where the radiation pulse generated by the first beamline with harmonic $h$ 1 could be used as the input seed laser pulse to the second beamline with harmonic $h$ 2 . Hence, the second radiator could potentially reach very high harmonics [$h$ = $h$ 1 (20)$h$ 2 (25–100)] from 500 to 2000, corresponding to tender and hard X-ray wavelengths. It is demonstrated that the cascaded EEHG scheme is compatible with almost any current or planned fourth-generation DL-SLS, with significant benefits for space-limited storage rings in particular. The main advantage is that this scheme requires almost no change of the storage-ring lattice and is fully compatible with other beamlines. Current proposals for rings with much longer straight sections would add self-amplified spontaneous emission as another viable option for storage-ring-based free-electron lasers.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Counter Data Paucity through Adversarial Invariance Encoding: A Case Study on Modeling Battery Thermal Runaway

Lithium-ion batteries, widely used for their durability and high energy storage, face the risk of internal short circuits leading to catastrophic thermal runaway events. These events, triggered by external stimuli like mechanical loads, pose safety concerns in applications such as electric vehicles. Detecting and understanding thermal runaway events is crucial, but physics-driven models struggle to explain the non-linear evolution of battery temperature during these events, considering factors like material composition and state-of-charge. Due to the rarity of these events and the cost of data collection, we propose a deep learning (DL) model to predict battery temperature responses during thermal runaway. The challenge lies in the scarcity of data, making traditional DL models prone to overfitting and learning low-quality representations of the complex process.Our approach introduces a novel few-shot architecture that incorporates an adversarially governed invariant encoding process. This architecture aims to distill "invariant" relationships by addressing distributional shifts in data across various battery properties, facilitating the detection of thermal runaway events. Specifically, our results demonstrate that deep learning models conditioned on these "invariant" representations outperform state-of-the-art baselines, achieving a remarkable 96.8% performance improvement in terms of the popular metric MAPE. This framework presents a promising direction for enhancing battery safety modeling, particularly in the context of rare and complex events like thermal runaway. Our code and code and dataset used for the paper are public1.

Tabassum, Anika [ORNL] (ORCID:0000000254600955)↗
DOE OSTI2022

Towards Low-Overhead Resilience for Data Parallel Deep Learning

Data parallel techniques have been widely adopted both in academia and industry as a tool to enable scalable training of deep learning models. At scale, DL training jobs can fail due to software or hardware bugs, may need to be preempted or terminated due to unexpected events, or may perform suboptimally because they were misconfigured. Under such circumstances, there is a need to recover and/or reconfigure data-parallel DL training jobs on-the-fly, while minimizing the impact on the accuracy of the DNN model and the runtime overhead. In this regard, state-of-art techniques adopted by the HPC community mostly rely on checkpoint-restart, which inevitably leads to loss of progress, thus increasing the runtime overhead. In this paper we explore alternative techniques that exploit the properties of modern deep learning frameworks (overlapping of gradient averaging and weight updates with local gradient computations through pipeline parallelism) to reduce the overhead of resilience/elasticity. To this end we introduce a failure simulation framework and two resilience strategies (immediate mini-batch rollback and lossy forward recovery), which we study compared with checkpoint-restart approaches in a variety of settings in order to understand the trade-offs between the accuracy loss of the DNN model and the runtime overhead.

data-parallel training↗
DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗
DOE OSTI2023

Uncertainty Aware Deep Learning for Fault Prediction Using Multivariate Time Series Signals

The superconducting radio-frequency cavities are a crucial component of the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. When a cavity faults, beam delivery to experimental end users is disrupted. Prediction of cavity faults prior to onset is essential to reduce operation and maintenance costs. In this work, a parallel long short-term memory (LSTM)-convolution neural network (CNN)-based deep learning (DL) model is proposed to predict impending faults using pre-fault signals. Further, we introduce an uncertainty quantification approach using Monte Carlo dropout with the LSTM-CNN model to ascertain confidence in the prediction. The model was tested using multivariate time series signals from stable cavity operations and before faults. Initial results show that on the test dataset, the model can identify impending faults before their onset with an average 10-fold cross validation accuracy of 97.39% and a standard deviation of 0.12% using a 100-ms time window. It is also observed that the model performs better as the prediction time moves closer to the fault onset. For additional context, we compare the performance of the model with three machine-learning-based (ML) fault prediction models. Our proposed parallel LSTM-CNN-based DL method shows better performance than the ML-based methods.

Rahman, Md Monibor↗
DOE OSTI2026

Dynamic Validation of CNN-Based Surrogate Models for Inverter-Based Resources in Open-Source Solvers

Traditionally, distribution system planning has focused on steady-state analyses, with limited consideration of dynamic behavior. However, as large or medium-scale inverter-based resources (IBRs), particularly grid-following (GFL) inverters in commercial or industry buildings, become more prevalent, understanding their dynamic impact is essential for grid planning and operation. This article presents an innovative deep-learning (DL)-approach using convolutional neural networks technique to model the GFL inverters. Developed from real grid-tied commercial IBR transient data, these dynamic DL models overcome proprietary constraints by requiring minimal knowledge of internal converter physics while maintaining high accuracy and flexibility. To demonstrate their applicability, the models were incorporated into GridLAB-D, an open-source, three-phase distribution analysis tool. This integration enables dynamic simulations of large-scale distribution networks with high IBR penetration stability analysis. Rigorous testing and validation, aligned with industry standards, confirmed the reliability and efficiency of this approach, paving the way for enhanced planning and operational assessments of modern power systems.

Deep-learning↗
DOE OSTI2021

Unbundling Smart Meter Services Through Spatio-Temporal Decomposition Agents in DER-rich Environment

Smart meters and the advanced metering infrastructure (AMI) facilitate distribution system operators (DSOs) to gather information on energy consumption at the customer level. With the increasing penetration of building-level intermittent distributed energy resources (DERs) behind the meter, DER information is not available to DSOs. At the same time, smart meter enables users to participate in grid, with real-time information. Information for behind the meter is needed by user to coordinate building level assets for maximum benefits. The concept of unbundled smart meter (USM) needs agents to decompose smart meter measurements to provide service to DSO as well as customers. In this paper, we propose a Spatio-Temporal Decomposition Agent (STDA) for USM based on Artificial Intelligence (AI). STDA can help users optimize their energy usage, help DSO to utilize building assets for the grid operation. The energy usage strategy developed by STDA is suitable for different users, and can be customized by deep learning (DL) models according to the different energy consumption habits of each user. The power prediction performance results of various DL models and evaluation using a set of data from a Hawaii utility is presented. Furthermore, STDA integration with Home Energy management Systems (HEMS) to manage resources is presented and validated. STDA pre-processes the measurements before model training, and provides the spatio-temporal decomposed forecasting.

42 ENGINEERING↗
DOE OSTI2022

Frequency Domain Detection and Precoding for Massive MIMO With Single Carrier Modulation

Single carrier modulation (SCM) schemes are attractive for uplink (UL) transmissions due to improved power efficiency at the user equipment (UE) transmitter compared with multi-carrier modulation schemes. In a massive MIMO scenario with SCM, the UL detection must mitigate the effects of inter-symbol interference and multiuser interference. This processing is effectively performed in the frequency domain (FD) using a minimum mean squared error (MMSE) detector when the transmission is framed with a cyclic prefix. This paper presents a MMSEbased detector that is compatible with distributed processing in a time-division duplex (TDD) system. The matrix inverses computed for the UL detection are then reused to perform multi-user precoding for the downlink (DL). We find that this scheme yields a tremendous savings in computational complexity compared to commonly used zero-forcing (ZF) precoding without sacrificing any performance. Since MMSE processing introduces a bias to the estimates, we then derive the scalar coefficients necessary to cancel the MMSE bias. The impact of channel estimation errors are analyzed for both the UL and DL cases in conjunction with a power-efficient approach to SCM channel estimation. Moreover, extensive simulations are performed to confirm our theoretical findings.

42 ENGINEERING↗
DOE OSTI2023

Effect of image resolution on automated classification of chest X-rays

Deep learning (DL) models have received much attention lately for their ability to achieve expert-level performance on the accurate automated analysis of chest X-rays (CXRs). Recently available public CXR datasets include high resolution images, but state-of-the-art models are trained on reduced size images due to limitations on graphics processing unit memory and training time. As computing hardware continues to advance, it has become feasible to train deep convolutional neural networks on high-resolution images without sacrificing detail by downscaling. This study examines the effect of increased resolution on CXR classification performance. We used the publicly available MIMIC-CXR-JPG dataset, comprising 377,110 high resolution CXR images for this study. We applied image downscaling from native resolution to 2048 × 2048 pixels, 1024 × 1024 pixels, 512 × 512 pixels, and 256 × 256 pixels and then we used the DenseNet121 and EfficientNet-B4 DL models to evaluate clinical task performance using these four downscaled image resolutions. We find that while some clinical findings are more reliably labeled using high resolutions, many other findings are actually labeled better using downscaled inputs. We qualitatively verify that tasks requiring a large receptive field are better suited to downscaled low resolution input images, by inspecting effective receptive fields and class activation maps of trained models. Lastly, we show that stacking an ensemble across resolutions outperforms each individual learner at all input resolutions while providing interpretable scale weights, indicating that diverse information is extracted across resolutions.

47 OTHER INSTRUMENTATION↗
DOE OSTI2015

Insights into Substrate Specificity of NlpC/P60 Cell Wall Hydrolases Containing Bacterial SH3 Domains

ABSTRACT Bacterial SH3 (SH3b) domains are commonly fused with papain-like Nlp/P60 cell wall hydrolase domains. To understand how the modular architecture of SH3b and NlpC/P60 affects the activity of the catalytic domain, three putative NlpC/P60 cell wall hydrolases were biochemically and structurally characterized. These enzymes all have γ- d -Glu-A 2 pm (A 2 pm is diaminopimelic acid) cysteine amidase (or dl -endopeptidase) activities but with different substrate specificities. One enzyme is a cell wall lysin that cleaves peptidoglycan (PG), while the other two are cell wall recycling enzymes that only cleave stem peptides with an N-terminal l -Ala. Their crystal structures revealed a highly conserved structure consisting of two SH3b domains and a C-terminal NlpC/P60 catalytic domain, despite very low sequence identity. Interestingly, loops from the first SH3b domain dock into the ends of the active site groove of the catalytic domain, remodel the substrate binding site, and modulate substrate specificity. Two amino acid differences at the domain interface alter the substrate binding specificity in favor of stem peptides in recycling enzymes, whereas the SH3b domain may extend the peptidoglycan binding surface in the cell wall lysins. Remarkably, the cell wall lysin can be converted into a recycling enzyme with a single mutation. IMPORTANCE Peptidoglycan is a meshlike polymer that envelops the bacterial plasma membrane and bestows structural integrity. Cell wall lysins and recycling enzymes are part of a set of lytic enzymes that target covalent bonds connecting the amino acid and amino sugar building blocks of the PG network. These hydrolases are involved in processes such as cell growth and division, autolysis, invasion, and PG turnover and recycling. To avoid cleavage of unintended substrates, these enzymes have very selective substrate specificities. Our biochemical and structural analysis of three modular NlpC/P60 hydrolases, one lysin, and two recycling enzymes, show that they may have evolved from a common molecular architecture, where the substrate preference is modulated by local changes. These results also suggest that new pathways for recycling PG turnover products, such as tracheal cytotoxin, may have evolved in bacteria in the human gut microbiome that involve NlpC/P60 cell wall hydrolases.

Xu, Qingping↗
DOE OSTI2018

SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks

Going deeper and wider in neural architectures improves their accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) practitioners either need to change to less de- sired network architectures, or nontrivially dissect a network across multiGPUs. These distract DL practitioners from concentrating on their original machine learning tasks. We present SuperNeurons: a dynamic GPU memory scheduling runtime to enable the network training far beyond the GPU DRAM capacity. SuperNeurons features 3 memory optimizations, Liveness Analysis, Unified Tensor Pool, and Cost-Aware Recomputation; together they effectively reduce the network-wide peak memory usage down to the maximal memory usage among layers. We also address the performance issues in these memory-saving techniques. Given the limited GPU DRAM, SuperNeurons not only provisions the necessary memory for the training, but also dynamically allocates the memory for convolution workspaces to achieve the high performance. Evaluations against Caffe, Torch, MXNet and TensorFlow have demonstrated that SuperNeurons trains at least 3.2432 deeper network than current ones with the leading performance. Particularly, SuperNeurons can train ResNet2500 that has 104 basic network layers on a 12GB K40c.

Wang, Linnan↗
DOE OSTI2021

Diagnosing autonomous vehicle driving criteria with an adversarial evolutionary algorithm

We repurposed an adversarial evolutionary algorithm, Gremlin, from finding driving scenarios where a model of an autonomous vehicle drove poorly to troubleshooting driving quality evaluation criteria. We evaluated the driving performance of a "perfect driver" robot in a virtual town environment using the same fitness criteria intended for a deep learner (DL) trained driver. We found that the fitness evaluation criteria poorly handled turns, and used Gremlin to iteratively improve that criteria. We were confident that the same criteria could then be applied to the DL-based models as originally intended, and that this approach could be used as a general means of troubleshooting autonomous vehicle driving criteria.

Coletti, Mark↗
DOE OSTI2020

Cryptococcus neoformans Secretes Small Molecules That Inhibit IL-1 β Inflammasome-Dependent Secretion

Cryptococcus neoformans is an encapsulated yeast that causes disease mainly in immunosuppressed hosts. It is considered a facultative intracellular pathogen because of its capacity to survive and replicate inside phagocytes, especially macrophages. This ability is heavily dependent on various virulence factors, particularly the glucuronoxylomannan (GXM) component of the polysaccharide capsule. Inflammasome activation in phagocytes is usually protective against fungal infections, including cryptococcosis. Nevertheless, recognition of C. neoformans by inflammasome receptors requires specific changes in morphology or the opsonization of the yeast, impairing proper inflammasome function. In this context, we analyzed the impact of molecules secreted by C. neoformans B3501 strain and its acapsular mutant Δcap67 in inflammasome activation in an in vitro model. Our results showed that conditioned media derived from B3501 was capable of inhibiting inflammasome-dependent events (i.e., IL-1β secretion and LDH release via pyroptosis) more strongly than conditioned media from Δcap67, regardless of GXM presence. We also demonstrated that macrophages treated with conditioned media were less responsive against infection with the virulent strain H99, exhibiting lower rates of phagocytosis, increased fungal burdens, and enhanced vomocytosis. Moreover, we showed that the aromatic metabolite DL-Indole-3-lactic acid (ILA) and DL-p-Hydroxyphenyllactic acid (HPLA) were present in B3501’s conditioned media and that ILA alone or with HPLA is involved in the regulation of inflammasome activation by C. neoformans. These results were confirmed by in vivo experiments, where exposure to conditioned media led to higher fungal burdens in Acanthamoeba castellanii culture as well as in higher fungal loads in the lungs of infected mice. Overall, the results presented show that conditioned media from a wild-type strain can inhibit a vital recognition pathway and subsequent fungicidal functions of macrophages, contributing to fungal survival in vitro and in vivo and suggesting that secretion of aromatic metabolites, such as ILA, during cryptococcal infections fundamentally impacts pathogenesis.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

PathRepHAN: Hierarchical attention networks for pathology report classification

Developers at ORNL explored how a deep learning (DL) approach based on hierarchical attention networks (HANs) can improve model performance for multiple information extraction tasks from unstructured cancer pathology reports compared to conventional methods that do not sufficiently capture syntactic and semantic contexts from free-text documents. The HAN was implemented for 2 information extraction tasks: (1) primary site, matched to 12 International Classification of Diseases for Oncology topography codes (7 breast, 5 lung primary sites), and (2) histological grade classification, matched to G1–G4. Our results demonstrate that for both information tasks, HAN performed significantly better compared to the conventional ML and DL techniques.

Ramanathan, Arvind↗