Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scalable deep learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Capsule network-based semantic segmentation model for thermal anomaly identification on building envelopes

Thermography technology is widely used to inspect thermal anomalies in building façade systems. Computer vision-based techniques provide opportunities to autonomously detect such heat anomalies to significantly improve the efficiency of decision-making for building envelope retrofitting and maintenance. Here, in this work, we propose a novel Capsule Network-based deep learning model – CapsLab – that detects and identifies thermal anomalies by semantic segmentation. CapsLab is built based on our proposed prediction-tuning capsule (PT-Capsule) layer. Different from a traditional capsule layer, which consists of part-whole transformation and capsule-routing process, the proposed layer is composed of a prediction and tuning process, which helps decreasing the number of model parameters significantly. While the applicability of traditional Capsule Networks (CapsNets) has been limited to simpler tasks and smaller datasets due to their scalability issue, we can leverage the lightweight of the proposed PT-Capsule layer, and apply it to the semantic segmentation task. In this work, we also employ our previously presented performance metric, referred to as the Anomaly Identification Metric (AIM) (Kakillioglua et al. 2021), to evaluate the segmentation outputs. Traditional performance metrics do not accurately reflect the true performance of the segmentation models in thermal anomaly identification due to the high subjectivity in the annotation process and higher overlap ratio sensitivity of the standard metrics. AIM, on the other hand, is robust to these drawbacks. Experimental results show, both qualitatively and quantitatively, that our proposed segmentation method can effectively segment the thermal anomalies. Specifically, our model provides 9.38% and 13.53% improvements over the baseline model – DeepLabV3+ – based on traditional mIoU score and the AIM score, respectively, while requiring less model parameters and less computation at the same time. In addition, the scores that the AIM metric generates better align with the scores provided by building performance experts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for alternating current optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvatureinformed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.

alternating current optimal power flow↗
DOE OSTI2022

Structure-Informed Graph Learning of Networked Dependencies for Online Prediction of Power System Transient Dynamics

Online transient analysis plays an increasingly important role in dynamic power grids as the renewable generation continues growing. Traditional numerical methods for transient analysis not only are computationally intensive but also require precise contingency information as input, and therefore, are not suitable for online applications. Existing online transient assessment studies focus on the determination of post-contingency system stability or stability margin. Here, this paper develops a novel graph-learning framework, Deep-learning Neural Representation or DNR, for online prediction, of the time-series trajectories of the system states using initial system responses that can be measured by phasor measurement units (PMUs). The proposed DNR framework consists of two sequential modules: a Network Constructor that captures network dependencies among generators, and a Dynamics Predictor that predicts the system trajectories. The key to improved prediction performance is the introduction of the spatio-temporal message-passing operations into graph neural networks with structural knowledge. Its effectiveness and scalability are validated through comparative studies, demonstrating the prediction performance under different contingency scenarios for systems of different sizes. This framework provides a solution to online predicting post-fault system dynamics based on real-time PMU measurements. Additionally, it can also be applied to facilitate the offline transient simulation without simulating the entire trajectories.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Machine learning enables reconstruction of past fire regimes from charcoal-derived fire intensity and fuel composition

Background Fire is a foundational ecological process that shapes ecosystem structure, diversity, and resilience. Quantifying paleofire regime attributes such as frequency, severity, and intensity is essential for understanding the historical range of variability in fire behavior and its ecological effects. While frequency and severity are often reconstructed in paleofire studies, quantitative reconstructions of fire intensity remain limited. Recent work has shown that maximum pyrolysis temperature—a proxy for fire intensity—and plant species type can be inferred from charcoal using transmission Fourier-transform infrared (FTIR) spectroscopy. However, the sample preparation for transmission FTIR is destructive and time-consuming, limiting application and reuse of materials for other analyses. We evaluated reflectance FTIR spectroscopy as a non-destructive alternative for reconstructing combustion temperature and plant species from laboratory-generated charcoal. We also examined the influence of contrasting airflow environments (ambient air versus nitrogen-rich) on pyrolysis temperature and plant species reconstruction prediction accuracies and compared predictive performance between a novel, neural network–based deep learning model with the traditional modern analogue technique (MAT) using k-nearest neighbor functions. As proof of concept, we apply our enhanced methodology to ancient charcoal to demonstrate applicability at improving long-term fire regime reconstructions and the ability to link paleofire records with contemporary fire ecology. Results Our analysis shows that transmission and reflectance FTIR spectra yield comparable spectral profiles. However, sample preparation for reflectance FTIR is minimal and non-destructive, unlike transmission FTIR which is destructive. We demonstrate that oxygen environments improved reconstruction accuracy relative to nitrogen-rich conditions. Finally, our deep learning neural network (DL) achieved testing accuracies of 98.7% for temperature and 96.2% for species identification, outperforming MAT’s k-NN approach (89.8% and 65.9%, respectively). A Shapley importance analysis identified 5 key spectral regions that greatly influenced the model’s temperature or species categorization. When applied to ancient charcoal, our results show historic fires from the most recent past primarily burned at low intensities (400–500 °C), reflective of natural fire regimes in ponderosa pine forests. Our results corroborate charcoal morphology data that suggests all ancient charcoal originated from burned woody plant types. Conclusions By combining reflectance FTIR spectroscopy with a deep learning approach, we provide the first accuracies high enough to confidently identify both species and temperature from laboratory-produced charcoal, improving quantitative reconstructions of fire intensity and fuel composition from paleofire records. This opens a wide range of research into the link between fire and larger drivers (i.e., climate or human) and greater ecological understanding of fire regimes beyond that of burn scars or recent observations. These methodological improvements have direct relevance for fire management by improving interpretation of historical fire behavior, informing fuel–fire relationships, and providing a scalable analytical framework applicable to both long-term ecological studies and contemporary fire science.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Evaluation of Portable Acceleration Solutions for LArTPC Simulation Using Wire-Cell Toolkit

The Liquid Argon Time Projection Chamber (LArTPC) technology plays an essential role in many current and future neutrino experiments. Accurate and fast simulation is critical to developing efficient analysis algorithms and precise physics model projections. The speed of simulation becomes more important as Deep Learning algorithms are getting more widely used in LArTPC analysis and their training requires a large simulated dataset. Heterogeneous computing is an efficient way to delegate computationally intensive tasks to specialized hardware. However, as the landscape of compute accelerators quickly evolves, it becomes increasingly difficult to manually adapt the code to the latest hardware or software environments. A solution which is portable to multiple hardware architectures without substantially compromising performance would thus be very beneficial, especially for long-term projects such as the LArTPC simulations. In search of a portable, scalable and maintainable software solution for LArTPC simulations, we have started to explore high-level portable programming frameworks that support several hardware backends. In this paper, we present our experience porting the LArTPC simulation code in the Wire-Cell Toolkit to NVIDIA GPUs, first with the CUDA programming model and then with a portable library called Kokkos. Preliminary performance results on NVIDIA V100 GPUs and multi-core CPUs are presented, followed by a discussion of the factors affiecting the performance and plans for future improvements.

Yu, Haiwang↗
DOE OSTI2025

Emerging applications: Neuromorphic computing and reservoir computing

The emergence of doped hafnium oxide (HfO 2 )-based ferroelectric films has enabled highly scalable and silicon-compatible ferroelectric devices, opening new frontiers in neuromorphic and reservoir computing. Among these, ferroelectric field-effect transistors (FeFETs) are particularly promising due to their analog memory characteristics and unique polarization dynamics. These properties make FeFETs ideal candidates for artificial synapses in neuromorphic architectures, supporting deep neural networks and spiking neural networks based on leaky-integrate-and-fire (LIF) mechanisms. Beyond neuromorphic computing, FeFETs also play a crucial role in physical reservoir computing, leveraging their intrinsic nonlinear and history-dependent behavior for efficient real-time learning. This approach offers significant advantages for time-series processing and edge artificial intelligence (AI) applications, addressing the growing need for energy-efficient computing. As a result, this article explores the principles, key demonstrations, and future potential of FeFET-based neuromorphic and reservoir computing, highlighting their impact on next-generation AI hardware.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Efficient distributed continual learning for steering experiments in real-time

Deep learning has emerged as a powerful method for extracting valuable information from large volumes of data. However, when new training data arrives continuously (i.e., is not fully available from the beginning), incremental training suffers from catastrophic forgetting (i.e., new patterns are reinforced at the expense of previously acquired knowledge). Training from scratch each time new training data becomes available would result in extremely long training times and massive data accumulation. Rehearsal-based continual learning has shown promise for addressing the catastrophic forgetting challenge, but research to date has not addressed performance and scalability. To fill this gap, we propose an approach based on a distributed rehearsal buffer that efficiently complements data-parallel training on multiple GPUs to achieve high accuracy, short runtime, and scalability. It leverages a set of buffers (local to each GPU) and uses several asynchronous techniques for updating these local buffers in an embarrassingly parallel fashion, all while handling the communication overheads necessary to augment input minibatches using unbiased, global sampling. We further propose a generalization of rehearsal buffers to support both classification and generative learning tasks, as well as more advanced rehearsal strategies (notably Dark Experience Replay, leveraging knowledge distillation). We illustrate this approach with a real-life HPC streaming application from the domain of ptychographic image reconstruction. Furthermore, we run extensive experiments on up to 128 GPUs of the ThetaGPU supercomputer to compare our approach with baselines representative of training-from-scratch (the upper bound in terms of accuracy) and incremental training (the lower bound). Results show that rehearsal-based continual learning achieves a top-5 validation accuracy close to the upper bound, while simultaneously exhibiting a runtime close to the lower bound.

Asynchronous data management↗
DOE OSTI2022

Autonomous continuous flow reactor synthesis for scalable atom-precision

With new instrumentation design, robotics, and in-operando hyphenated analytical tool automation, the intelligent discovery of synthesis pathways is becoming feasible. It can potentially bridge the gap for the scale-up of new materials. In this article, we review current progress and describe a new system that uses an autonomous continuous flow chemistry framework to translate high-quality lead molecules and materials to quantities that can meet scalability demands. At the core is a continuous flow synthesis platform that can design its viable synthesis pathway to a particular molecule or material and then autonomously carry it out. This is realized by integrating: (1) A workflow/architecture for multimode chemical/materials characterization in-line. The in-line characterization modes are NMR, ESR, IR, Raman, UV-Vis, GC-MS, and HPLC, along with ex-situ modes for X-Ray and neutron scattering; (2) Integration for feedback/analysis/data storage of the control variables; (3) A core software stack that includes deep learning and reinforcement learning alongside quantum chemistry and molecular dynamics; (4) On-demand compute architectures that parse calculations to compute resources needed which include light-weight edge, mid-level edge (NVIDA DGX-2), and high-performance computing. We demonstrate preliminary results on how this autonomous reactor system can enhance our ability to deliver deuterated materials, copolymers, and site-substituted molecules.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRSDate not supplied

Machine-Learned Committor Functions for Reactive Molecular Dynamics

Reactive molecular dynamics (MD) is a powerful tool for atomistic-scale modeling of a diverse range of chemical processes. However, scaling these simulations to large systems and long times scales remains a challenge because of the complexity of the potential energy function required. The authors previously developed a heuristic approach, called REACTER, that incorporates reactivity in MD simulations in a less general but much more computationally efficient manner. REACTER uses standard, fixed valence force fields as the underlying potentialenergy surface for describing all interatomic interactions but adds a procedure for enforcing user-defined reactions that occur when certain geometric constraints on relative atomic positions are satisfied. Further, these bonding changes can be accepted or rejected with a probability related tothe local thermal energy. This work seeks to generalize this approach by replacing the set of user defined geometric constraints and energetic criteria with a committor function that specifies the probability of a reaction occurring on the basis of the local atomic configuration. The committor function is a useful mathematical tool for modeling rare events but, unfortunately, is very difficult to compute for realistic systems in a general way. This work describes a method for approximating the committor function using a machine learning approach, specifically a deep neural network trained with data from reactive MD and DFT-based dynamics simulations. This network is coupled to the existing REACTER protocol, as implemented in the LAMMPS MD package, and used to make on-the-fly predictions of reaction probabilities without the more extensive user input previously required. The new method is demonstrated using the polymerization of polystyrene as a case study. Although very dependent on the quality and quantity of training data, machine-learned committor functions show promise as a method for incorporating reaction probability from higher level calculations into highly scalable MD simulations.

polymer simulations↗
DOE OSTI2025

Exascale granular microstructure reconstruction in 3D volumes of arbitrary geometries with generative learning

Reconstructing 3D granular microstructures within volumes of arbitrary geometries from limited 2D image data is crucial for predicting the material properties, as well as performances of structural components accounting for material microstructural effects. We present a novel generative learning framework that enables exascale reconstruction of granular microstructures within complex 3D geometric volumes. Building upon existing transfer learning techniques using pre-trained convolutional neural networks (CNN), we introduce several key innovations to overcome the difficulties inherent in arbitrary geometries. Our framework incorporates periodic boundary conditions using circular padding techniques, ensuring continuity and representativeness of the reconstructed microstructures. We also introduce a novel seamless transition reconstruction (STR) method that creates statistically equivalent transition zones to integrate multiple pre-existing 3D microstructure volumes. Based on STR, we propose a cost-effective strategy for reconstructing microstructures within complex geometric volumes, minimizing computational waste. Validation through numerical experiments using kinetic Monte Carlo simulations demonstrates accurate reproduction of grain statistics, including grain size distributions and morphology. A case study involving the reconstruction of a 4-blade propeller microstructure illustrates the method’s capability to efficiently handle complex geometries. In conclusion, the proposed framework significantly reduces computational demands while maintaining high reconstruction quality, paving the way for scalable microstructure reconstruction in materials design and analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Deep Analysis Net with Causal Embedding for Coal-fired Power Plant Fault Detection and Diagnosis (DANCE4CFDD)

Fault detection and diagnosis is critical to power plant operation to ensure attaining high reliability while reducing operation cost. As more renewable power is introduced to the power grid, traditional fossil power plants take on the extra burden of excessive load cycling to compensate the generation variability from renewable power. Such load cycling will pose more reliability challenges to power plant operation. There are a number of challenges faced by today’s asset health management system in coal- fired (or gas) power plants: 1) high-dimensional nonlinear interaction among multiple time series measurements; 2) high measurement variance induced by operational conditions/modes; 3) variation among asset types and plant configurations; and 4) a small number of faulty events to learn from. To cope with these challenges, today’s fielded asset health management systems rely heavily on manual efforts from domain experts and hand-crafted features or rules based on domain knowledge. Despite its role in plant reliability, such a practice is costly and hinders its scalability and sustainability, particularly when a plant undergoes modifications. The objective of this project is to develop a novel end-to-end AI learning system that is trainable (i.e., the AI representation of a complex system behavior can be directly learned from properly labeled data) for accurate fault detection and root cause analysis. The ability to create a fault detection model directly from time series could alleviate the efforts associated with today’s asset management solution development. In the course of this project, we have achieved the following: Created an AI model development environment incorporating state-of-the-art neural network architectures for rapid model development and evaluation; Developed novel learning strategies for training of fault detection model; Developed special-purpose neural network architecture embedded with variable association graph aiming for better interpretability; Developed a learning strategy to leverage a small number of faulty events for enhanced fault detection capability; Conducted detailed experimental study based on public benchmark datasets and demonstrated the effectiveness of the proposed solution; and Validated the developed system with data from both a coal-fired plant boiler dynamic simulation model and real-world coal-fired power plant covering multiple asset and fault types. Overall, the project attained a technology readiness level of TRL 5 from TRL 2 at the beginning of the project.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2026

HydroDCM: Hydrological Domain-Conditioned Modulation for Cross-Reservoir Inflow Prediction

Deep learning models have shown promise in reservoir inflow prediction, yet their performance often deteriorates when applied to different reservoirs due to distributional differences, referred to as the domain shift problem. Domain generalization (DG) solutions aim to address this issue by extracting domain-invariant representations that mitigate errors in unseen domains. However, in hydrological settings, each reservoir exhibits unique inflow patterns, while some metadata beyond observations like spatial information exerts indirect but significant influence. This mismatch limits the applicability of conventional DG techniques to many-domain hydrological systems. To overcome these challenges, we propose HydroDCM, a scalable DG framework for cross-reservoir inflow forecasting. Spatial metadata of reservoirs is used to construct pseudo-domain labels that guide adversarial learning of invariant temporal features. During inference, HydroDCM adapts these features through light-weight conditioning layers informed by the target reservoir’s metadata, reconciling DG’s invariance with location-specific adaptation. Experiment results on 30 real-world reservoirs in the Upper Colorado River Basin demonstrate that our method substantially outperforms state-of-the-art DG baselines under many-domain conditions and remains computationally efficient.

Hu, Pengfei [ORNL] (ORCID:0009000367130950)↗
DOE OSTI2023

The LSST AGN Data Challenge: Selection Methods

Abstract Development of the Rubin Observatory Legacy Survey of Space and Time (LSST) includes a series of Data Challenges (DCs) arranged by various LSST Scientific Collaborations that are taking place during the project's preoperational phase. The AGN Science Collaboration Data Challenge (AGNSC-DC) is a partial prototype of the expected LSST data on active galactic nuclei (AGNs), aimed at validating machine learning approaches for AGN selection and characterization in large surveys like LSST. The AGNSC-DC took place in 2021, focusing on accuracy, robustness, and scalability. The training and the blinded data sets were constructed to mimic the future LSST release catalogs using the data from the Sloan Digital Sky Survey Stripe 82 region and the XMM-Newton Large Scale Structure Survey region. Data features were divided into astrometry, photometry, color, morphology, redshift, and class label with the addition of variability features and images. We present the results of four submitted solutions to DCs using both classical and machine learning methods. We systematically test the performance of supervised models (support vector machine, random forest, extreme gradient boosting, artificial neural network, convolutional neural network) and unsupervised ones (deep embedding clustering) when applied to the problem of classifying/clustering sources as stars, galaxies, or AGNs. We obtained classification accuracy of 97.5% for supervised models and clustering accuracy of 96.0% for unsupervised ones and 95.0% with a classic approach for a blinded data set. We find that variability features significantly improve the accuracy of the trained models, and correlation analysis among different bands enables a fast and inexpensive first-order selection of quasar candidates.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Automatically parallelizing batch inference on deep neural networks using Fiats and Fortran 2023 `do concurrent`

This paper introduces novel programming strategies that leverage features of the Fortran 2023 standard of the International Standards Organization (ISO) to automatically parallelize computations on deep neural networks. The paper focuses on the interplay of object-oriented, parallel, and functional programming paradigms in the Fiats deep learning library. We demonstrate how several infrequently used language features play a role in enabling efficient, parallel execution. Specifically, the ability to explicitly declare that a procedure is pure facilitates inference in the context of the language’s loop-parallelism construct `do concurrent`. Also, explicitly prohibiting the overriding of a parent type’s type-bound procedures eliminates the need for dynamic dispatch in performance-critical code. Finally, this paper uses batch inference calculations on a neural network surrogate for atmospheric aerosol dynamics to demonstrate that LLVM Flang compiler’s automatic parallelization of `do concurrent` achieves roughly the same performance and scalability as achieved by OpenMP compiler directives. We also demonstrate that double-precision inference costs 37–72% longer runtime than default-real precision with most values in the range 57-60%.

Rouson, Damian↗
DOE OSTI2022

2022 AI Testbed Expeditions Report

By exploiting the coherent properties of a light source, coherent diffraction imaging (CDI) is able to obtain the sample image at a nanoscale resolution using the measured diffraction pattern. Bragg Coherent Diffraction Imaging (BCDI) has become valuable for recovering the displacement and strain field of crystals, providing a valuable tool in material science and solid-state physics. X-ray ptychography is another emerging CDI technique that can produce a high-resolution image of the extended sample and has become popular in many research areas (e.g., materials science, biology, electronics, and optics characterization). CDI including BCDI and ptychography has become an established technique in Synchrotron Facilities including the Advanced Photon Source (APS) and will greatly benefit from the 100x coherent flux increase of the upcoming APS Upgrade (APSU). The current image formation process in CDI employs iterative phase retrieval algorithms, which is a time-consuming and computationally expensive process. Especially after APSU, the traditional iterative methods will not be able to match the experimental data acquisition speed. We employ deep learning (DL) approach to replace the iterative approaches, therefore allowing hundreds of times faster recovery of the object. We developed AutoPhaseNN, a DL-based approach which learns to solve the inverse problem without labeled data. Taking 3D BCDI as a representative technique, AutoPhaseNN has been demonstrated to be one hundred times faster than traditional iterative phase retrieval methods while providing comparable image quality. The current network is trained with 64 x 64 x 64 data size, to achieve higher resolution imaging, we will need to scale the network to input and train/infer 3D arrays of size 256 x 256 x 256 (today) and of size 2560x2560x2560 (APSU). However, the scalability of the network is restricted due to the memory-intensive training process. To perform the training for a 256 x 256 x 256 data size, the required memory exceeds the capacity of the current machine. In this project, we explore using Sambanova system to train the network for the direct data inversion for CDI.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Signal Whisperers: Enhancing Wireless Reception Using DRL-Guided Reflector Arrays

This paper presents a multi-agent reinforcement learning (MARL) approach for controlling adjustable metallic reflector arrays to enhance wireless signal reception in non-line-of-sight (NLOS) scenarios. Unlike conventional reconfigurable intelligent surfaces (RIS) that require complex channel estimation, our system employs a centralized training with decentralized execution (CTDE) paradigm where individual agents corresponding to reflector segments autonomously optimize reflector element orientation in three-dimensional space using spatial intelligence based on user location information. Through extensive ray-tracing simulations with dynamic user mobility, the proposed multi-agent beam-focusing framework demonstrates substantial performance improvements over single-agent reinforcement learning baselines, while maintaining rapid adaptation to user movement within one simulation step. Comprehensive evaluation across varying user densities and reflector configurations validates system scalability and robustness. The results demonstrate the potential of learning-based approaches for adaptive wireless propagation control.

deep reinforcement learning↗
DOE OSTI2023

Fission with Exotic Nuclei (Full Technical Report)

Despite its importance for stockpile stewardship or nuclear forensics, data on fission is fragmentary: most experiments have only been performed on stable actinide nuclei and are often incomplete, and theoretical simulations often contain many parameters hard to constrain, which results in large uncertainties. Consequently, nuclear libraries have large gaps for important isotopes. The goal of this project was to prepare to take advantage of the unprecedented yields of radioactive isotopes at the upcoming DOE Facility for Rare Ion Beams and to leverage recent progress in the field of machine learning to develop a comprehensive program of fission studies that could address some of the most pressing problems in nuclear fission over the next several decades. Our project leveraged synergies between experimental nuclear physics, nuclear theory, and data science expertise at LLNL. On the experimental front, our main objective was to develop in-house expertise for inverse kinematics reactions with relativistic beams as well as to field and test new detectors to perform correlated measurements of fission properties. On the theoretical side, the objective was to develop a novel, high-fidelity and scalable approach of fissionfragment calculations based on emulating computationally expensive, quantum-mechanical calculations of nuclear properties with deep neural networks.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Improving scalability of parallel CNN training by adaptively adjusting parameter update frequency

Synchronous SGD with data parallelism, the most popular parallelization strategy for CNN training, suffers from the expensive communication cost of averaging gradients among all workers. The iterative parameter updates of SGD cause frequent communications and it becomes the performance bottleneck. In this paper, we propose a lazy parameter update algorithm that adaptively adjusts the parameter update frequency to address the expensive communication cost issue. Our algorithm accumulates the gradients if the difference of the accumulated gradients and the latest gradients is sufficiently small. Here, the less frequent parameter updates reduce the per-iteration communication cost while maintaining the model accuracy. Our experimental results demonstrate that the lazy update method remarkably improves the scalability while maintaining the model accuracy. For ResNet50 training on ImageNet, the proposed algorithm achieves a significantly higher speedup (739.6 on 2048 Cori KNL nodes) as compared to the vanilla synchronous SGD (276.6) while the model accuracy is almost not affected (<0.2% difference).

97 MATHEMATICS AND COMPUTING↗