Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Data-Driven Multi-agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control with High Penetration of PVs

This paper proposes a novel model-free/data-driven centralized training and decentralized execution multi-agent deep reinforcement learning (MADRL) framework for distribution system voltage control with high penetration of PVs. The proposed MADRL can coordinate both the real and reactive power control of PVs with existing static var compensators and battery storage systems. Unlike the existing DRL-based voltage control methods, our proposed method does not rely on a system model during both the training and execution stages. This is achieved by developing a new interaction scheme between the surrogate modeling of the original system and the multi-agent soft actor critic (MASAC) MADRL algorithm. In particular, the sparse pseudo-Gaussian process with a few-shots of measurements is utilized to construct the surrogate model of the original environment, i.e., power flow model. This is a data-driven process and no model parameters are needed. Furthermore, the MASAC enabled MADRL allows to achieve better scalability by dividing the original system into different voltage control regions with the aid of real and reactive power sensitivities to voltage, where each region is treated as an agent. This also serves as the foundation for the centralized training and decentralized execution, thus significantly reducing the communication requirements as only local measurements are required for control. Comparative results with other alternatives on the IEEE 123-nodes and 342-nodes systems demonstrate the superiority of the proposed method.

14 SOLAR ENERGY↗
DOE OSTI2021

Ensemble learning-iterative training machine learning for uncertainty quantification and automated experiment in atom-resolved microscopy

Deep learning has emerged as a technique of choice for rapid feature extraction across imaging disciplines, allowing rapid conversion of the data streams to spatial or spatiotemporal arrays of features of interest. However, applications of deep learning in experimental domains are often limited by the out-of-distribution drift between the experiments, where the network trained for one set of imaging conditions becomes sub-optimal for different ones. This limitation is particularly stringent in the quest to have an automated experiment setting, where retraining or transfer learning becomes impractical due to the need for human intervention and associated latencies. Here we explore the reproducibility of deep learning for feature extraction in atom-resolved electron microscopy and introduce workflows based on ensemble learning and iterative training to greatly improve feature detection. This approach allows incorporating uncertainty quantification into the deep learning analysis and also enables rapid automated experimental workflows where retraining of the network to compensate for out-of-distribution drift due to subtle change in imaging conditions is substituted for human operator or programmatic selection of networks from the ensemble. This methodology can be further applied to machine learning workflows in other imaging areas including optical and chemical imaging.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Preliminary Results for Using Uncertainty and Out-of-distribution Detection to Identify Unreliable Predictions.

As machine learning (ML) models are deployed into an ever-diversifying set of application spaces, ranging from self-driving cars to cybersecurity to climate modeling, the need to carefully evaluate model credibility becomes increasingly important. Uncertainty quantification (UQ) provides important information about the ability of a learned model to make sound predictions, often with respect to individual test cases. However, most UQ methods for ML are themselves data-driven and therefore susceptible to the same knowledge gaps as the models themselves. Specifically, UQ helps to identify points near decision boundaries where the models fit the data poorly, yet predictions can score as certain for points that are under-represented by the training data and thus out-of-distribution (OOD). One method for evaluating the quality of both ML models and their associated uncertainty estimates is out-of-distribution detection (OODD). We combine OODD with UQ to provide insights into the reliability of the individual predictions made by an ML model.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Solar Energy Technologies Office Workforce Request for Information and Convenings (Summary)

On May 4, 2021, the U.S. Department of Energy (DOE) Solar Energy Technologies Office (SETO) published a Request for Information (RFI) on programs that support the development of a diverse and skilled clean energy workforce. The purpose of the RFI was to solicit feedback from industry, academia, government agencies, worker organizations (including unions), and other stakeholders on issues related to the employment needs of the solar industry, and the perceived value of different workforce development programs, training strategies, and tools. To supplement the RFI, SETO hosted four virtual convenings that brought together the utility-scale solar industry, the distributed generation solar industry, and labor and other workforce training organizations to hear direct feedback on the questions in the RFI. In addition, SETO held listening sessions with about a dozen other organizations and staff who could not participate in the virtual convenings. Altogether, SETO received 45 responses from the RFI and heard directly from 80-100 other stakeholders via the convenings and listening sessions. This document summarizes the stakeholder feedback that SETO received as a result of this process. While both the RFI and convening series were focused on solar deployment and solar industry members, much of this information is relevant across clean energy technologies and programs. It is important to recognize that DOE is intentionally reviewing our workforce development programming and support to focus on clean energy careers more holistically.

14 SOLAR ENERGY↗
DOE OSTI2022

Toward designing effective exascale scientific computing workflows: experiences and best practices

Many fields within scientific computing have embraced advances in big-data analysis and machine learning, which often requires the deployment of large, distributed and complicated workflows that may combine training neural networks, performing simulations, running inference, and performing database queries and data analysis in asynchronous, parallel and pipelined execution frameworks. Such a shift has brought into focus the need for scalable, efficient workflow management solutions with reproducibility, error and provenance handling, traceability, and checkpoint-restart capabilities, among other needs. Here, we discuss challenges and best-practices for deploying exascale-generation computational science workflows on resources at the Oak Ridge Leadership Computing Facility (OLCF). We present our experiences with large-scale deployment of distributed workflows on the Summit supercomputer, including for bioinformatics and computational biophysics, materials science, and deep learning model optimization. We also present problems and solutions created by working within a Python-centric software base on traditional HPC systems, and discuss steps that will be required before the convergence of HPC, AI, and data science can be fully realized. Our results point to a wealth of exciting new possibilities for harnessing this convergence to tackle new scientific challenges.

Coletti, Mark↗
DOE OSTI2024

Investigating the influence of particle distribution on force and torque statistics using hierarchical machine learning

An accurate representation of hydrodynamic force and torque experienced by every particle in a distribution can be obtained from particle resolved (PR) simulations. These unique quantities are influenced by the deterministic position of surrounding particles. However, systems simulated with this methodology are typically limited to particles due to the involved computational cost. This resource requirement is a major bottleneck in analyzing the effect of variations in particle distribution. Here, this article attempts to address this bottleneck by availing relatively inexpensive deep learning models. The surrogate models that we employ in this article use a physics‐based hierarchical framework and symmetry‐preserving neural networks to achieve robustness with limited training data. This article first performs additional generalizability tests on PR data of distinct distributions that are not involved in the training process. The models are then deployed on several different particle distributions. Impact of clustering and structure on the observed statistics are investigated.

42 ENGINEERING↗
DOE OSTI2026

Improved energies and local energies with weighted variational Monte Carlo

Neural network parametrizations have increasingly been used to represent the ground and excited states in variational Monte Carlo (VMC) with promising results. However, traditional VMC methods only optimize the wave function in regions of peak probability. The wave function is uncontrolled in the tails of the probability distribution, which can limit the accuracy of the trained wave function. To improve the approximation accuracy in the probability tails, this paper interprets VMC as a gradient flow in the space of wave functions, followed by a projection step. From this perspective, arbitrary probability distributions can be used in the projection step, allowing the user to prioritize accuracy in different regions of state space. Motivated by this theoretical perspective, the paper tests a weighted VMC method on the antiferromagnetic Heisenberg model for a periodic spin chain. Compared to traditional VMC, weighted VMC reduces the error in the ground state energy by a factor of 2, and it reduces the errors in the local energies away from the mode by large factors of 10 2 –10 4 .

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Neoclassical toroidal viscosity torque prediction via deep learning

GPECnet is a densely connected neural network that has been trained on GPEC data, to predict the plasma stability, neoclassical toroidal viscosity (NTV) torque, and optimized 3D coil current distributions for desired NTV torque profiles. Using NTV torque, driven by non-axisymmetric field perturbations in a tokamak, can be vital in optimizing pedestal performance by controlling the rotation profile in both the core, to ensure tearing stability, and the edge, to avoid edge localized modes (ELMs). The generalized perturbed equilibrium code (GPEC) software package can be used to calculate the plasma stability to 3D perturbations and the NTV torque profile generated by applied 3D magnetic fields. These calculations, however, involve complex integrations over space and energy distributions, which takes time to compute. Initially, GPECnet has been trained solely on data representative of the quiescent H-mode (QH) scenario, in which neutral beams are often balanced and toroidal rotation is low across the plasma profile. Lastly, this work provides the foundation for active control of the rotation shear using a combination of beams and 3D fields for robust and high performance QH mode operation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

A Convolution Neural Network for Voltage Event Classification at a Photovoltaic Inverter

This paper presents a convolutional neural network (CNN) developed to identify voltage events in photovoltaic (PV) inverters. The CNN is trained on synthetic data generated using the IEEE 13-bus distribution feeder model and evaluated on field measured data collected from Energy Northwest’s Horn Rapids Solar, Storage, and Training (HRSST) facility. The study focuses on two common voltage events: faults and voltage sags. The CNN is configured to analyze voltage and current waveforms from three-phase PV systems, demonstrating excellent accuracy during training. Field data from the HRSST facility is employed to assess its real-world performance, where the CNN achieves perfect identification of faults and voltage sags in a sample of nine events. This work highlights the potential of the proposed method to enhance PV protection schemes, providing a robust foundation for improved voltage event detection and grid reliability.

Cornachione, Matthew A.↗
DOE OSTI2022

EQC: Ensembled Quantum Computing for Variational Quantum Algorithms

Variational quantum algorithms (VQA), which are comprised of a classical optimizer and a parameterized quantum circuit, emerges as one of the most promising approaches of harvesting quantum power in the noisy-intermediate-scale-quantum (NISQ) era. However, the deployment of VQAs on today's NISQ devices often faces considerable system noise and prohibitively slow training speeds. On the other hand, the expensive supporting sources and infrastructure make quantum computers extremely keen on high utilization. In this paper, we propose a novel way of thinking about a quantum backend: rather than relying on one physical device which tends to introduce platform-specific noise and bias, a quantum ensemble, which distributes quantum tasks across parallel devices, can serve as a virtualized quantum computer for offering reduced noise levels through an adaptive mixture and also provide significantly improved training speeds through parallelization. With this idea, we build a distributive VQA optimization framework called DVQA, serving as the first effort in adopting parallel quantum devices for cooperative VQA training. To further constraint noise and speed-up convergence, we design a model for individual NISQ devices concerning their properties and running conditions, and propose a weighting mechanism for regularizing the returned gradients. Extensive evaluations on 10 IBM-Q quantum devices using the VQE example show that the distributive VQA training framework can substantially boost the training speed by 10.5x on average (up to 86x and at least 5.2x) with improved training accuracy.

Stein, Samuel A.↗
DOE OSTI2026

Colorado Technology Primer for Economists and Social Scientists: Cooperative Research and Development Final Report, CRADA Number CRD-19-00790

NLR will assist the Colorado School of Mines (Mines) in supporting a series of one-week training workshops. This proposed training program will be two, week-long summer school sessions in each of the next two years to help give early career economists and social scientists a solid introduction and grounding on the technical components of the electrical distribution, transmission, and generation systems as well as basic to advanced overview of clean energy and traditional generation systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Cations Control Lipid Bilayer Memcapacitance Associated with Long-Term Potentiation

Phospholipid bilayers can be described as capacitors whose capacitance per unit area (specific capacitance, C m ) is determined by their thickness and dielectric constant–independent of applied voltage. It is also widely assumed that the C m of membranes can be treated as a “biological constant”. Recently, using droplet interface bilayers (DIBs), it was shown that zwitterionic phosphatidylcholine (PC) lipid bilayers can act as voltage-dependent, nonlinear memory capacitors, or memcapacitors. When exposed to an electrical “training” stimulation protocol, capacitive energy storage in lipid membranes was enhanced in the form of long-term potentiation (LTP), which enables biological learning and long-term memory. LTP was the result of membrane restructuring and the progressive asymmetric distribution of ions across the lipid bilayer during training, which is analogous, for example, to exponential capacitive energy harvesting from self-powered nanogenerators. Here, we describe how LTP could be produced from a membrane that is continuously pumped into a nonequilibrium steady state, altering its dielectric properties. During this time, the membrane undergoes static and dynamic changes that are fed back to the system’s potential energy, ultimately resulting in a membrane whose modified molecular structure supports long-term memory storage and LTP. Here, we also show that LTP is very sensitive to different salts (KCl, NaCl, LiCl, and TmCl 3 ), with LiCl and TmCl 3 having the most profound effect in depressing LTP, relative to KCl. This effect is related to how the different cations interact with the bilayer zwitterionic PC lipid headgroups primarily through electric-field-induced changes to the statistically averaged orientations of water dipoles at the bilayer headgroup interface.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Li-ion Battery Material phase prediction through Hierarchical Curriculum Learning

Li-ion Batteries (LIB), one of the most efficient energy storage devices, are widely adopted in many industrial applications. Imaging data of these battery electrodes obtained from X-ray tomography can explain the distribution of material constituents and allow reconstructions to study electron transport pathways. Therefore, it can eventually help quantify various associated properties of electrodes (e.g., volume-specific surface area, porosity) which determine the performance of batteries. However, these images often suffer from low image contrast between multiple material constituents , making it difficult for humans to distinguish and characterize these constituents through visualization. A minor error in detecting distributions among the material constituents can lead to a high error in the calculated parameters of material properties.We present a novel hierarchical curriculum learning framework to address the complex task of estimating material constituent distribution in battery electrodes. To provide spatially smooth prediction, our framework comprises three modules: (i) an uncertainty-aware model trained to yield inferences conditioned upon global knowledge of material distribution, (ii) a technique to capture relatively more fine-grained (local) distributional signals, (iii) an aggregator to appropriately fuse the local and global effects towards obtaining the final distribution.

Tabassum, Anika↗
DOE OSTI2024

Normalizing flows for domain adaptation when identifying Λ hyperon events

Here this study focuses on the application of a normalizing flow as a method of domain adaptation when classifying physics data. Normalizing flows offer a way to transform data points between two different distributions. The present study investigates a novel method of transforming latent representations of physics data to a normal distribution and then to a physics distribution again. The final distribution models a simulated distribution. After being transformed, the data can be classified by a neural network trained on labeled simulation data. The present study succeeds in training two normalizing flows that can transform between data (or simulation) and a Gaussian distribution.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

EV Charging Infrastructure Energization An Overview of Approaches for Simplifying and Accelerating Timelines to Processing EV Charging Load Service Requests

The United States has seen significant growth in electric vehicle (EV) adoption, leading to increased demand for EV charging infrastructure. Over the past decade, EV charging infrastructure site developers, site hosts, and electric distribution utilities have navigated the process to integrate chargers onto the electric grid. Site developers and site hosts have raised the alarm that the integration process for high-powered EV charging projects does not meet the needs of the EV market for timeliness or cost. High-powered charging stations typically require a load service request or an agreement with the local utility to connect to the grid. The process of energizing a new high-powered charging site can be complex and time-consuming, often taking up to 2 years. This timeline is the result of current utility energization processes having been designed for construction projects that take longer to build (i.e., buildings). The specific challenges stem from various factors, including compartmentalization in application processes, the integration of EV charging process approvals with other distributed energy resources (DERs), and the need to ensure grid reliability. The energization process needs to evolve to meet the growing demand for high-powered EV charging. This white paper compiles information gathered through various conversations with key stakeholders, including utilities, utility regulators, EV charging operators, site developers, and authorities having jurisdiction (AHJ) as well as through an extensive literature review. This document identifies the challenges and provides potential solutions to streamline the process of connecting EV charging infrastructure to the power grid in the United States, serving as a starting point for future conversations around these solutions. The solutions noted in this white paper require collaborative efforts among utilities, regulators, and EV charging infrastructure developers to streamline the grid connection process for EV charging infrastructure. They are broadly organized into four areas: 1. Increase data access and transparency: Develop automated load service request tools, integrate hosting capacity and load service request analyses, incorporate EV adoption forecasts, and provide transparency on the processing queue. 2. Improve energization processes and timing: Create fast-track options based on prescreening criteria, provide flexibility or phased approvals in the load service request/interconnection process, build internal knowledge within utilities about EV charging technologies, and provide standardized workforce training. 3. Promote economic efficiency: Right size distribution components to accurately reflect the load requirements of EV charging infrastructure, make proactive investments in grid infrastructure based on EV adoption forecasts and growth projections, and consider energy equity and environmental justice factors such as equitable access to EV charging when planning infrastructure. 4. Improve grid reliability and resilience: Use load management/power control systems (PCS) at EV charging stations, adopt and implement harmonized standards for communication protocols and information models between the EV charging and grid control infrastructure, and address cybersecurity considerations by implementing robust security measures and standards for EV charging infrastructure—with particular emphasis on clarifying the security requirements for the interface to the grid. The objective of the solutions proposed in this white paper is to accelerate the timeline and decrease costs associated with connecting EV charging infrastructure to the grid. Electric utilities, utility regulators, EV charging infrastructure developers, and site hosts will first need to understand which solutions are available in their service territory, and if warranted, which combination of solutions would support their specific needs. Through the successful implementations of solutions at scale detailed here, industry will demonstrate a new and innovative ecosystem where timely deployment and energization of EV charging infrastructure with greater grid resiliency and reliability is a reality.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

On the Generalizability of Time-of-Flight Convolutional Neural Networks for Noninvasive Acoustic Measurements

Bulk wave acoustic time-of-flight (ToF) measurements in pipes and closed containers can be hindered by guided waves with similar arrival times propagating in the container wall, especially when a low excitation frequency is used to mitigate sound attenuation from the material. Convolutional neural networks (CNNs) have emerged as a new paradigm for obtaining accurate ToF in non-destructive evaluation (NDE) and have been demonstrated for such complicated conditions. However, the generalizability of ToF-CNNs has not been investigated. In this work, we analyze the generalizability of the ToF-CNN for broader applications, given limited training data. We first investigate the CNN performance with respect to training dataset size and different training data and test data parameters (container dimensions and material properties). Furthermore, we perform a series of tests to understand the distribution of data parameters that need to be incorporated in training for enhanced model generalizability. This is investigated by training the model on a set of small- and large-container datasets regardless of the test data. We observe that the quantity of data partitioned for training must be of a good representation of the entire sets and sufficient to span through the input space. The result of the network also shows that the learning model with the training data on small containers delivers a sufficiently stable result on different feature interactions compared to the learning model with the training data on large containers. To check the robustness of the model, we tested the trained model to predict the ToF of different sound speed mediums, which shows excellent accuracy. Furthermore, to mimic real experimental scenarios, data are augmented by adding noise. We envision that the proposed approach will extend the applications of CNNs for ToF prediction in a broader range.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Operator learning for predicting multiscale bubble growth dynamics

We report simulating and predicting multiscale problems that couple multiple physics and dynamics across many orders of spatiotemporal scales is a great challenge that has not been investigated systematically by deep neural networks (DNNs). Herein, we develop a framework based on operator regression, the so-called deep operator network (DeepONet), with the long-term objective to simplify multiscale modeling by avoiding the fragile and time-consuming “hand-shaking” interface algorithms for stitching together heterogeneous descriptions of multiscale phenomena. To this end, as a first step, we investigate if a DeepONet can learn the dynamics of different scale regimes, one at the deterministic macroscale and the other at the stochastic microscale regime with inherent thermal fluctuations. Specifically, we test the effectiveness and accuracy of the DeepONet in predicting multirate bubble growth dynamics, which is described by a Rayleigh–Plesset (R–P) equation at the macroscale and modeled as a stochastic nucleation and cavitation process at the microscale by dissipative particle dynamics (DPD). First, we generate data using the R–P equation for multirate bubble growth dynamics caused by randomly time-varying liquid pressures drawn from Gaussian random fields (GRFs). Our results show that properly trained DeepONets can accurately predict the macroscale bubble growth dynamics and can outperform long short-term memory networks. We also demonstrate that the DeepONet can extrapolate accurately outside the input distribution using only very few new measurements. Subsequently, we train the DeepONet with DPD data corresponding to stochastic bubble growth dynamics. Although the DPD data are noisy and we only collect sparse data points on the trajectories, the trained DeepONet model is able to predict accurately the mean bubble dynamics for time-varying GRF pressures. Taken together, our findings demonstrate that DeepONets can be employed to unify the macroscale and microscale models of the multirate bubble growth problem, hence providing new insight into the role of operator regression via DNNs in tackling realistic multiscale problems and in simplifying modeling with heterogeneous descriptions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

ThunderSecure: deploying real-time intrusion detection for 100G research networks by leveraging stream-based features and one-class classification network

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks (40/100G) across distributed sites for processing, storage, and analysis. Like general purpose networks, research networks experience intrusions. However, monitoring anomalies in such high-speed network traffics is challenging given current cyber-infrastructure. Moreover, traditional network intrusion detection systems (NIDS) are signature based. However, anomaly patterns are difficult to define and that rulesets are often not updated frequently enough to reflect the changes of attack behaviors. We present ThunderSecure, a high-throughput, unsupervised learning-based intrusions detection system for 100G research networks. ThunderSecure implements an efficient packet processing and detection pipeline using multi-cores and GPUs. It extracts statistical and temporal features from real-time network data streams and feeds them to a one-class anomaly detection network. A baseline of normal distribution will be created based on the training observation. Testing traffic deviated from the learned profile will be marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets mirrored from two 100G network connections at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. Results show that ThunderSecure can recognize science data traffic captured long after the training and made nearly certain detection on the segment of the streams where anomalous flows were injected.

100G research network↗