Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Better together: Elements of successful scientific software development in a distributed collaborative community

Many scientific disciplines rely on computational methods for data analysis, model generation, and prediction. Implementing these methods is often accomplished by researchers with domain expertise but without formal training in software engineering or computer science. This arrangement has led to underappreciation of sustainability and maintainability of scientific software tools developed in academic environments. Some software tools have avoided this fate, including the scientific library Rosetta. We use this software and its community as a case study to show how modern software development can be accomplished successfully, irrespective of subject area. Rosetta is one of the largest software suites for macromolecular modeling, with 3.1 million lines of code and many state-of-the-art applications. Since the mid 1990s, the software has been developed collaboratively by the RosettaCommons, a community of academics from over 60 institutions worldwide with diverse backgrounds including chemistry, biology, physiology, physics, engineering, mathematics, and computer science. Developing this software suite has provided us with more than two decades of experience in how to effectively develop advanced scientific software in a global community with hundreds of contributors. Here we illustrate the functioning of this development community by addressing technical aspects (like version control, testing, and maintenance), community-building strategies, diversity efforts, software dissemination, and user support. We demonstrate how modern computational research can thrive in a distributed collaborative community. The practices described here are independent of subject area and can be readily adopted by other software development communities

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Flexible Machine Learning-Based Cyberattack Detection Using Spatiotemporal Patterns for Distribution Systems

This letter develops a flexible machine learning detection method for cyberattacks in distribution systems considering spatiotemporal patterns. Spatiotemporal patterns are recognized by the graph Laplacian based on system-wide measurements. A flexible Bayes classifier (BC) is used to train spatiotemporal patterns which could be violated when cyberattacks occur. Cyberattacks are detected by using flexible BCs online. The effectiveness of the developed method is demonstrated through standard IEEE 13- and 123-node test feeders.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Self-healing capability is a critical factor for a resilient distribution system, which requires intelligent agents to automatically perform service restoration online, including network reconfiguration and reactive power dispatch. Here, the article proposes the imitation learning framework for training such an agent, where the agent will interact with an expert built based on the mixed-integer program to learn its optimal policy, and therefore significantly improve the training efficiency compared with exploration-dominant reinforcement learning (RL) methods. This significantly improved training efficiency makes the training problem under N-k scenarios tractable. A hybrid policy network is proposed to handle tie-line operations and reactive power dispatch simultaneously to further improve the restoration performance. The 33-bus and 119-bus systems with N-k disturbances are employed to conduct the training. The results indicate that the proposed method outperforms traditional RL algorithms such as the deep-Q network.

42 ENGINEERING↗
DOE OSTI2022

PI3NN: Out-of-distribution-aware Prediction Intervals from Three Neural Networks

We propose a novel prediction interval (PI) method for uncertainty quantification, which addresses three major issues with the state-of-the-art PI methods. First, existing PI methods require retraining of neural networks (NNs) for every given confidence level and suffer from the crossing issue in calculating multiple PIs. Second, they usually rely on customized loss functions with extra sensitive hyperparameters for which fine tuning is required to achieve a well-calibrated PI. Third, they usually underestimate uncertainties of out-of-distribution (OOD) samples leading to over-confident PIs. Our PI3NN method calculates PIs from linear combinations of three NNs, each of which is independently trained using the standard mean squared error loss. The coefficients of the linear combinations are computed using root-finding algorithms to ensure tight PIs for a given confidence level. We theoretically prove that PI3NN can calculate PIs for a series of confidence levels without retraining NNs and it completely avoids the crossing issue. Additionally, PI3NN does not introduce any unusual hyperparameters resulting in a stable performance. Furthermore, we address OOD identification challenge by introducing an initialization scheme which provides reasonably larger PIs of the OOD samples than those of the in-distribution samples. Benchmark and real-world experiments show that our method outperforms several state-of-the-art approaches with respect to predictive uncertainty quality, robustness, and OOD samples identification.

Liu, Siyan↗
DOE OSTI2022

Dynamic Role-Based Access Control Policy for Smart Grid Applications: An Offline Deep Reinforcement Learning Approach

Role-based access control (RBAC) is adopted in the information and communication technology domain for authentication purposes. However, due to a very large number of entities within organizational access control (AC) systems, static RBAC management can be inefficient, costly, and can lead to cybersecurity threats. In this paper, a novel hybrid RBAC model is proposed, based on the principles of offline deep reinforcement learning (RL) and Bayesian belief networks. The considered framework utilizes a fully offline RL agent, which models the behavioral history of users as a Bayesian belief-based trust indicator. Thus, the initial static RBAC policy is improved in a dynamic manner through off-policy learning while guaranteeing compliance of the internal users with the security rules of the system. By deploying our implementation within the smart grid domain and specifically within a Distributed Energy Resources (DER) ecosystem, we provide an end-to-end proof of concept of our model. Finally, detailed analysis and evaluation regarding the offline training phase of the RL agent are provided, while the online deployment of the hybrid RL-based RBAC model into the DER ecosystem highlights its key operation features and salient benefits over traditional RBAC models.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Cyber-Physical Event Emulation-Based Transmission-and-Distribution Co-Simulation for Situational Awareness of Grid Anomalies (SAGA)

Energy management of transmission and distribution networks (T&D) is becoming more challenging with the accelerated adoption of distributed energy resources (DERs)-such as distributed photovoltaic generation and battery energy storage systems (BESS)-on the electric grid. To better analyze the impacts of DERs on both transmission and distribution systems, a comprehensive T&D co-simulation platform is developed. Further, with DERs more actively participating in system operation-e.g., by providing real-time grid services-their cyber vulnerability needs to be better understood to maintain system reliability. This paper discusses a cyber-physical events emulation-based T&D co-simulation platform to perform comprehensive cyber events emulations, physical simulation, and analysis of interdependent impacts. Results from the case studies-which show how cyber events on a synthetic distribution network can impact operations on the transmission and distribution network-validate that the proposed T&D cosimulation platform can perform cyber-physical events emulation and produce response in near realtime; therefore, with extensive simulation using the proposed co-simulation platform, the system operators can accumulate adequate training data for system situational awareness of grid anomalies.

anomalies detection↗
DOE OSTI2020

Predicting the proximity to macroscopic failure using local strain populations from dynamic in situ X-ray tomography triaxial compression experiments on rocks

Predicting the proximity of large-scale dynamic failure is a critical concern in the engineering and geophysical sciences. Here we use evolving contractive, dilatational, and shear strain deformation preceding failure in dynamic X-ray tomography experiments to examine which strain components best predict the proximity to failure. We develop machine learning models to predict the proximity to failure using time series of three-dimensional local incremental strain tensor fields acquired in rock deformation experiments under stress conditions of the upper crust. Three-dimensional scans acquired in situ throughout triaxial compression experiments provide a distribution of density contrasts from which we estimate the three-dimensional incremental strain that accumulates between each scan acquisition. Training machine learning models on multiple experiments of six rock types provides suites of feature importance that indicate the predictive power of each feature. Comparing the average importance of groups of features that include information about each strain component quantifies the ability of the contractive, dilatational and shear strain to predict the proximity of macroscopic failure. A total of 24 models of four machine learning algorithms with six rock types indicate that 1) the dilatational strain provides the best predictive power of the strain components, and 2) the intermediate values (25th-75th percentile) of the strain population provide the best predictive power of the statistics of the strain populations. In addition, the success of the predictions of models trained on one rock type and tested on other rock types quantifies the similarities and differences of the precursory strain accumulation process in the six rock types. These similarities suggest the potential existence of a unified theory of brittle rock deformation for a range of rock types.

58 GEOSCIENCES↗
DOE OSTI2023

Physics informed deep neural network embedded in a chemical transport model for the Amazon rainforest

Secondary organic aerosols (SOA) are fine particles in the atmosphere, which interact with clouds, radiation and affect the Earth’s energy budget. SOA formation involves chemistry in gas phase, aqueous aerosols, and clouds. Simulating these chemical processes involve solving a stiff set of differential equations, which are computationally expensive steps for three-dimensional chemical transport models. Deep neural networks (DNNs) are universal function approximators that could be used to represent the complex nonlinear changes in aerosol physical and chemical processes; however, key challenges such as generalizability to extended time periods, preservation of mass balance, simulating sparse model outputs, and maintaining physical constraints have limited their use in atmospheric chemistry. Here, we develop an approach of using a physics-informed DNN that overcomes previous such challenges and demonstrates its applicability for the chemical formation processes of isoprene epoxydiol SOA (IEPOX-SOA) over the Amazon rainforest. The DNN is trained with data generated by simulating IEPOX-SOA over the entire atmospheric column, using the Weather Research and Forecasting Model coupled with Chemistry (WRF-Chem). The trained DNN is then embedded within WRF-Chem to replace the computationally expensive default solver of IEPOX-SOA formation. The trained DNN predictions generalizes well with the default model simulation of the IEPOX-SOA mass concentrations and its size distribution (20 size bins) over several days of simulations in both dry and wet seasons. The embedded DNN reduces the computational expense of WRF-Chem by a factor of 2. Our approach shows promise in terms of application to other computationally expensive chemistry solvers in climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A physics-constrained deep learning treatment of runaway electron dynamics

An adjoint formulation leveraging a physics-informed neural network (PINN) is employed to advance the density moment of a runaway electron (RE) distribution forward in time. A distinguishing feature of this approach is that once the adjoint problem is solved, its solution can be used to project the RE density forward in time for an arbitrary initial momentum space distribution of REs. Furthermore, by employing a PINN, a parametric solution to the adjoint problem can be learned. Thus, once trained, this adjoint-deep learning framework is able to efficiently project the RE density forward in time across various plasma conditions while still including a fully kinetic description of RE dynamics. As an example application, the temporal evolution of the density of primary electrons is studied, with particular emphasis on evaluating the decay of a RE population when below threshold. Predictions from the adjoint-deep learning framework are found to be in good agreement with a traditional relativistic electron Fokker–Planck solver, for several distinct initial conditions, and across an array of physics parameters. Once trained, the PINN thus provides a means of generating RE density time histories with exceptionally low online execution time.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

A Review of Current Research Trends in Power-Electronic Innovations in Cyber–Physical Systems

Here, in this article, a broad overview of the current research trends in power-electronic innovations in cyber–physical systems (CPSs) is presented. The recent advances in semiconductor device technologies, control architectures, and communication methodologies have enabled researchers to develop integrated smart CPSs that can cater to the emerging requirements of smart grids, renewable energy, electric vehicles, trains, ships, the Internet of Things (IoT), and so on. The topics presented in this article include novel power-distribution architectures, protection techniques considering large renewable integration in smart grids, wireless charging in electric vehicles, simultaneous power and information transmission, multihop network-based coordination, power technologies for renewable energy and smart transformer, CPS reliability, transactive smart railway grid, and real-time simulation of shipboard power systems. It is anticipated that the research trends presented in this article will provide a timely and useful overview to the power-electronics researchers with broad applications in CPSs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Reducing Urban Traffic Congestion Using Deep Learning and Model Predictive Control

This article proposes a deep learning (DL)-based control algorithm—DL velocity-based model predictive control (VMPC)—for reducing traffic congestion with slowly time-varying traffic signal controls. This control algorithm consists of system identification using DL and traffic signal control using VMPC. For the training process of DL, we established a modeling error entropy loss as the criteria inspired by the theory of stochastic distribution control (SDC) originated by the fourth author. Herein, simulation results show that the proposed algorithm can reduce traffic congestion with a slowly varying traffic signal control input. Results of an ablation study demonstrate that this algorithm compares favorably to other model-based controllers in terms of prediction error, signal varying speed, and control effectiveness.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Generative Ensemble Regression: Learning Particle Dynamics from Observations of Ensembles with Physics-informed Deep Generative Models

Here, we propose a new method for inferring the governing stochastic ordinary differential equations (SODEs) by observing particle ensembles at discrete and sparse time instants, i.e., multiple “snapshots.” Particle coordinates at a single time instant, possibly noisy or truncated, are recorded in each snapshot but are unpaired across the snapshots. By training a physics-informed generative model that generates “fake” sample paths, we aim to fit the observed particle ensemble distributions with a curve in the probability measure space, which is induced from the inferred particle dynamics. We employ different metrics to quantify the differences between distributions, e.g., the sliced Wasserstein distances and the adversarial losses in generative adversarial networks. We refer to this method as generative “ensemble-regression” (GER), in analogy to the classic “point-regression,” where we infer the dynamics by performing regression in the Euclidean space. We illustrate the GER by learning the drift and diffusion terms of particle ensembles governed by SODEs with Brownian motions and Lévy processes up to 100 dimensions. We also discuss how to treat cases with noisy or truncated observations. Apart from systems consisting of independent particles, we also tackle nonlocal interacting particle systems with unknown interaction potential parameters by constructing a physics-informed loss function. Finally, we investigate scenarios of paired observations and discuss how to reduce the dimensionality in such cases by proving a convergence theorem that provides theoretical support.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

DSS-SimPy-RL (Open-DSS and SimPy based Cyber-Physical RL environment) [SWR-23-29]

Recently, numerous data-driven approaches to control an electric grid using machine learning techniques have been investigated. With the advancement of reinforcement learning (RL) based techniques, gradually the conventional optimization based solvers are being replaced with RL approach where there is uncertainty in the environment such as renewable generation or cyber system emulation. However, to train an agent efficiently, it requires numerous interactions with an environment to learn the best policies. There are numerous RL environments for the power systems based on some well-known simulators, similarly there are environment for communication domains. While majority of the cyber emulators are based in an UNIX environment, the power simulators are based in the Windows-based operating system, the generation of cyber-physical mixed domain RL environment has been challenging. Existing co-simulation methods are efficient but resource and time intensive to generate large scale data set for training RL agents. Hence, this software focuses on development and validation of a mixed domain RL environment using Open DSS for the physical side and leverages a discrete event simulator python package, SimPy, for cyber-side emulation which is Operating Systems agnostic. Further utilizing this software co-simulation and training RL agents for re-routing based resilient control for network reconfiguration and volt-var control in power distribution feeder are performed.

Sahu, Abhijeet↗
DOE OSTI2022

Machine learning analysis reveals relationship between pomacentrid calls and environmental cues

Sound production rates of fishes can be used as an indicator for coral reef health, providing an opportunity to utilize long-term acoustic recordings to assess environmental change. As acoustic datasets become more common, computational techniques need to be developed to facilitate analysis of the massive data files produced by long-term monitoring. Machine learning techniques demonstrate an advantage in the identification of fish sounds over manual sampling approaches. Here we evaluated the ability of convolutional neural networks to identify and monitor call patterns for pomacentrids (damselfishes) in a tropical reef region of the western Pacific. A stationary hydrophone was deployed for 39 mo (2014-2018) in the National Park of American Samoa to continuously record the local marine acoustic environment. A neural network was trained—achieving 94% identification accuracy of pomacentrids—to demonstrate the applicability of machine learning in fish acoustics and ecology. The distribution of sound production was found to vary on diel and interannual timescales. Additionally, the distribution of sound production was correlated with wind speed, water temperature, tidal amplitude, and sound pressure level. This research has broad implications for state-of-the-art acoustic analysis and promises to be an efficient, scalable asset for ecological research, environmental monitoring, and conservation planning.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics

Large language models have revolutionized artificial intelligence by enabling large, generalizable models trained through self-supervision. This paradigm has inspired the development of scientific foundation models (FMs). However, applying this capability to experimental particle physics is challenging due to the sparse, spatially distributed nature of detector data, which differs dramatically from natural language. This work addresses if an FM for particle physics can scale and generalize across diverse tasks. We introduce a new dataset with more than 11 million particle collision events and a suite of downstream tasks and labeled data for evaluation. We propose a novel self-supervised training method for detector data and demonstrate its neural scalability with models that feature up to 188 million parameters. With frozen weights and task-specific adapters, this FM consistently outperforms baseline models across all downstream tasks. The performance also exhibits robust data-efficient adaptation. Further analysis reveals that the representations extracted by the FM are task-agnostic but can be specialized via a single linear mapping for different downstream tasks.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Quantitative Imaging of Cobalt Phthalocyanine Distribution on Carbon Nanotubes: A Deep Learning Approach to Catalyst Characterization

Electrochemical reduction of carbon dioxide (CO 2 ) offers a pathway to valuable products, with catalysts playing a crucial role. This study investigates the distribution of cobalt tetraaminophthalocyanine (CoPc-NH 2 ) immobilized on carbon nanotubes (CNTs), utilizing high-angle annular dark-field scanning transmission electron microscopy (HAADF-STEM) to characterize CoPc-NH 2 distribution. A challenge in the quantitative HAADF-STEM analysis is the introduction of bias from manual Co atom identification. To address this, we developed and trained a convolutional neural network (CNN) using a data set generated from images of CoPc-NH 2 /CNT samples with varying Co loadings. The CNN, implemented in TensorFlow and Keras, facilitated Co atom detections. Analysis of the CNN-generated data confirmed a correlation between Co loading and surface density, consistent with findings from UV–vis spectroscopy. Furthermore, the application of Ripley’s L(d) function highlighted the presence of slight Co atom clustering. Furthermore, this work demonstrates the utility of the combined HAADF-STEM and CNN approach for providing spatially resolved information about catalyst distribution on nonplanar supports, revealing structural details that are typically lost through other characterization methods.

HAADF-STEM↗
DOE OSTI2025

Information-entropy-driven generation of material-agnostic datasets for machine-learning interatomic potentials

In contrast to their empirical counterparts, machine-learning interatomic potentials (MLIAPs) promise to deliver near-quantum accuracy over broad regions of configuration space. However, due to their generic functional forms and extreme flexibility, they can catastrophically fail to capture the properties of novel, out-of-sample configurations, making the quality of the training set a determining factor, especially when investigating materials under extreme conditions. We propose a novel automated dataset generation method based on the maximization of the information entropy of the feature distribution, aiming at an extremely broad coverage of the configuration space in a way that is agnostic to the properties of specific target materials. The ability of the dataset to capture unique material properties is demonstrated on a range of unary materials, including elements with the FCC (Al), BCC (W), HCP (Be, Re and Os), graphite (C), and trigonal (Sb, Te) ground states. MLIAPs trained to this dataset are shown to be accurate over a range of application-relevant metrics, as well as extremely robust over very broad swaths of configurations space, even without dataset fine-tuning or hyper-parameter optimization, making the approach extremely attractive to rapidly and autonomously develop general-purpose MLIAPs suitable for simulations in extreme conditions.

36 MATERIALS SCIENCE↗