SEARCH · Engineering Papers

Results for “Distributed System and Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Asynchronous Iterative Solvers for Extreme-Scale Computing

The Asynchronous Iterative Solvers for Extreme-Scale Computing (AsyncIS) project aims to explore more efficient numerical algorithms by decreasing their overhead. AsyncIS does this by replacing the outer Krylov subspace solver with an asynchronous optimized Schwarz method, thereby removing the global synchronization and bulk synchronous operations typically used in numerical codes. AsyncIS—a U.S. Department of Energy (DOE)-funded collaboration between Georgia Tech, the University of Tennessee, Knoxville, Temple University, and Sandia National Laboratories—also focuses on the development and optimization of asynchronous preconditioners (i.e., preconditioners that are generated and/or applied in an asynchronous fashion). The novel preconditioning algorithms that provide fine-grained parallelism enable preconditioned Krylov solvers to run efficiently on large-scale distributed systems and manycore accelerators like GPUs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

A parallel, distributed memory implementation of the adaptive sampling configuration interaction method

The many-body simulation of quantum systems is an active field of research that involves several different methods targeting various computing platforms. Many methods commonly employed, particularly coupled cluster methods, have been adapted to leverage the latest advances in modern high-performance computing. Selected configuration interaction (sCI) methods have seen extensive usage and development in recent years. However, the development of sCI methods targeting massively parallel resources has been explored only in a few research works. Here, we present a parallel, distributed memory implementation of the adaptive sampling configuration interaction approach (ASCI) for sCI. In particular, we will address the key concerns pertaining to the parallelization of the determinant search and selection, Hamiltonian formation, and the variational eigenvalue calculation for the ASCI method. Load balancing in the search step is achieved through the application of memory-efficient determinant constraints originally developed for the ASCI-PT2 method. The presented benchmarks demonstrate near optimal speedup for ASCI calculations of Cr 2 (24e, 30o) with 10 6 , 10 7 , and 3 × 10 8 variational determinants on up to 16 384 CPUs. Importantly, to the best of the authors’ knowledge, this is the largest variational ASCI calculation to date.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Distributional Deep Reinforcement Learning-Based Emergency Frequency Control

Emergency frequency control is one of the most critical approaches to maintain power system stability after major disturbances. With the increasing number of grid-connected renewable energy sources, existing model-based methods of frequency control are facing up with challenges of computational speed and scalability for large-scale systems. In this paper, the emergency frequency control problem is formulated as a Markov Decision Process (MDP) and solved through a novel Distributional Deep Reinforcement Learning (DDRL) method, namely the distributional soft actor critic (DSAC) method. Compared with other RL methods that only estimate the mean value, the proposed DSAC model estimates the distribution of value function over returns. This advancement can lead to more insights and knowledge for the agent, with the benefit of a much faster and more stable learning process, and the improved frequency control performance. Here, the simulation results on IEEE 39-bus and IEEE 118-bus systems demonstrate the effectiveness and robustness of proposed models, as well as the advantage compared to other state-of-the-art DRL algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

Performance Evaluation of Peer-to-Peer Distributed Microgrids Coordination for Voltage Regulation

This paper presents the performance evaluation of a peer-to-peer microgrids coordination algorithm for sub-transmission systems. As distributed energy resources (DERs) in distribution system start to show negative impact to the bulk power system, a paradigm shift is needed for transmission planning and operation. Because distribution substations are located far from the sub-transmission system, and it is hard to use traditional centralized control for real-time control and coordination. Thus, distributed control is a natural choice because it requires less communication and central computation. In this paper, each distribution substation is treated as a microgrid, and the peer-to-peer distributed microgrids control is formulated as a real-time optimal power flow problem to reduce the negative impact in sub-transmission systems. A distributed primal-dual optimization algorithm is adopted to solve the problem. Validation of the peer-to-peer algorithm is performed through the simulation of a real-world sub-transmission system composing of many distribution systems with high renewable penetration. Simulation results show that the peer-to-peer algorithm can achieve satisfactory performance (e.g., voltage regulation) in sub-transmission system by coordinating and controlling DERs in distribution systems.

distributed control↗

DOE OSTI2022

Performance Evaluation of Peer-to-Peer Distributed Microgrids Coordination for Voltage Regulation: Preprint

distributed control↗

DOE OSTI2021

Isotopic signatures as tools to reconstruct the primordial architecture of the Solar System

The significant reorganization of the early Solar System due to giant planet migration has hampered our understanding of where planetary bodies formed. Previously employed proxies for reconstructing the primordial planetary architecture, such as water content or oxidation state, are complicated by post-accretionary processes. Furthermore we investigate basaltic achondrites for their nucleosynthetic isotope signatures in the elements neodymium (Nd) and zirconium (Zr) and show that they are—similar to previously investigated chondritic meteorites—characterized by a relative deficit in isotopes produced by the s-process of nucleosynthesis. Importantly, these data are well correlated with nucleosynthetic signatures observed in other elements, demonstrating that s-process matter was heterogeneously distributed throughout the early Solar System. By comparing these isotopic signatures with potential proxies for Solar System reconstruction and computer modeling, we here argue that this isotopic heterogeneity in bulk meteoritic materials is linked to the original heliocentric distance of formation. Such scaling of nucleosynthetic signatures with heliocentric distance could permit reconstruction of the primordial architecture of the Solar System by ‘cosmolocating’ the accretion orbits of meteoritic parent bodies as a function of incorporated s-process matter.

58 GEOSCIENCES↗

DOE OSTI2025

Exploiting Kubernetes to Simplify the Deployment and Management of the Multi-purpose CMS Pilot Job Factory

GlideinWMS, a widely utilized workload management system in high-energy physics (HEP) research, serves as the backbone for efficient job provisioning across distributed computing resources. It is utilized by various experiments and organizations, including CMS, OSG, Dune, and FIFE, to create HTCondor pools as large as 600k cores. In particular, a shared factory service historically deployed at UCSD has been configured to interface with more than 500 routes to compute clusters. As part of our team’s initiative to modernize infrastructure and enhance scalability, we undertook the migration of the GlideinWMS factory service into the Kubernetes environment. Leveraging the flexibility and orchestration capabilities of Kubernetes, we successfully deployed the factory service within the OSG Tiger Kubernetes cluster. The major benefits Kubernetes gives us is it streamlines the management and monitoring of the factory infrastructure, and improves fault tolerance through its resilient deployment strategies. Through this case study, we aim to share insights, challenges, and best practices encountered during the migration process. Our experience underscores the benefits of embracing containerization and Kubernetes orchestration for HEP computing infrastructure, paving the way for scalability and resilience in distributed computing environments.

Dost, Jeffrey Michael [UC, San Diego (main)]↗

DOE OSTI2021

Multi-Period Active Distribution Network Planning Using Multi-Stage Stochastic Programming and Nested Decomposition by SDDIP

This paper presents a multi-period active distribution network planning (ADNP) with distributed generation (DG). The objective of the proposed ADNP is to minimize the total planning cost, subject to both investment and operation constraints. The paper proposes a multi-stage stochastic optimization model to address DG uncertainties over several periods, in which the decisions are made sequentially by only using the present-stage information. A nested decomposition method is proposed which applies the stochastic dual dynamic integer programming (SDDIP) method to address computational intractabilities of the proposed ADNP approach. The presented numerical results and discussions on a 33-bus distribution system and a large-scale 906-bus system verify the effectiveness of the proposed ADNP method and its solution method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

pnnl/arena

CFA ARENA is a novel programming model with the support of a runtime targeting asynchronous data-centric execution paradigm in a distributed system. All the machine nodes in ARENA are connected by a ring network to bring the specialized computation to the data rather than the reverse to minimize data movement. The programming interfaces are implemented using C++

Tan, Cheng↗

DOE OSTI2021

GASNet-EX Memory Kinds: Support for Device Memory in PGAS Programming Models

There is an emerging need for adaptive, lightweight communication in irregular HPC applications at exascale, where GPU accelerators provide the majority of available compute cycles. To address this need, Lawrence Berkeley National Lab is developing a programming system to support distributed-memory HPC application development using the Partitioned Global Address Space (PGAS) model. This work includes two major components: UPC++ and GASNet-EX. UPC++ is a C++ template library providing Remote Memory Access (RMA) and Remote Procedure Call (RPC) communication interfaces. GASNet-EX is a portable, high-performance communication middleware library, used by the implementations of UPC++ and many other PGAS programming models. We describe recent advances in GASNet-EX to efficiently implement zero-copy Remote Memory Access (RMA) communication to and from memory on accelerator devices such as GPUs. We demonstrate performance improvements via benchmark results from UPC++ (on Summit) and the Legion programming system (on DGX-1), both using GASNet-EX for communication.

Hargrove, Paul H↗

DOE OSTI2022

Validation of Power Distribution Models using Load Flow Analysis in an ADMS Environment

Electric utilities are facing the need for better monitoring, analysis, and control of their distribution systems. An accurate mathematical model is a key to both the development of cutting-edge, scalable model-based algorithms and the assessment of emerging technologies such as distributed energy resources (DER) for grid planning and operation. However, the constantly evolving nature of power distribution systems poses challenges to maintaining accurate models. In this paper, we propose a novel load flow based approach to validate power distribution models. Networked equipment models described according to the Common Information Model (CIM) standard and a measurement model are used to formulate the distribution load flow problem. First, a system admittance matrix (Ybus) is derived from device-level CIM parameters. Next, the operational parameters (dynamic Ybus and nodal injections) are extracted from the measurement model using sensor configuration and equipment state. An iterative power flow method is then used to compute nodal voltages and branch flows that are compared against the measurement data to find any inconsistencies in the networked equipment model. This approach is implemented within GridAPPS-D, an open-source standards-based platform for advanced distribution management system (ADMS) application development, and demonstrated on the IEEE 13-bus, 123-bus, and 8500-node test feeders.

Common information model, model validation, power ↗

DOE OSTI2022

Programming approaches for scalability, performance, and portability of combustion physics codes

Here, this paper presents the process, strategy, and results associated with porting a typical combustion physics flow solver to current state-of-the-art and future massively-parallel computer architectures. Major focus is placed on the distinct algorithmic structure of these types of codes and how it can be integrated with modern programming paradigms for heterogeneous platforms (i.e., distributed many-core systems with accelerators). An end-to-end case study is presented that exemplifies the process in a generic manner, which then serves as a clear guide with respect to the strategy and best practices leading to a robust and adaptable framework that performs well, is durable over time, is portable, and requires minimal human-effort. This end is accomplished beginning with the use of a mature, validated, structured, multiblock code framework optimized for application of both Large Eddy Simulation (LES) and Direct Numerical Simulation (DNS). This code has been ported to a variety of platforms over the past decade, including most recently the Oak Ridge Leadership Computing Facility’s “Summit” Platform. The experience gained on these multiple platforms provides general insights and thus the results presented are not specific to any one code or platform other than the overarching trend toward distributed many-core systems with accelerators in order to move toward exascale performance. The resultant performance and scalability of the ported code is demonstrated on a real-world application; a state-of-the-art rotating detonation rocket engine simulation that matches the complex geometry and boundary conditions imposed as part of a companion experimental campaign.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Machine learning-driven predictive resource management in complex science workflows

Here, the collaborative efforts of large communities in science experiments, often comprising thousands of global members, reflect a monumental commitment to exploration and discovery. Recently, advanced and complex data processing has gained increasing importance in science experiments. Data processing workflows typically consist of multiple intricate steps, and the precise specification of resource requirements is crucial for each step to allocate optimal resources for effective processing. Estimating resource requirements in advance is challenging due to a wide range of analysis scenarios, varying skill levels among community members, and the continuously increasing spectrum of computing options. One practical approach to mitigate these challenges involves initially processing a subset of each step to measure precise resource utilization from actual processing profiles before completing the entire step. While this two-staged approach enables processing on optimal resources for most of the workflow, it has drawbacks such as initial inaccuracies leading to potential failures and suboptimal resource usage, along with overhead from waiting for initial processing completion, which is critical for fast-turnaround analyses. In this context, our study introduces a novel pipeline of machine learning models within a comprehensive workflow management system, the Production and Distributed Analysis (PanDA) system. These models employ advanced machine learning techniques to predict key resource requirements, overcoming challenges posed by limited upfront knowledge of characteristics at each step. Accurate forecasts of resource requirements enable informed and proactive decision-making in workflow management, enhancing the efficiency of handling diverse, complex workflows across heterogeneous resources.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Cyber Resilient Design and Controls for Energy Systems

As the grid of today is evolving into a highly distributed and autonomous cyber-physical system with higher penetration of Distributed Energy Resources (DERs) and autonomous controls that are dependent on the cyber infrastructure, there are novel and increasing challenges to cyber-resilient operation of the grid. While the traditional mechanisms take a "bolt on" approach to cybersecurity and resiliency, this talk presents two novel technologies developed at NREL that "bake in" security and resilience into the design and control of the grid. An Adaptive Resilience Metrics framework is a novel mechanism for system owners and operators to understand the impact of cyber and physical system design and topology on the resilient operation, while also helping determine optimal policies in response to adverse cyber events. Along with that a zero-knowledge proof-based technique helps incorporate security into the controls layer by focusing on computational integrity rather than data integrity. By leveraging design and control properties unique to Operational Technology systems these technologies will not only help enhance cyber-resilience for the grid of today, but a highly distributed and autonomous grid of tomorrow.

cyber resilience↗

DOE OSTI2020

Methods and Experiences for Developing Abstractions for Data-intensive, Scientific Applications

Developing software for scientific applications that require the integration of diverse types of computing, instruments, and data present challenges that are distinct from commercial software. These applications require scale, and the need to integrate various programming and computational models with evolving and heterogeneous infrastructure. Pervasive and effective abstractions for distributed infrastructures are thus critical; however, the process of developing abstractions for scientific applications and infrastructures is not well understood. While theory-based approaches for system development are suited for well-defined, closed environments, they have severe limitations for designing abstractions for scientific systems and applications. The design science research (DSR) method provides the basis for designing practical systems that can handle real-world complexities at all levels. In contrast to theory-centric approaches, DSR emphasizes both practical relevance and knowledge creation by building and rigorously evaluating all artifacts. In this work, we show how DSR provides a well-defined framework for developing abstractions and middleware systems for distributed systems. Specifically, we address the critical problem of distributed resource management on heterogeneous infrastructure over a dynamic range of scales, a challenge that currently limits many scientific applications. We use the pilot-abstraction, a widely used resource management abstraction for high-performance, high throughput, big data, and streaming applications, as a case study for evaluating the DSR activities. For this purpose, we analyze the research process and artifacts produced during the design and evaluation of the pilot-abstraction. We find DSR provides a concise framework for iteratively designing and evaluating systems. Finally, we capture our experiences and formulate different lessons learned.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗

DOE OSTI2023

Can the United States Maintain Its Leadership in High-Performance Computing? - A report from the ASCAC Subcommittee on American Competitiveness and Innovation to the ASCR Office

The United States (U.S.) is no longer the unambiguous leader in the vitally important field of high performance computing (HPC). Japan, the European Union (EU), and China have fielded systems that are on par with our fastest supercomputers. The supply chain for everything from semiconductors to scientific software is globally distributed. Yet our economic future and security depend critically on our ability to innovate faster than our competitors, and the speed of innovation depends increasingly on large-scale computational science and engineering and thus HPC. How should the United States respond to this challenge? This report seeks to initiate a new and potentially transformative national discussion on this vital question. The Department of Energy’s (DOE) Advanced Scientific Computing Research (ASCR) program is well-positioned to make informed, targeted decisions about where the United States should cooperate and where it should compete in the global market for scientific exploration and discovery. By setting its sights on problems critical to our nation and the world, by establishing productive new collaborations, and by making strategic investments, ASCR can restore and maintain U.S. scientific leadership in the critical areas described in this report while strengthening our research infrastructure and training a large, diverse cohort of scientists. In doing so, ASCR and its scientists will pave the way for a secure and prosperous future for America. For more than 30 years, the ASCR program has provided the HPC and networking capabilities and expertise needed to support DOE’s mission to advance the national, economic, and energy security of the United States. The program now faces the challenge of developing and deploying the next generation of HPC systems and technologies, as well as supporting the application of HPC and artificial intelligence (AI) technologies to a wide range of scientific and engineering research problems. Through its research and development efforts, the ASCR program must also advance the state of the art in HPC and accelerate the pace of scientific discovery and technological innovation. Fulfilling this promise will require significantly increased investments, as well as innovative policies and programs. This subcommittee is aware that we are making recommendations and calls for action at a time when federal resources are limited. We understand that a wide range of competing priorities must be balanced by the nation’s leaders and that there is a need to leverage resources in new ways and seek efficiencies in facilities and operations. However, we must not let these realities limit our imagination or silence our advocacy. The ASCR program is a key part of the U.S. research infrastructure and an important component of economic growth and U.S. competitiveness. ASCR has a responsibility to pursue its mission, including advanced scientific computing, applications of AI technologies, and the required advanced research facilities, with determination and enthusiasm. To fulfill the scientific enterprise’s responsibility to the nation, the ASCR program must not only develop and publish a clear vision with an associated list of goals, priorities, and recommendations but also demonstrate scientific leadership by consistently securing long-term funding. This will allow the program to build on its achievements to date, to realize its ambitious vision, and to make lasting contributions to the field.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2020

Distributed Conditions for Small-signal Stability of Power Grids and Local Control Design

Operating modern power grids with stability guarantees is markedly important. Typical methods for analyzing and certifying power grid stability are largely centralized relying on the ability of the system operator to gather network-wide information and accurately compute the system's eigenvalues. These methods are oftentimes not privacy-preserving and computationally burdensome. They are therefore, not well-suited to modern power grids where small-signal stability has to be evaluated timely, efficiently and in a privacy-preserving fashion. Herein, we introduce a distributed methodology for certifying small-signal stability of power grids and designing the local controllers. First, we analytically derive distributed conditions for network-wide stability that bus agents can inspect using local information. By leveraging these conditions, we then introduce a distributed control design algorithm (DCDA) that can guide the local control design so that stability of the interconnected system is guaranteed. The agents that adopt the proposed distributed algorithm are responsible for tuning their local controllers, producing their local control commands and ensuring that their local stability condition is met. The system operator is only responsible for verifying network-wide stability upon receiving affirmative responses from all agents and, announcing, that the overall system is stable. The proposed DCDA algorithm is numerically validated via simulations using the IEEE 39-bus system.

24 POWER TRANSMISSION AND DISTRIBUTION↗