Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “multiagent systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Novel Multiagent Resource Sharing Algorithm for Control of Advanced Energy Systems

This paper implements a novel resource sharing control strategy on a fuel cell–gas turbine hybrid power system at the National Energy Technology Laboratory’s Hybrid Performance Facility (Hyper). In a fuel cell–gas turbine hybrid power system, the simultaneous interaction of the gas turbine and the fuel cell creates a tightly coupled environment characterized by conflicting dynamics. In this paper, a model-free control approach is applied to solve the tightly coupled control problem posed by this challenging environment. Specifically, this control problem is presented as a resource sharing problem that can be solved using a resource sharing algorithm that is defined based on the distribution construction concept. This algorithm creates computational agents and solves the problem through the distribution and redistribution of shared resources defined as blocks. Furthermore, two agents were created; the first agent (agent 1) controls the gas turbine speed by adjusting the electric load, and the second agent (agent 2) controls the cathode mass flow through the fuel cell using the cold-air bypass valve. A parametric study was performed over the course of 15 experimental tests for both agents 1 and 2 through an evaluation of the responses based on setpoint changes. The algorithm was shown to have behavior comparable to a previously implemented multi-input multioutput state-space controller, which was designed through a model-based control approach. The resource sharing algorithm was able to find stable performance during run-time operations without any prior system knowledge identification on the power plant and without creating models used in traditional control strategies.

25 ENERGY STORAGE↗
DOE OSTI2023

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration

Grid resilience has become a critical topic recently because of the increasing occurrence of extreme events and the growing integration of intermittent renewable energy sources. To build a resilient distribution system, this paper develops a multiagent reinforcement learning-based (MARL) method to coordinate distribution energy resources (DERs) dispatch, load pickup, and network reconfiguration for load restoration after a system outage. With the help of two types of control agents, namely critical load restoration (CLR) and coordination (COR) agents, system loads can be restored efficiently, given available resources. The effectiveness and superiority of the proposed algorithm are demonstrated through simulations and comparative studies on a real distribution feeder in Western Colorado.

distribution system↗
DOE OSTI2016

The Earth System Prediction Suite: Toward a Coordinated U.S. Modeling Capability

The Earth System Prediction Suite (ESPS) is a collection of flagship U.S. weather and climate models and model components that are being instrumented to conform to interoperability conventions, documented to follow metadata standards, and made available either under open-source terms or to credentialed users. Furthermore, the ESPS represents a culmination of efforts to create a common Earth system model architecture, and the advent of increasingly coordinated model development activities in the United States. ESPS component interfaces are based on the Earth System Modeling Framework (ESMF), community-developed software for building and coupling models, and the National Unified Operational Prediction Capability (NUOPC) Layer, a set of ESMF-based component templates and interoperability conventions. Our shared infrastructure simplifies the process of model coupling by guaranteeing that components conform to a set of technical and semantic behaviors. The ESPS encourages distributed, multiagency development of coupled modeling systems; controlled experimentation and testing; and exploration of novel model configurations, such as those motivated by research involving managed and interactive ensembles. ESPS codes include the Navy Global Environmental Model (NAVGEM), the Hybrid Coordinate Ocean Model (HYCOM), and the Coupled Ocean–Atmosphere Mesoscale Prediction System (COAMPS); the NOAA Environmental Modeling System (NEMS) and the Modular Ocean Model (MOM); the Community Earth System Model (CESM); and the NASA ModelE climate model and the Goddard Earth Observing System Model, version 5 (GEOS-5), atmospheric general circulation model.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A Multiagent Deep Reinforcement Learning-Enabled Dual-Branch Damping Controller for Multimode Oscillation

Here, this study develops a multiagent deep reinforcement learning (MADRL)-enabled framework for the decentralized cooperative control of a novel dual-branch (DB) damping controller for both low-frequency oscillation (LFO) and ultralow-frequency oscillation (ULFO). It has two branches, each of which consists of a proportional resonance (PR) and a second-order polynomial that is designed to handle target oscillation modes. To improve the robustness of the controller to system uncertainties, MADRL is developed, where multiagents are centrally trained to obtain the coordinated adaptive control policy while being executed in a decentralized manner to provide the optimal parameter setting for each controller with only local states. Comparisons with the IEEE 10-machine 39-bus system demonstrate that the proposed method achieves better robustness to uncertainties, lower communication delay, and single-point failure, as well as damping control performances for both LFO and ULFO.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Agent-Based Distributed Energy Resources for Supporting Intelligence at the Grid Edge

This article proposes a novel multi-agent framework that can link various forms of resources and power electronic systems into distributed energy resources (DER). The proposed multiagent architecture can also integrate DERs to a central controller for optimization and control to support the grid. To demonstrate the flexibility of this novel framework, the developed agent system is applied to a set of end-use systems. Furthermore, the agent framework is validated in hardware using controller-hardware-in-the-loop simulation platform.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2012

Agent-based Cyber Control Strategy Design for Resilient Control Systems: Concepts, Architecture and Methodologies

The implementation of automated regulatory control has been around since the middle of the last century through analog means. It has allowed engineers to operate the plant more consistently by focusing on overall operations and settings instead of individual monitoring of local instruments (inside and outside of a control room). A similar approach is proposed for cyber security, where current border-protection designs have been inherited from information technology developments that lack consideration of the high-reliability, high consequence nature of industrial control systems. Instead of an independent development, however, an integrated approach is taken to develop a holistic understanding of performance. This performance takes shape inside a multiagent design, which provides a notional context to model highly decentralized and complex industrial process control systems, the nervous system of critical infrastructure. The resulting strategy will provide a framework for researching solutions to security and unrecognized interdependency concerns with industrial control systems.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Multi-agent AI collaboration for digital twin development and assessment

Developing a digital twin (DT) model involves different steps that encompass formulating requirements, model development, implementation, and assessment with respect to real applications. Human expertise is required to coordinate and implement different steps in the DT development and assessment process. However, certain parts of this process can be automated using artificial intelligence (AI) agents for efficient workflow development. In this work, we test and analyze a multiagent AI collaboration with humans in the loop to automate different elements of the DT development and assessment process. To implement the workflow for multiagent AI DT development and assessment, we use Autogen, a multiagent framework developed by Microsoft. Autogen offers a modular and flexible framework for configuring and designing task-specific multiagent workflows. In this framework, large language models (LLMs) form the core intelligence of the AI agents where the quality and performance of the automated element is governed by the inherent capabilities and knowledge base of the LLM. We use retrieval augmented generation to supplement the LLM with relevant domain-specific information for DT requirement formulation. We illustrate this multiagent workflow using a case study on a thermal energy storage system, focusing on how AI agents can collaborate with humans to expedite and optimize different elements of DT development and assessment process.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

This paper proposes a model-free decentralized control framework for the voltage regulation of large-scale distribution systems through the coordinated control of PV inverters. This is achieved by developing a novel interaction mechanism between the surrogate model and the centralized training and decentralized execution multiagent deep reinforcement learning framework. Specifically, the sparse Gaussian processes regression method is first utilized to develop the surrogate model of the original distribution system for reward calculation during the training stage, where each agent represents a sub-region in the centralized fashion for coordination strategy learning. After that, the learned control rules are used to inform controllers within each sub-region for real-time decisions with only local measurements. Comparative tests among various methods on the EPRI Ckt5 test system demonstrate the effectiveness of the proposed method.

distribution system↗
DOE OSTI2023

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning

Constrained multiagent reinforcement learning (C-MARL) is gaining importance as MARL algorithms find new applications in real-world systems ranging from energy systems to drone swarms. Most C-MARL algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of this penalty term on the MARL problem. First, we show that the standard practice of using the constraint function as the penalty leads to a weak notion of safety. However, by making simple modifications to the penalty term, we can enforce meaningful probabilistic (chance and conditional value at risk) constraints. Second, we quantify the effect of the penalty term on the value function, uncovering an improved value estimation procedure. We use these insights to propose a constrained multiagent advantage actor critic (C-MAA2C) algorithm. Simulations in a simple constrained multiagent environment affirm that our reinterpretation of the primal-dual method in terms of probabilistic constraints is effective, and that our proposed value estimate accelerates convergence to a safe joint policy.

chance constraints↗
DOE OSTI2021

A coupled human–natural system analysis of freshwater security under climate and population change

Limited water availability, population growth, and climate change have resulted in freshwater crises in many countries. Jordan’s situation is emblematic, compounded by conflict-induced population shocks. Integrating knowledge across hydrology, climatology, agriculture, political science, geography, and economics, we present the Jordan Water Model, a nationwide coupled human–natural-engineered systems model that is used to evaluate Jordan’s freshwater security under climate and socioeconomic changes. The complex systems model simulates the trajectory of Jordan’s water system, representing dynamic interactions between a hierarchy of actors and the natural and engineered water environment. A multiagent modeling approach enables the quantification of impacts at the level of thousands of representative agents across sectors, allowing for the evaluation of both systemwide and distributional outcomes translated into a suite of water-security metrics (vulnerability, equity, shortage duration, and economic well-being). Model results indicate severe, potentially destabilizing, declines in freshwater security. Per capita water availability decreases by approximately 50% by the end of the century. Without intervening measures, >90% of the low-income household population experiences critical insecurity by the end of the century, receiving <40 L per capita per day. Widening disparity in freshwater use, lengthening shortage durations, and declining economic welfare are prevalent across narratives. To gain a foothold on its freshwater future, Jordan must enact a sweeping portfolio of ambitious interventions that include large-scale desalinization and comprehensive water sector reform, with model results revealing exponential improvements in water security through the coordination of supply- and demand-side measures.

42 ENGINEERING↗
DOE OSTI2023

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning: Preprint

We study multiagent reinforcement learning (MARL) with constraints. This setting is gaining importance as MARL algorithms find new applications in real-world systems ranging from power grids to drone swarms. Most constrained MARL (C-MARL) algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of the primal-dual approach on the constraints and value function. First, we show that using the constraint evaluation as the penalty leads to a weak notion of safety, but by making simple modifications to the penalty function, we can enforce meaningful probabilistic safety constraints. Second, we show that the penalty term changes the value function in a way that is easy to model, and demonstrate the consequences of not doing so. We conclude with simulations in a simple constrained multiagent environment to back up the theoretical results.

data-driven control↗
DOE OSTI2012

Weather Research and Forecasting Model's Community Variational/Ensemble Data Assimilation System: WRFDA

Data assimilation is the process by which observations are combined with short-range NWP model output to produce an analysis of the state of the atmosphere at a specified time. Since its inception in the late 1990s, the multiagency Weather Research and Forecasting (WRF) model effort has had a strong data assimilation component, dedicating two working groups to the subject. This article documents the history of the WRF data assimilation effort, and discusses the challenges associated with balancing academic, research, and operational data assimilation requirements in the context of the WRF effort to date. The WRF Model's Community Variational/Ensemble Data Assimilation System (WRFDA) has evolved over the past 10 years, and has resulted in over 30 refereed publications to date, as well as implementation in a wide range of real-time and operational NWP systems.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Quantum for Energy Systems and Technologies

Quantum Information Science (QIS) is expected to profoundly change the practice of science and engineering in the coming decades. QIS technology exploits quantum phenomena for performing tasks that are impossible to do today and is a rapidly progressing field, fueled by large investments from the private sector and governments. Its importance to the U.S. economy and national security is underscored by the National Quantum Initiative Act (NQIA) passed in December 2018, which creates a coordinated multiagency program to support research and training in QIS. After the NIQA signed into law, NETL has launched an initiative to apply QIS to problems encountered in energy technology development. In Jan. 2020, an Quantum for Energy Systems & Technologies (QUEST) working group was formed to establish a workforce capable of developing, reviewing, managing, and advising on QIS-related technologies for NETL and FECM. Since then, the QUEST team have been working on developing quantum sensing technology and performing quantum computing to solve energy-related problems. This poster summarized the QUEST team’s activities & accomplishments on QIS targeting energy-related applications.

Paudel, Hari P.↗
DOE OSTI2011

Abramovo Counterterrorism Training Center

The U.S. government has been assisting the Russian Federation (RF) Ministry of Defense (MOD) for many years with nuclear weapons transportation security (NWTS) through the provision of specialized guard escort railcars and cargo railcars with integrated physical security and communication systems, armored transport vehicles, and armored escort vehicles. As a natural continuation of the NWTS program, a partnership has been formed to construct a training center that will provide counterterrorism training to personnel in all branches of the RF MOD. The Abramovo Counterterrorism Training Center (ACTC) is a multinational, multiagency project with funding from Canada, RF and the U.S. Departments of Defense and Energy. ACTC will be a facility where MOD personnel can conduct basic through advanced training in various security measures to protect Category IA material against the threat of terrorist attack. The training will enhance defense-in-depth principles by integrating MOD guard force personnel into the overall physical protection systems and improving their overall response time and neutralization capabilities. The ACTC project includes infrastructure improvements, renovation of existing buildings, construction of new buildings, construction of new training facilities, and provision of training and other equipment. Classroom training will be conducted in a renovated training building. Basic and intermediate training will be conducted on three different security training areas where various obstacles and static training devices will be constructed. The central element of ACTC, where advanced training will be held, is the 'autodrome,' a 3 km road along which various terrorist events can be staged to challenge MOD personnel in realistic and dynamic nuclear weapons transportation scenarios. This paper will address the ACTC project elements and the vision for training development and integrating this training into actual nuclear weapons transportation operations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2021

A Modular and Transferable Reinforcement Learning Framework for the Fleet Rebalancing Problem

Mobility on demand (MoD) systems show great promise in realizing flexible and efficient urban transportation. However, significant technical challenges arise from operational decision making associated with MoD vehicle dispatch and fleet rebalancing. For this reason, operators tend to employ simplified algorithms that have been demonstrated to work well in a particular setting. To help bridge the gap between novel and existing methods, we propose a modular framework for fleet rebalancing based on model-free reinforcement learning (RL) that can leverage an existing dispatch method to minimize system cost. In particular, by treating dispatch as part of the environment dynamics, a centralized agent can learn to intermittently direct the dispatcher to reposition free vehicles and mitigate against fleet imbalance. We formulate RL state and action spaces as distributions over a grid partitioning of the operating area, making the framework scalable and avoiding the complexities associated with multiagent RL. Numerical experiments, using real-world trip and network data, demonstrate that RL reduces waiting time by 28% to 38% for the same-day evaluation, 17% to 44% for cross-day evaluation, and 22% to 25% for cross-season evaluation compared with no rebalancing scenarios. This approach has several distinct advantages over baseline methods including: improved system cost; high degree of adaptability to the selected dispatch method; and the ability to perform scale-invariant transfer learning between problem instances with similar vehicle and request distributions.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2014

Assessment of Offshore Wind System Design, Safety, and Operation Standards

This report is a deliverable for a project sponsored by the U.S. Department of Energy (DOE) entitled National Offshore Wind Energy Resource and Design Data Campaign -- Analysis and Collaboration (contract number DE-EE0005372; prime contractor -- AWS Truepower). The project objective is to supplement, facilitate, and enhance ongoing multiagency efforts to develop an integrated national offshore wind energy data network. The results of this initiative are intended to 1) produce a comprehensive definition of relevant met-ocean resource assets and needs and design standards, and 2) provide a basis for recommendations for meeting offshore wind energy industry data and design certification requirements.

17 WIND ENERGY↗