Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Multiagent systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

The 1995 NASA guide to graduate support

The future of the United States is in the classrooms of America and tomorrow's scientific and technological capabilities are derived from today's investments in research. In 1980, NASA initiated the Graduate Student Researchers Program (GSRP) to cultivate additional research ties to the academic community and to support promising students pursuing advanced degrees in science and engineering. Since then, approximately 1300 students have completed the program's requirements. In 1987, the program was expanded to include the Underrepresented Minority and Disabled Focus (UMDF) Component. This program was designed to increase participation of underrepresented groups in graduate study and research and, ultimately, in space science and aerospace technology careers. Approximately 270 minority students have completed the program's requirements while making significant contributions to the nation's aerospace efforts. Continuing to expand fellowship opportunities, NASA announced the Graduate Student Fellowships in Global Change Research in 1990. Designed to support the rapid growth in the study of earth as a system, more than 250 fellowships have been awarded. And, in 1992, NASA announced opportunities in the multiagency High Performance Computing and Communications (HPCC) Program designed to accelerate the development and application of massively parallel processing. Approximately five new fellowships will be awarded yearly. This booklet will guide you in your efforts to participate in programs for graduate student support.

Source record↗
DOE OSTI2023

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning

Constrained multiagent reinforcement learning (C-MARL) is gaining importance as MARL algorithms find new applications in real-world systems ranging from energy systems to drone swarms. Most C-MARL algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of this penalty term on the MARL problem. First, we show that the standard practice of using the constraint function as the penalty leads to a weak notion of safety. However, by making simple modifications to the penalty term, we can enforce meaningful probabilistic (chance and conditional value at risk) constraints. Second, we quantify the effect of the penalty term on the value function, uncovering an improved value estimation procedure. We use these insights to propose a constrained multiagent advantage actor critic (C-MAA2C) algorithm. Simulations in a simple constrained multiagent environment affirm that our reinterpretation of the primal-dual method in terms of probabilistic constraints is effective, and that our proposed value estimate accelerates convergence to a safe joint policy.

chance constraints↗
DOE OSTI2021

A coupled human–natural system analysis of freshwater security under climate and population change

Limited water availability, population growth, and climate change have resulted in freshwater crises in many countries. Jordan’s situation is emblematic, compounded by conflict-induced population shocks. Integrating knowledge across hydrology, climatology, agriculture, political science, geography, and economics, we present the Jordan Water Model, a nationwide coupled human–natural-engineered systems model that is used to evaluate Jordan’s freshwater security under climate and socioeconomic changes. The complex systems model simulates the trajectory of Jordan’s water system, representing dynamic interactions between a hierarchy of actors and the natural and engineered water environment. A multiagent modeling approach enables the quantification of impacts at the level of thousands of representative agents across sectors, allowing for the evaluation of both systemwide and distributional outcomes translated into a suite of water-security metrics (vulnerability, equity, shortage duration, and economic well-being). Model results indicate severe, potentially destabilizing, declines in freshwater security. Per capita water availability decreases by approximately 50% by the end of the century. Without intervening measures, >90% of the low-income household population experiences critical insecurity by the end of the century, receiving <40 L per capita per day. Widening disparity in freshwater use, lengthening shortage durations, and declining economic welfare are prevalent across narratives. To gain a foothold on its freshwater future, Jordan must enact a sweeping portfolio of ambitious interventions that include large-scale desalinization and comprehensive water sector reform, with model results revealing exponential improvements in water security through the coordination of supply- and demand-side measures.

42 ENGINEERING↗
DOE OSTI2023

Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning: Preprint

We study multiagent reinforcement learning (MARL) with constraints. This setting is gaining importance as MARL algorithms find new applications in real-world systems ranging from power grids to drone swarms. Most constrained MARL (C-MARL) algorithms use a primal-dual approach to enforce constraints through a penalty function added to the reward. In this paper, we study the structural effects of the primal-dual approach on the constraints and value function. First, we show that using the constraint evaluation as the penalty leads to a weak notion of safety, but by making simple modifications to the penalty function, we can enforce meaningful probabilistic safety constraints. Second, we show that the penalty term changes the value function in a way that is easy to model, and demonstrate the consequences of not doing so. We conclude with simulations in a simple constrained multiagent environment to back up the theoretical results.

data-driven control↗
NASA NTRS2004

A Multiagent Modeling Environment for Simulating Work Practice in Organizations

In this paper we position Brahms as a tool for simulating organizational processes. Brahms is a modeling and simulation environment for analyzing human work practice, and for using such models to develop intelligent software agents to support the work practice in organizations. Brahms is the result of more than ten years of research at the Institute for Research on Learning (IRL), NYNEX Science & Technology (the former R&D institute of the Baby Bell telephone company in New York, now Verizon), and for the last six years at NASA Ames Research Center, in the Work Systems Design and Evaluation group, part of the Computational Sciences Division (Code IC). Brahms has been used on more than ten modeling and simulation research projects, and recently has been used as a distributed multiagent development environment for developing work practice support tools for human in-situ science exploration on planetary surfaces, in particular a human mission to Mars. Brahms was originally conceived of as a business process modeling and simulation tool that incorporates the social systems of work, by illuminating how formal process flow descriptions relate to people s actual located activities in the workplace. Our research started in the early nineties as a reaction to experiences with work process modeling and simulation . Although an effective tool for convincing management of the potential cost-savings of the newly designed work processes, the modeling and simulation environment was only able to describe work as a normative workflow. However, the social systems, uncovered in work practices studied by the design team played a significant role in how work actually got done-actual lived work. Multi- tasking, informal assistance and circumstantial work interactions could not easily be represented in a tool with a strict workflow modeling paradigm. In response, we began to develop a tool that would have the benefits of work process modeling and simulation, but be distinctively able to represent the relations of people, locations, systems, artifacts, communication and information content.

Sierhuis, Maarten↗
DOE OSTI2024

Quantum for Energy Systems and Technologies

Quantum Information Science (QIS) is expected to profoundly change the practice of science and engineering in the coming decades. QIS technology exploits quantum phenomena for performing tasks that are impossible to do today and is a rapidly progressing field, fueled by large investments from the private sector and governments. Its importance to the U.S. economy and national security is underscored by the National Quantum Initiative Act (NQIA) passed in December 2018, which creates a coordinated multiagency program to support research and training in QIS. After the NIQA signed into law, NETL has launched an initiative to apply QIS to problems encountered in energy technology development. In Jan. 2020, an Quantum for Energy Systems & Technologies (QUEST) working group was formed to establish a workforce capable of developing, reviewing, managing, and advising on QIS-related technologies for NETL and FECM. Since then, the QUEST team have been working on developing quantum sensing technology and performing quantum computing to solve energy-related problems. This poster summarized the QUEST team’s activities & accomplishments on QIS targeting energy-related applications.

Paudel, Hari P.↗
NASA NTRS2005

Distributed Evaluation Functions for Fault Tolerant Multi-Rover Systems

The ability to evolve fault tolerant control strategies for large collections of agents is critical to the successful application of evolutionary strategies to domains where failures are common. Furthermore, while evolutionary algorithms have been highly successful in discovering single-agent control strategies, extending such algorithms to multiagent domains has proven to be difficult. In this paper we present a method for shaping evaluation functions for agents that provide control strategies that both are tolerant to different types of failures and lead to coordinated behavior in a multi-agent setting. This method neither relies of a centralized strategy (susceptible to single point of failures) nor a distributed strategy where each agent uses a system wide evaluation function (severe credit assignment problem). In a multi-rover problem, we show that agents using our agent-specific evaluation perform up to 500% better than agents using the system evaluation. In addition we show that agents are still able to maintain a high level of performance when up to 60% of the agents fail due to actuator, communication or controller faults.

Agogino, Adrian↗
DOE OSTI2021

A Modular and Transferable Reinforcement Learning Framework for the Fleet Rebalancing Problem

Mobility on demand (MoD) systems show great promise in realizing flexible and efficient urban transportation. However, significant technical challenges arise from operational decision making associated with MoD vehicle dispatch and fleet rebalancing. For this reason, operators tend to employ simplified algorithms that have been demonstrated to work well in a particular setting. To help bridge the gap between novel and existing methods, we propose a modular framework for fleet rebalancing based on model-free reinforcement learning (RL) that can leverage an existing dispatch method to minimize system cost. In particular, by treating dispatch as part of the environment dynamics, a centralized agent can learn to intermittently direct the dispatcher to reposition free vehicles and mitigate against fleet imbalance. We formulate RL state and action spaces as distributions over a grid partitioning of the operating area, making the framework scalable and avoiding the complexities associated with multiagent RL. Numerical experiments, using real-world trip and network data, demonstrate that RL reduces waiting time by 28% to 38% for the same-day evaluation, 17% to 44% for cross-day evaluation, and 22% to 25% for cross-season evaluation compared with no rebalancing scenarios. This approach has several distinct advantages over baseline methods including: improved system cost; high degree of adaptability to the selected dispatch method; and the ability to perform scale-invariant transfer learning between problem instances with similar vehicle and request distributions.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1992

Nuclear Electric Propulsion Technology Panel findings and recommendations

Summarized are the findings and recommendations of a triagency (NASA/DOE/DOD) panel on Nuclear Electric Propulsion (NEP) Technology. NEP has been identified as a candidate nuclear propulsion technology for exploration of the Moon and Mars as part of the Space Exploration Initiative (SEI). The findings are stated in areas of system and subsystem considerations, technology readiness, and ground test facilities. Recommendations made by the panel are summarized concerning: (1) existing space nuclear power and propulsion programs, and (2) the proposed multiagency NEP technology development program.

Doherty, Michael P.↗
DOE OSTI2023

Understanding and Predictability of Integrated Mountain Hydroclimate (Workshop Report)

Mountainous systems cover approximately 23% of Earth’s land and are distributed across all continents. They can capture and store atmospheric moisture that is then cycled through the terrestrial surface and subsurface system, released to downstream communities, and cycled back to the atmosphere. Mountain hydroclimate—characterized by steep gradients, geological, ecological, and biogeochemical diversity—is influenced by topographic forcing and elevated warming and susceptible to large subseasonal to multidecadal variability and rapid changes. Terrestrial hydrological and biogeochemical cycles also experience cascading effects from global warming impacts, such as multidecadal declines in mountain snowpack, longer growing seasons, and increased frequency and severity of extreme events like droughts and wildfires. However, little is known about the effects of these impacts and their feedbacks on climate systems and surface-subsurface compartments. Also unknown are the full implications of changing hydroclimate and extreme events on hydro biogeochemical cycles across atmosphere, terrestrial, and human systems in mountain regions and beyond. This knowledge gap is critical, given human reliance on mountain systems for stable water supply and quality. Mountain systems’ increasing vulnerability to climate change and human perturbations motivates the need to improve understanding of integrated mountain hydroclimate (IMHC) systems and their feedbacks and impacts on humans across scales. However, due to large heterogeneity and strong gradients, coupled natural-human processes in mountain regions present significant challenges for observations, modeling, predictions, and projections. Motivated by gaps in mountain hydroclimate understanding, observations, and modeling and the need for credible projections of future changes, the U.S. Department of Energy’s (DOE) Biological and Environmental Research (BER) program organized a virtual workshop on “Understanding and Predictability of Integrated Mountain Hydroclimate.” Sponsored by BER’s Earth and Environmental Systems Sciences Division (EESSD), the workshop aimed to inform and catalyze EESSD’s growing interests in enhancing predictive understanding of IMHC. Organizers structured the workshop to identify (1) knowledge gaps, (2) observational and modeling challenges, (3) short-term (1 to 3 years) and long-term (10 years and beyond) research opportunities, and (4) strategies for fostering collaboration and coordination. To address the outstanding challenges of IMHC, the workshop included two sessions organized by disciplinary, cross-disciplinary, and crosscutting science topics. The disciplinary and cross-disciplinary topics focused on essential IMHC elements: atmosphere, terrestrial, and human systems and their interactions. Breakout sessions on disciplinary and cross-disciplinary topics facilitated identification of crosscutting topics and central emerging themes. Session 1 focused on connecting existing DOE investments to accelerate progress related to scientific challenges in understanding mountain hydroclimate. In Session 2, participants further explored key Session 1 takeaways through the lens of multiagency collaborations and coordination.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Multirobot Collaborative Pursuit Target Robot by Improved MADDPG

Policy formulation is one of the main problems in multirobot systems, especially in multirobot pursuit-evasion scenarios, where both sparse rewards and random environment changes bring great difficulties to find better strategy. Existing multirobot decision-making methods mostly use environmental rewards to promote robots to complete the target task that cannot achieve good results. This paper proposes a multirobot pursuit method based on improved multiagent deep deterministic policy gradient (MADDPG), which solves the problem of sparse rewards in multirobot pursuit-evasion scenarios by combining the intrinsic reward and the external environment. The state similarity module based on the threshold constraint is as a part of the intrinsic reward signal output by the intrinsic curiosity module, which is used to balance overexploration and insufficient exploration, so that the agent can use the intrinsic reward more effectively to learn better strategies. The simulation experiment results show that the proposed method can improve the reward value of robots and the success rate of the pursuit task significantly. The intuitive change is obviously reflected in the real-time distance between the pursuer and the escapee, the pursuer using the improved algorithm for training can get closer to the escapee more quickly, and the average following distance also decreases.

Zhou, Xiao↗
DOE OSTI2026

Extreme-scale EV charging infrastructure planning for last-mile delivery using high-performance parallel computing

Here, this paper addresses stochastic charger location and allocation problems under queue congestion for last-mile delivery using electric vehicles (EVs). The objective is to decide where to open charging stations and how many chargers of each type to install, subject to budgetary and waiting-time constraints. We formulate the problem as a mixed-integer non-linear program, where each station-charger pair is modeled as a multiserver queue with stochastic arrivals and service times to capture the notion of waiting in fleet operations. The model is extremely large, with billions of variables and constraints for a typical metropolitan area; even loading the model in solver memory is difficult, let alone solving it. To address this challenge, we develop a Lagrangian-based dual decomposition framework that decomposes the problem by station and leverages parallelization on high-performance computing systems, where the subproblems are solved by using a cutting plane method and their solutions are collected at the master level. We also develop a three-step rounding heuristic to transform the fractional subproblem solutions into feasible integral solutions. Computational experiments on data from the Chicago metropolitan area with hundreds of thousands of households and thousands of candidate stations show that our approach produces high-quality solutions in cases where existing exact methods cannot even load the model in memory. We also analyze various policy scenarios, demonstrating that combining existing depots with newly built stations under multiagency collaboration substantially reduces costs and congestion. These findings offer a scalable and efficient framework for developing sustainable large-scale EV charging networks.

Capacity allocation↗