Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Multiagent systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

29 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Quantum for Energy Systems and Technologies

Quantum Information Science (QIS) is expected to profoundly change the practice of science and engineering in the coming decades. QIS technology exploits quantum phenomena for performing tasks that are impossible to do today and is a rapidly progressing field, fueled by large investments from the private sector and governments. Its importance to the U.S. economy and national security is underscored by the National Quantum Initiative Act (NQIA) passed in December 2018, which creates a coordinated multiagency program to support research and training in QIS. After the NIQA signed into law, NETL has launched an initiative to apply QIS to problems encountered in energy technology development. In Jan. 2020, an Quantum for Energy Systems & Technologies (QUEST) working group was formed to establish a workforce capable of developing, reviewing, managing, and advising on QIS-related technologies for NETL and FECM. Since then, the QUEST team have been working on developing quantum sensing technology and performing quantum computing to solve energy-related problems. This poster summarized the QUEST team’s activities & accomplishments on QIS targeting energy-related applications.

Paudel, Hari P.↗
DOE OSTI2021

A Modular and Transferable Reinforcement Learning Framework for the Fleet Rebalancing Problem

Mobility on demand (MoD) systems show great promise in realizing flexible and efficient urban transportation. However, significant technical challenges arise from operational decision making associated with MoD vehicle dispatch and fleet rebalancing. For this reason, operators tend to employ simplified algorithms that have been demonstrated to work well in a particular setting. To help bridge the gap between novel and existing methods, we propose a modular framework for fleet rebalancing based on model-free reinforcement learning (RL) that can leverage an existing dispatch method to minimize system cost. In particular, by treating dispatch as part of the environment dynamics, a centralized agent can learn to intermittently direct the dispatcher to reposition free vehicles and mitigate against fleet imbalance. We formulate RL state and action spaces as distributions over a grid partitioning of the operating area, making the framework scalable and avoiding the complexities associated with multiagent RL. Numerical experiments, using real-world trip and network data, demonstrate that RL reduces waiting time by 28% to 38% for the same-day evaluation, 17% to 44% for cross-day evaluation, and 22% to 25% for cross-season evaluation compared with no rebalancing scenarios. This approach has several distinct advantages over baseline methods including: improved system cost; high degree of adaptability to the selected dispatch method; and the ability to perform scale-invariant transfer learning between problem instances with similar vehicle and request distributions.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Understanding and Predictability of Integrated Mountain Hydroclimate (Workshop Report)

Mountainous systems cover approximately 23% of Earth’s land and are distributed across all continents. They can capture and store atmospheric moisture that is then cycled through the terrestrial surface and subsurface system, released to downstream communities, and cycled back to the atmosphere. Mountain hydroclimate—characterized by steep gradients, geological, ecological, and biogeochemical diversity—is influenced by topographic forcing and elevated warming and susceptible to large subseasonal to multidecadal variability and rapid changes. Terrestrial hydrological and biogeochemical cycles also experience cascading effects from global warming impacts, such as multidecadal declines in mountain snowpack, longer growing seasons, and increased frequency and severity of extreme events like droughts and wildfires. However, little is known about the effects of these impacts and their feedbacks on climate systems and surface-subsurface compartments. Also unknown are the full implications of changing hydroclimate and extreme events on hydro biogeochemical cycles across atmosphere, terrestrial, and human systems in mountain regions and beyond. This knowledge gap is critical, given human reliance on mountain systems for stable water supply and quality. Mountain systems’ increasing vulnerability to climate change and human perturbations motivates the need to improve understanding of integrated mountain hydroclimate (IMHC) systems and their feedbacks and impacts on humans across scales. However, due to large heterogeneity and strong gradients, coupled natural-human processes in mountain regions present significant challenges for observations, modeling, predictions, and projections. Motivated by gaps in mountain hydroclimate understanding, observations, and modeling and the need for credible projections of future changes, the U.S. Department of Energy’s (DOE) Biological and Environmental Research (BER) program organized a virtual workshop on “Understanding and Predictability of Integrated Mountain Hydroclimate.” Sponsored by BER’s Earth and Environmental Systems Sciences Division (EESSD), the workshop aimed to inform and catalyze EESSD’s growing interests in enhancing predictive understanding of IMHC. Organizers structured the workshop to identify (1) knowledge gaps, (2) observational and modeling challenges, (3) short-term (1 to 3 years) and long-term (10 years and beyond) research opportunities, and (4) strategies for fostering collaboration and coordination. To address the outstanding challenges of IMHC, the workshop included two sessions organized by disciplinary, cross-disciplinary, and crosscutting science topics. The disciplinary and cross-disciplinary topics focused on essential IMHC elements: atmosphere, terrestrial, and human systems and their interactions. Breakout sessions on disciplinary and cross-disciplinary topics facilitated identification of crosscutting topics and central emerging themes. Session 1 focused on connecting existing DOE investments to accelerate progress related to scientific challenges in understanding mountain hydroclimate. In Session 2, participants further explored key Session 1 takeaways through the lens of multiagency collaborations and coordination.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Multirobot Collaborative Pursuit Target Robot by Improved MADDPG

Policy formulation is one of the main problems in multirobot systems, especially in multirobot pursuit-evasion scenarios, where both sparse rewards and random environment changes bring great difficulties to find better strategy. Existing multirobot decision-making methods mostly use environmental rewards to promote robots to complete the target task that cannot achieve good results. This paper proposes a multirobot pursuit method based on improved multiagent deep deterministic policy gradient (MADDPG), which solves the problem of sparse rewards in multirobot pursuit-evasion scenarios by combining the intrinsic reward and the external environment. The state similarity module based on the threshold constraint is as a part of the intrinsic reward signal output by the intrinsic curiosity module, which is used to balance overexploration and insufficient exploration, so that the agent can use the intrinsic reward more effectively to learn better strategies. The simulation experiment results show that the proposed method can improve the reward value of robots and the success rate of the pursuit task significantly. The intuitive change is obviously reflected in the real-time distance between the pursuer and the escapee, the pursuer using the improved algorithm for training can get closer to the escapee more quickly, and the average following distance also decreases.

Zhou, Xiao↗
DOE OSTI2026

Extreme-scale EV charging infrastructure planning for last-mile delivery using high-performance parallel computing

Here, this paper addresses stochastic charger location and allocation problems under queue congestion for last-mile delivery using electric vehicles (EVs). The objective is to decide where to open charging stations and how many chargers of each type to install, subject to budgetary and waiting-time constraints. We formulate the problem as a mixed-integer non-linear program, where each station-charger pair is modeled as a multiserver queue with stochastic arrivals and service times to capture the notion of waiting in fleet operations. The model is extremely large, with billions of variables and constraints for a typical metropolitan area; even loading the model in solver memory is difficult, let alone solving it. To address this challenge, we develop a Lagrangian-based dual decomposition framework that decomposes the problem by station and leverages parallelization on high-performance computing systems, where the subproblems are solved by using a cutting plane method and their solutions are collected at the master level. We also develop a three-step rounding heuristic to transform the fractional subproblem solutions into feasible integral solutions. Computational experiments on data from the Chicago metropolitan area with hundreds of thousands of households and thousands of candidate stations show that our approach produces high-quality solutions in cases where existing exact methods cannot even load the model in memory. We also analyze various policy scenarios, demonstrating that combining existing depots with newly built stations under multiagency collaboration substantially reduces costs and congestion. These findings offer a scalable and efficient framework for developing sustainable large-scale EV charging networks.

Capacity allocation↗
DOE OSTI2025

Controlled environment agriculture: An opportunity to strengthen interagency research collaboration in the US government

Challenges facing food production and agricultural systems are increasingly interconnected with economic, security, health, and equity issues, among others. Threats such as extreme weather, economic volatility, and shrinking water resources and arable land, influence our ability to maintain a safe and resilient food supply. One promising solution to these threats is controlled environment agriculture (CEA). In many cases, CEA can drastically reduce the amount of water and land used in crop production while increasing productivity. Operations may be established in nearly any environment and harvests can take place year-round, supporting food system resiliency and sustainability. CEA sits at the nexus of a number of disciplines and industries, making it well suited for transdisciplinary and multi-institutional research coordination. Herein, authors from multiple US government agencies present CEA as a case study in improving cross-agency research collaboration. The federal government houses a range of scientific expertise and research capabilities, positioning scientists to lead national and global efforts in transdisciplinary, interagency approaches to complex challenges. Navigating cross-agency collaboration can be a challenge, especially coordinating across different scientific disciplines, geographic locations, and funding mechanisms. To enhance multiagency efforts, collaborators could prioritize (i) organizing personnel and resources, (ii) enhancing existing multiagency collaborations, and (iii) focusing on further opportunities for coordination. Adopting these approaches could enable federal researchers to reinforce and advance academic and industry efforts to address current CEA challenges while solidifying the United States as a leader in this arena.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

Joint Planning of EV Fast Charging Stations and Power Distribution Systems With Balanced Traffic Flow Assignment

To tackle the challenges introduced by the fast-growing charging demand of electric vehicles (EVs), the power distribution systems (PDSs) and fast charging stations (FCSs) of EVs should be planned and operated in a more coordinated fashion. However, existing planning approaches generally aim to minimize investment costs in PDSs while ignoring the risk of worsening traffic conditions. To overcome this research gap, this article integrates the interests of traffic networks into PDS and FCS joint planning model to mitigate negative impacts on traffic conditions caused by installing FCSs. First, a novel microscopic method that is different from traditional assignment methods is proposed to simulate the influences of FCSs on traffic flows and EV charging loads. Then, a multiobjective joint planning model is developed to minimize both the planning costs and unbalanced traffic flows. A new bilayer Benders decomposition algorithm is designed to solve the proposed joint planning model. Numerical results on two practical systems in China validate the feasibility of our microscopic method by comparing the simulated results with real data. Compared with existing approaches, it is also demonstrated that the proposed joint planning approach helps to balance traffic flow assignments and relieve traffic congestion.

bilayer expanded Benders decomposition↗
DOE OSTI2021

PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems [SWR-22-07]

NREL's PowerGridworld provides a modular simulation environment for training heterogenous, grid-aware, multi-agent reinforcement learning (RL) policies at scale. The package enables the user to create component gym environments that can be composed into more complex agents. For example, a grid interactive building environment can be created by composing together component environments each encapsulating the building, PV, and battery physics. These multi-component environments can then be combined into multi-agent simulation where each agent's power consumption/injection becomes an input for solving the optimal power flow on a distribution feeder modeled in OpenDSS. Information from OpenDSS, such as bus voltages and line flows, can be included in the agents' observation spaces to enable grid-aware rewards. The default API for the PowerGridworld simulator conforms to RLLib's MultiAgent API and thus enables distributed training using HPC and cloud resources.

Biagioni, David↗
DOE OSTI2024

Directing Nanoparticle Organization in Response to Diverse Chemical Inputs

Signaling cascades are crucial for transducing stimuli in biological systems, enabling multiple stimuli to regulate a downstream target with precisely controlled timing and amplifying signals through a series of intermediary reactions. Developing a robust signaling system with such capabilities would be pivotal for programming complex behaviors in synthetic DNA-based molecular devices. However, although “software” such as nucleic acid circuits could potentially be harnessed to relay signals to DNA-based nanostructure hardware, such explorations have been limited. Here, in this study, we develop a platform for transducing a variety of stimuli via messenger-mediated reactions to regulate the release and reloading of gold nanoparticles (AuNPs) in a 3D DNA framework. In the first step, an in vitro transcription circuit is engineered to sense and amplify chemical stimuli, including arbitrary DNA sequences and proteins, producing RNA. In the second step, the RNA releases the DNA-coated AuNPs from the DNA framework via a strand displacement reaction. AuNP reloading is controlled by a separate step driven by degradation of the RNA. Our platform holds promise for applications requiring dynamic multiagent control over DNA-based devices, offering a versatile tool for advanced molecular device engineering.

36 MATERIALS SCIENCE↗