SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration: Preprint

Grid resilience has become a critical topic recently because of the increasing occurrence of extreme events and the growing integration of intermittent renewable energy sources. To build a resilient distribution system, this paper develops a multi-agent reinforcement learning-based (MARL) method to coordinate distribution energy resources (DERs) dispatch, load pickup, and network reconfiguration for load restoration after a system outage. With the help of two types of control agents, namely critical load restoration (CLR) and coordination (COR) agents, system loads can be restored efficiently, given available resources. The effectiveness and superiority of the proposed algorithm are demonstrated through simulations and comparative studies on a real distribution feeder in Western Colorado.

distribution system↗

DOE OSTI2025

From Pollutant Removal to Renewable Energy: MoS2-Enhanced P25-Graphene Photocatalysts for Malathion Degradation and H2 Evolution

The widespread presence of pesticides—especially malathion—in aquatic environments presents a major obstacle to conventional remediation strategies, while the ongoing global energy crisis underscores the urgency of developing renewable energy sources such as hydrogen. In this context, photocatalytic water splitting emerges as a promising approach, though its practical application remains limited by poor charge carrier dynamics and insufficient visible-light utilization. Herein, we report the design and evaluation of a series of TiO2-based ternary nanocomposites comprising commercial P25 TiO2, reduced graphene oxide (rGO), and molybdenum disulfide (MoS2), with MoS2 loadings ranging from 1% to 10% by weight. The photocatalysts were fabricated via a two-step method: hydrothermal integration of rGO into P25 followed by solution-phase self-assembly of exfoliated MoS2 nanosheets. The composites were systematically characterized using X-ray diffraction (XRD), Raman spectroscopy, transmission electron microscopy (TEM), UV-Vis diffuse reflectance spectroscopy (DRS), and photoluminescence (PL) spectroscopy. Photocatalytic activity was assessed through two key applications: the degradation of malathion (20 mg/L) under simulated solar irradiation and hydrogen evolution from water in the presence of sacrificial agents. Quantification was performed using UV-Vis spectroscopy, gas chromatography–mass spectrometry (GC-MS), and thermal conductivity detection (GC-TCD). Results showed that the integration of rGO significantly enhanced surface area and charge mobility, while MoS2 served as an effective co-catalyst, promoting interfacial charge separation and acting as an active site for hydrogen evolution. Nearly complete malathion degradation (~100%) was achieved within two hours, and hydrogen production reached up to 6000 µmol g−1 h−1 under optimal MoS2 loading. Notably, photocatalytic performance declined with higher MoS2 content due to recombination effects. Overall, this work demonstrates the synergistic enhancement provided by rGO and MoS2 in a stable P25-based system and underscores the viability of such ternary nanocomposites for addressing both environmental remediation and sustainable energy conversion challenges.

Chemistry↗

DOE OSTI2020

Model Exploration of an Information-Based Healthcare Intervention Using Parallelization and Active Learning

This paper describes the application of a large-scale active learning method to characterize the parameter space of a computational agent-based model developed to investigate the impact of CommunityRx, a clinical information-based health intervention that provides patients with personalized information about local community resources to meet basic and self-care needs. Additionally, the diffusion of information about community resources and their use is modeled via networked interactions and their subsequent effect on agents' use of community resources across an urban population. A random forest model is iteratively fitted to model evaluations to characterize the model parameter space with respect to observed empirical data. We demonstrate the feasibility of using high-performance computing and active learning model exploration techniques to characterize large parameter spaces; by partitioning the parameter space into potentially viable and non-viable regions, we rule out regions of space where simulation output is implausible to observed empirical data. We argue that such methods are necessary to enable model exploration in complex computational models that incorporate increasingly available micro-level behavior data. We provide public access to the model and high-performance computing experimentation code.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗

DOE OSTI2025

Swing Contract-Based Valuation for Distributed Energy Resources in Transactive Energy Systems: A Reinforcement Learning Approach

With the proliferation of distributed energy resources (DERs) and power grids with high fractions of renewable energy, market constructs are evolving to allow DERs to participate in multiple possible markets, at different levels of grid hierarchy. The effective participation of DERs in market environments is aided by swing contract-based pricing mechanisms, whereby DERs have a two-part compensation structure – one for their reservation/commitment and another for performancedriven ex-post payment for their actual mobilization during dispatch. In this paper, we propose a reinforcement learningbased (Q-learning) approach that allows a rational DER agent to select the market it wants to participate in within a composite market environment where individual markets are coordinated by possibly different actors. The proposed Q-learning framework aids DERs in their self-valuation by implicitly maximizing their own payoff through market participation, assuming a swing contract-based compensation structure. We complement our work through simulation-based investigations where factors affecting the DER decision making process, such as parametric uncertainties in market (and grid) environments, are studied.

Naqvi, Syed Ahsan Raza↗

NASA NTRSDate not supplied

Deep Learning-Based Negotiation Strategy Selection for Cooperative Conflict Resolution in Urban Air Mobility

This paper presents a collaborative conflict resolution technique using deep neural network-based intelligent search of the solution space. This approach offers a rapid convergence to a mutually acceptable solution for real-time conflict resolution, suitable for urban air mobility operations. Furthermore, the presented technique allows operational flexibility to the urban air mobility agents where these agents can collaboratively devise the solution via integrative negotiation, based on their local utility functions, as long as such a solution does not violate the global safety thresholds. The presented machine-to-machine negotiation method is built on our prior work on holistic assessment of the airspace and potential conflict detection implemented at-the-edge, onboard the unmanned aircraft systems. This paper extends the prior work to augment decision-making at-the-edge, thereby, promising a true distributed control architecture for urban air mobility. In this approach, each agent (a) builds a potential in-flight conflict map, (b) identifies the conflicting agents, (c) dynamically prepares a list of alternatives based on its current utility functions, (d) negotiates with the conflicting agents to pick one of these alternatives, and (e) implements the negotiated alternative to mutually resolve the conflict. Note that such an approach does not require a contingency plan to be made pre-flight, as the conflict resolution strategies are decided and negotiated in real time based on the present state of the agent. The contingency plan, if available, can serve as an input to the real-time conflict resolution strategy formulation, and also can be used as a fallback plan in case the negotiation fails and the impacted agents need to switch to a rule-based/supervisory resolution mode from the discussed distributed resolution mode. The presented collaborative negotiation-based conflict resolution technique incorporates a time-dependent reward function to catalyze collaborative resolution by incentivizing the agents with local and global rewards beneficial to their business operations.

Advanced Air Mobility↗

DOE OSTI2023

Ringing Artifact Reduction Method for Ultrasound Reconstruction Using Multi-Agent Consensus Equilibrium

Non-destructive characterization of multi-layered structures that can be accessed from only a single side is important for applications such as well-bore integrity inspection. Existing methods related to Synthetic Aperture Focusing Technique (SAFT) rapidly produce acceptable results but with significant artifacts. Recently, ultrasound model-based iterative reconstruction (UMBIR) approaches have shown significant improvements over SAFT. However, even these methods produce ringing artifacts due to the high fractional-bandwidth of the excitation signal.In this paper, we propose a ringing artifact reduction method for ultrasound image reconstruction that uses a multi-agent consensus equilibrium (RARE-MACE) framework. Our approach integrates a physics-based forward model that accounts for the propagation of a collimated ultrasonic beam in multi-layered media, a spatially varying image prior, and a denoiser designed to suppress the ringing artifacts that are characteristic of reconstructions from high-fractional bandwidth ultrasound sensor data. We test our method on simulated and experimental measurements and show substantial improvements in image quality compared to SAFT and UMBIR.

Alanazi, Abdulrahman M↗

DOE OSTI2020

Computation-Efficient Algorithm for Distributed Feedback Optimization of Distribution Grids

Feedback-based optimization algorithms use real-time measurements to update the optimal control for the underlying system which may not be fully identified. Recently, we have developed a distributed feedback-based algorithm [1] that avoids the requirement of fast communication between central computing and local actuator/sensor agents. This paper extends the work by greatly reducing the number of copies of variables involved in the distributed feedback-based algorithm, which results in faster convergence and lower communication requirement. The main idea is to leverage the specific structural properties of the admittance matrix for distribution systems with tree network topology. We also show the effectiveness of the proposed algorithm in simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Electric Vehicle Charging Demand in the Chicago Metropolitan Area through 2030

This report outlines the collaborative efforts between Argonne National Laboratory and Exelon in advancing the Agent-Based Transportation Energy Analysis Model (ATEAM). Aligning with ComEd’s beneficial electrification plan, this study developed eight scenarios to access the temporal and spatial distribution of charging load and demand stemming from the widespread adoption of battery electric vehicles (BEV) adoption, augmented public charging infrastructure deployment, and increased multi-unit dwelling (MUD) charging availability. Enhancements to the ATEAM model encompassed the simulation of multiple days of travel behavior, estimation of total public charging infrastructure needs, user interface refinements, and output tracking at both vehicle and charging station levels. The total electricity consumption for residential and public charging to support over 800,000 BEVs in Chicago in 2029 is projected at approximately 10.2 GWh. Enhanced MUD home charging accessibility (70%) amplifies the home charging load in the study area by 1.5% compared to the baseline scenario (10%). The widespread adoption of BEVs reduces peak charging loads, owing to their inclusion across households with diverse income levels, thus fostering a more dispersed charging activity pattern. However, widespread BEV adoption increases the peak home charging load in areas with lower median household incomes, reflecting a higher BEV concentration in these locales and, subsequently, heightened peak charging demands. In the Widespread BEV adoption scenario, fewer census tracts exhibit elevated peak loads for combined home and public charging, indicating a more even distribution of charging demand across the study area. Predominantly, peak loads for combined charging—both home and public— occur between 2 p.m. and 10 p.m. across all scenarios, encompassing the majority of census tracts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS2022

Neurosymbolic Hybrid Approach to Driver Collision Warning

There are two main algorithmic approaches to autonomous driving systems: (1) An end-to-end system in which a single deep neural network learns to map sensory input directly into appropriate warning and driving responses. (2) A mediated hybrid recognition system in which a system is created by combining independent modules that detect each semantic feature. While some researchers believe that deep learning can solve any problem, others believe that a more engineered and symbolic approach is needed to cope with complex environments with less data. Deep learning alone has achieved state-of-the-art results in many areas, from complex gameplay to predicting protein structures. In particular, in image classification and recognition, deep learning models have achieved accuracies as high as humans. But sometimes it can be very difficult to debug if the deep learning model doesn't work. Deep learning models can be vulnerable and are very sensitive to changes in data distribution. Generalization can be problematic. It's usually hard to prove why it works or doesn't. Deep learning models can also be vulnerable to adversarial attacks. Here, we combine deep learning-based object recognition and tracking with an adaptive neurosymbolic network agent, called the Non-Axiomatic Reasoning System (NARS), that can adapt to its environment by building concepts based on perceptual sequences. We achieved an improved intersection-over-union (IOU) object recognition performance of 0.65 in the adaptive retraining model compared to IOU 0.31 in the COCO data pre-trained model. We improved the object detection limits using RADAR sensors in a simulated environment, and demonstrated the weaving car detection capability by combining deep learning-based object detection and tracking with a neurosymbolic model.

Wang, Pei↗

DOE OSTI2022

The development and use of decision support framework for informing selection of select agent toxins with modelling studies to inform permissible toxin amounts

Many countries have worked diligently to establish and implement policies and processes to regulate high consequence pathogens and toxins that could have a significant public health impact if misused. In the United States, the Antiterrorism and Effective Death Penalty Act of 1996 (Public Law 104-132, 1996), as amended by the Bioterrorism Preparedness and Response Act of 2002 (Public Law 107-188, 2002) requires that the Department of Health and Human Services (HHS) [through the Centers for Disease Control and Prevention (CDC)] establish a list of bacteria, viruses, and toxins that have the potential to pose a severe threat to public health and safety. Currently, this list is reviewed and updated on a biennial basis using input from subject matter experts (SMEs). We have developed decision support framework (DSF) approaches to facilitate selection of select toxins and, where toxicity data are known, conducted modelling studies to inform selection of toxin amounts that should be excluded from select agent regulations. Exclusion limits allow laboratories to possess toxins under an established limit to support their research or teaching activities without the requirement to register with the Federal Select Agent Program. Fact sheets capturing data from a previously vetted SME workshop convened by CDC, literature review and SME input were developed to assist in evaluating toxins using the DSF approach. The output of the DSF analysis agrees with the current select toxin designations, and no other toxins evaluated in this study were recommended for inclusion on the select agent and toxin list. To inform the selection of exclusion limits, attack scenarios were developed to estimate the amount of toxin needed to impact public health. Scenarios consisted of simulated aerosol releases of a toxin in high-population-density public facilities and the introduction of a toxin into a daily consumable product supply chain. Using published inhalation and ingestion median toxic dose (TD 50 ) and median lethal dose (LD 50 ) values, where available, a range of toxin amounts was examined to estimate the number of people exposed to these amounts in these scenarios. Based on data generated by these models, we proposed toxin exclusion values corresponding to levels below those that would trigger a significant public health response (i.e., amounts estimated to expose up to ten people by inhalation or one hundred people by ingestion to LD 50 or TD 50 levels of toxin in the modeled scenarios).

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Diffuse Radiation Forcing Constraints on Gross Primary Productivity and Global Terrestrial Evapotranspiration

Abstract The diffuse radiation fertilization effect—the increase in plant productivity in the presence of higher diffuse radiation ( K ↓,d )—is an important yet understudied aspect of atmosphere‐biosphere interactions and can modify the terrestrial carbon, energy, and water budgets. The K ↓,d fertilization effect links the carbon cycle with clouds and aerosols, all of which are large sources of uncertainties for our current understanding of the Earth system and for future climate projections. Here we establish to what extent observational and modeling uncertainty in sunlight's diffuse fraction ( k d ) affects simulated gross primary productivity (GPP) and terrestrial evapotranspiration ( λE ). We find only 48 eddy covariance sites with simultaneous sufficient measurements of K ↓,d with none in the tropical climate zone, making it difficult to constrain this mechanism globally using observations. Using a land modeling framework based on the latest version of the Community Land Model, we find that global GPP ranges from 114 Pg C year −1 when using k d forcing from the Modern‐Era Retrospective analysis for Research and Applications, version 2 reanalysis to a ∼7% higher value of 122 Pg C year −1 when using the Clouds and the Earth's Radiant Energy System satellite product, with especially strong differences apparent over the tropical region (mean increase ∼9%). The differences in λE , although smaller (−0.4%) due to competing changes in shaded and sunlit leaf transpiration, can be greater than regional impacts of individual forcing agents like aerosols. Our results demonstrate the importance of comprehensively and systematically validating the simulated k d by atmosphere modules as well as the response to differences in k d within land modules across Earth System Models.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Optimizing Patient-Specific Medication Regimen Policies Using Wearable Sensors in Parkinson’s Disease

Effective treatment of Parkinson’s disease (PD) is a continual challenge for healthcare providers, and providers can benefit from leveraging emerging technologies to supplement traditional clinic care. We develop a data-driven reinforcement learning (RL) framework to optimize PD medication regimens through wearable sensors. We leverage a data set of n = 26 PD patients who wore wrist-mounted movement trackers for two separate six-day periods. Using these data, we first build and validate a simulation model of how individual patients’ movement symptoms respond to medication administration. We then pair this simulation model with an on-policy RL algorithm that recommends optimal medication types, timing, and dosages during the day while incorporating human-in-the-loop considerations on medication administration. The results show that the RL-prescribed medication regimens outperform physicians’ medication regimens, despite physicians having access to the same data as the RL agent. To validate our results, we assess our wearable-based RL medication regimens using n = 399 PD patients from the Parkinson’s Progression Markers Initiative data set. We show that the wearable-based RL medication regimens would lead to significant symptom improvement for these patients, even more so than training RL policies directly from this data set. In doing so, we show that RL models from even small data sets of wearable data can offer novel, generalizable clinical insights and medication strategies, which may outperform those derived from larger data sets without wearable data.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

42 ENGINEERING↗

NASA NTRS2004

Human-Centric Teaming in a Multi-Agent EVA Assembly Task

NASA's Human Space Flight program depends heavily on spacewalks performed by pairs of suited human astronauts. These Extra-Vehicular Activities (EVAs) are severely restricted in both duration and scope by consumables and available manpower.An expanded multi-agent EVA team combining the information-gathering and problem-solving skills of human astronauts with the survivability and physical capabilities of highly dexterous space robots is proposed. A 1-g test featuring two NASA/DARPA Robonaut systems working side-by-side with a suited human subject is conducted to evaluate human-robot teaming strategies in the context of a simulated EVA assembly task based on the STS-61B ACCESS flight experiment.

Rehnmark, Fredrik↗

NASA NTRS2008

Emergency Response Virtual Environment for Safe Schools

An intelligent emergency response virtual environment (ERVE) that provides emergency first responders, response planners, and managers with situational awareness as well as training and support for safe schools is presented. ERVE incorporates an intelligent agent facility for guiding and assisting the user in the context of the emergency response operations. Response information folders capture key information about the school. The system enables interactive 3D visualization of schools and academic campuses, including the terrain and the buildings' exteriors and interiors in an easy to use Web..based interface. ERVE incorporates live camera and sensors feeds and can be integrated with other simulations such as chemical plume simulation. The system is integrated with a Geographical Information System (GIS) to enable situational awareness of emergency events and assessment of their effect on schools in a geographic area. ERVE can also be integrated with emergency text messaging notification systems. Using ERVE, it is now possible to address safe schools' emergency management needs with a scaleable, seamlessly integrated and fully interactive intelligent and visually compelling solution.

Wasfy, Ayman↗

DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

actuated signal control↗

NASA NTRS2006

A Robust Scalable Transportation System Concept

This report documents the 2005 Revolutionary System Concept for Aeronautics (RSCA) study entitled "A Robust, Scalable Transportation System Concept". The objective of the study was to generate, at a high-level of abstraction, characteristics of a new concept for the National Airspace System, or the new NAS, under which transportation goals such as increased throughput, delay reduction, and improved robustness could be realized. Since such an objective can be overwhelmingly complex if pursued at the lowest levels of detail, instead a System-of-Systems (SoS) approach was adopted to model alternative air transportation architectures at a high level. The SoS approach allows the consideration of not only the technical aspects of the NAS", but also incorporates policy, socio-economic, and alternative transportation system considerations into one architecture. While the representations of the individual systems are basic, the higher level approach allows for ways to optimize the SoS at the network level, determining the best topology (i.e. configuration of nodes and links). The final product (concept) is a set of rules of behavior and network structure that not only satisfies national transportation goals, but represents the high impact rules that accomplish those goals by getting the agents to "do the right thing" naturally. The novel combination of Agent Based Modeling and Network Theory provides the core analysis methodology in the System-of-Systems approach. Our method of approach is non-deterministic which means, fundamentally, it asks and answers different questions than deterministic models. The nondeterministic method is necessary primarily due to our marriage of human systems with technological ones in a partially unknown set of future worlds. Our goal is to understand and simulate how the SoS, human and technological components combined, evolve.

Hahn, Andrew↗