Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Towards Autonomous Lunar Resource Excavation via Deep Reinforcement Learning

To support sustainable infrastructure on the Moon, NASA needs to leverage lunar resources for in-situ processing and construction. NASA’s Regolith Advanced Surface Systems Operations Robot (RASSOR) is principally designed to mine and deliver regolith for these tasks. To reliably perform these operations on the lunar surface, RASSOR's sensors and control systems need to be robust and maximize information extracted from a reduced sensor payload. Herein, we present our findings from the Intelligent Capabilities Enhanced RASSOR project. We created reduced-order simulation environments in which we applied reinforcement learning algorithms to learn autonomous trenching controllers and produced state estimation architectures. We developed two simulations: a 2D excavation simulation used to facilitate parameter selection, and a 3D simulation developed using a game physics engine to simulate simplified soil interactions and incorporate robotic agents parameterized by dynamic models. Within these simulations, we learned autonomous excavation routines that exceed excavation efficiency measures as compared against RASSOR's existing control and teleoperation-based methods.

RASSOR↗
NASA NTRSDate not supplied

Towards Autonomous Lunar Resource Excavation via Deep Reinforcement Learning

To support sustainable infrastructure on the Moon, NASA needs to leverage lunar resources for in-situ processing and construction. NASA’s Regolith Advanced Surface Systems Operations Robot (RASSOR) is principally designed to mine and deliver regolith for these tasks. To reliably perform these operations on the lunar surface, RASSOR's sensors and control systems need to be robust and maximize information extracted from a reduced sensor payload. Herein, we present our findings from the Intelligent Capabilities Enhanced RASSOR project. We created reduced-order simulation environments in which we applied reinforcement learning algorithms to learn autonomous trenching controllers and produced state estimation architectures. We developed two simulations: a 2D excavation simulation used to facilitate parameter selection, and a 3D simulation developed using a game physics engine to simulate simplified soil interactions and incorporate robotic agents parameterized by dynamic models. Within these simulations, we learned autonomous excavation routines that exceed excavation efficiency measures as compared against RASSOR's existing control and teleoperation-based methods.

RASSOR↗
DOE OSTI2023

EV charging site day-ahead load prediction in a synthetic environment for RL based grid-informed charging

Ensuring grid health in the face of increasing demand for power is an emerging challenge especially due to transportation electrification. A free market approach to influencing electric vehicle (EV) load through grid-informed hourly dynamic pricing is introduced in this work. The setting of charging price is done by a reinforcement learning (RL) agent that learns the complicated dynamics by interacting with a synthetic environment. This synthetic environment is a combination of distribution feeder simulation, EV charger user behavior dynamics, and EV charging simulation. A key module in this synthetic environment involves obtaining the day-ahead charging profile of EV charging stations based on real-world past data. The day-ahead prediction is also useful in other traditional optimizations related to EV charge scheduling. The proposed approach involves using EV charging data from two different past time horizons – one to determine the shape of the daily profile and the other to determine a scaling value to capture actual energy consumption. Real-world charging data over many years from the ACN charging network has been used to demonstrate the ability to predict the day-ahead profile with only charge session data. Both Python and MATLAB have been used for data cleaning, processing, analysis, and prediction.

Suryanarayana, Harish↗
DOE OSTI2023

Traffic Signal Optimization by Integrating Reinforcement Learning and Digital Twins

Machine learning (ML) methods, especially reinforcement learning (RL), have been widely considered for traffic signal optimization in intelligent transportation systems. Most of these ML methods are centralized, lacking in scalability and adaptability in large traffic networks. Further, it is challenging to train such ML models due to the lack of training platforms and/or the cost of deploying and training in a real traffic networks. This paper presents an approach for the integration of decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin (DT) to optimize traffic signals for the reduction of traffic congestion and network-wide fuel consumption related to stopping. Specifically, the DGMARL agents learn traffic state patterns and make decisions regarding traffic signal control with assistance from a Digital Twin module, which simulates and replicates the traffic behaviors of a real traffic network. The proposed approach was evaluated using PTV-Vissim [1], a microscopic traffic simulation platform. PTV-Vissim is also the simulation engine of the DT, enabling emulation and optimization of the traffic signals on the MLK Smart Corridor in Chattanooga, Tennessee. Compared to an actuated signal control baseline approach, experiment results show that Eco_PI, a developed performance measure capturing the impact of stops on fuel consumption, was reduced by 44.27% in a 24-hour and an average of 29.88% in a PM-peak-hour scenario.

Multi-Agent Reinforcement Learning, Digital Twin, ↗
DOE OSTI2023

Massive Trajectory Data Based on Patterns of Life

Individual human location trajectory and check-in data have been the driving force for human mobility research in recent years. However, existing human mobility datasets are very limited in size and representativeness. For example, one of the largest and most commonly used datasets of individual human location trajectories, GeoLife, captures fewer than two hundred individuals. To help fill this gap, this Data and Resources paper leverages an existing data generator based on fine-grained simulation of individual human patterns of life to produce large-scale trajectory, check-in, and social network data. In this simulation, individual human agents commute between their home and work locations, visit restaurants to eat, and visit recreational sites to meet friends. We provide large datasets of months of simulated trajectories for two example regions in the United States: San Francisco and New Orleans. In addition to making the datasets available, we also provide instructions on how the simulation can be used to re-generate data, thus allowing researchers to generate the data locally without downloading prohibitively large files.

Amiri, Hossein↗
NASA NTRS2005

Apex Reference Manual 3.0 Beta

Apex is a toolkit for constructing software that behaves intelligently and responsively in demanding task environments. Reflecting its origin at NASA where Apex continues to be developed, current applications include: a) Providing autonomous mission management and tactical control capabilities for unmanned aerial vehicles including an autonomous surveillance helicopter and a simulation prototype of an unmanned fixed-wing aircraft to be used for wildfire mapping; b) Simulating human air traffic controllers, pilots and astronauts to help predict how people might respond to changes in equipment or procedures; and c) Predicting the precise duration and sequence of routine human behaviors based on a human-computer interaction engineering technique called CPM-GOMS. Among Apex s components are a set of implemented reasoning services, such as those for reactive planning and temporal pattern recognition; a software architecture that embeds and integrates these services and allows additional reasoning elements to be added as extensions; a formal language for specifying agent knowledge; a simulation environment to facilitate prototyping and analysis; and Sherpa, a set of tools for visualizing autonomy logic and runtime behavior. In combination, these are meant to provide a flexible and usable framework for creating, testing, and deploying intelligent agent software. Overall, our goal in developing Apex is to lower economic barriers to developing intelligent software agents. New ideas about how to extend or modify the system are evaluated in terms of their impact in reducing the time, expertise, and inventiveness required to build and maintain applications. For example, potential enhancements to the AI reasoning capabilities in the system are reviewed not only for usefulness and distinctiveness, but also for their impact on the readability and general usability of Apex s behavior representation language (PDL) and on the transparency of resulting behavior. A second central part of our approach is to iteratively refine Apex based on lessons learned from as diverse a set of applications as possible. Many applications have been developed by users outside the core development team including engineers, researchers, and students. Usability is thus a central concern for every aspect of Apex visible to a user, including PDL, Sherpa, the Apex installation process, APIs, and user documentation. Apex users vary in their areas of expertise and in their familiarity with autonomy technology. Focusing on usability, a development philosophy summarized by the project motto "Usable Autonomy," has been important part of enabling diverse users to employ Apex successfully and to provide feedback needed to guide iterative, user-centered refinement.

Freed, Michael A.↗
DOE OSTI2023

Utilizing grid–supportive load response to shape resilient frequency control of the power grid

The increasing penetration of renewable energy sources and the retirement of conventional generation units have decreased system inertia, making power systems more vulnerable to resilience and stability issues. To address this problem, this paper proposes a novel approach using grid-supportive loads (GSLs) to provide a fast and concise primary frequency response and a deep deterministic policy gradient agent-based secondary controller to restore the system frequency to the nominal value. The proposed method is evaluated on the single-area and multi-area test systems. The simulation results demonstrate that using GSLs enhances the power system's stability and resilience. Compared to conventional controllers, the frequency nadir is improved with GSLs. Additionally, the proposed method effectively enhances resilience even with high penetration. These findings indicate that the proposed approach can improve the resilience and stability of power systems and provide a promising solution for future power systems. The results of this study underscore the importance of utilizing innovative approaches to enhance the stability and resilience of power systems in the context of high penetration of renewable energy sources and the retirement of conventional generation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Bayesian calibration of stochastic agent based model via random forest

SAND2024-11403O The Bayesian Calibration of Stochastic Agent-based Model via Random Forest is a code that was developed in concurrence with an article by the same name that was written for a journal. The code reproduces simulation results and plots from the article. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
DOE OSTI2023

Building a Computational and Experimental Rapid Response Pipeline to Counter the Coronavirus Disease 2019 Outbreak and Emerging Biothreats

The COVID-19 pandemic underscored the promise of monoclonal antibody-based prophylactic and therapeutic drugs, especially where protective candidates can be rapidly identified or developed for emerging biothreats and escape variants. Current cutting-edge technologies for this purpose still rely on pathogen-exposed convalescent volunteers and a large screening effort to find a proverbial needle in a haystack. Computational design of protective antibodies based on pre-existing templates skips those requirements and allows for greater control over the breadth and target epitope, while also co-optimizing for potency and developability or other biophysical characteristics. We approached this problem by building and expanding an in vitro experimental rapid antibody production and characterization pipeline to support development of an autonomous, closed loop, active learning software system based on structural simulation and ground truth experimental data to design and evaluate antibody antigen interactions. Starting from early in the pandemic, we targeted SARS-CoV-2, the causative agent of COVID-19, by re-purposing neutralizing antibodies against SARS-CoV-1 that had been identified in the wake of that outbreak in the early 2000’s. We successfully re-targeted three different anti-SARS-CoV-1 antibodies to neutralize SARS-CoV-2 in vitro, where the antibodies were generated externally and tested through conventional binding and neutralization assays internally or with collaborators. As antibodies were identified from the blood of humans infected with SARS-CoV-2, we shifted to collaborate with academic partners to develop improved versions of their human-derived antibodies. This work reached its most important stage in rapid response to the emergence of the Omicron variant of concern (VOC) in late 2021. In a matter of weeks, enabled by on demand innovation to our screening pipeline, we computationally designed and experimentally characterized derivative antibodies of COV2-2130, one of two antibodies from Vanderbilt that form the basis of the AstraZeneca Evusheld prophylactic drug product. This drug product suffers a serious loss of efficacy against Omicron BA.1 and BA.1.1, the first Omicron strains. Due to tight integration of computational design and experimental evaluation, we were able to identify a pair of designs with potent neutralization of the main targets Omicron BA.1 and BA.1.1; but also the earlier Delta variant, and subsequent Omicron strains including BA.2, BA.4, BA.5, and BA.2.75, demonstrating that our multi-target design process can, by its nature, produce robust antibody designs that strictly improve over the parental antibody. These results, recognized by a 2022 Director’s Science and Technology award, have enabled the follow-on GUIDE program, to commence in FY23. While earlier design campaigns were substantially outsourced, we have engineered better and faster processes internally to better compliment, calibrate, and speed computational designs. As part of the follow-on GUIDE program, we will stand up a rapid and high-throughput antibody production and characterization facility staffed with the expertise and capabilities to foster our current collaboration across PLS and ENG as well as other partnerships toward computational design of biologics.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2007

Intelligent Motion and Interaction Within Virtual Environments

What makes virtual actors and objects in virtual environments seem real? How can the illusion of their reality be supported? What sorts of training or user-interface applications benefit from realistic user-environment interactions? These are some of the central questions that designers of virtual environments face. To be sure simulation realism is not necessarily the major, or even a required goal, of a virtual environment intended to communicate specific information. But for some applications in entertainment, marketing, or aspects of vehicle simulation training, realism is essential. The following chapters will examine how a sense of truly interacting with dynamic, intelligent agents may arise in users of virtual environments. These chapters are based on presentations at the London conference on Intelligent Motion and Interaction within a Virtual Environments which was held at University College, London, U.K., 15-17 September 2003.

Ellis, Stephen R.↗
NASA NTRS2013

Analysis of Traffic Conflicts in a Mixed-Airspace Evaluation of Airborne Separation Assurance

A pair of human-in-the-loop simulation evaluations of a distributed air/ground separation assurance system have been conducted to investigate the function allocation between humans and automation systems as well as ground-based and airborne agents in the Next Generation Air Transportation System and beyond. This paper focuses on an analysis of certain critical conflicts observed between self-separating aircraft and ground-managed traffic in the same airspace. The principal cause of each conflict is identified and potential mitigations are discussed, such as: the sharing of trajectory intent information between the ground and the air; more cautious trajectory planning by the self-separating aircraft; and more equitable rules-of-the-road between the self-separating aircraft and ground-managed aircraft. This analysis will inform the ongoing design of an airborne separation assurance automation tool.

Lewis, Timothy A.↗
DOE OSTI2020

Janus: A Python Package for Agent-Based Modeling of Land Use and Land Cover Change

Janus is an open source Python package for agent-based modeling (ABM) of land use and land cover change (LULCC). Many ABMs of LULCC have been created across platforms, some of which are not ideal for large scale, high resolution scenarios. This model provides a simple object-oriented framework for creating ABMs specific to LULCC. The organizational philosophy of the modeling framework is to create software objects (agents) that are associated with specific and contextual attributes which are isolated from where those agents exist in the spatial setting of the model, yet provide clear linkages between the agent, their environment, and other agents in the simulation. In this way, the framework allows for assembly of LULCC ABMs with low (programmatic) overhead, making the models extensible and providing clear mechanisms for integrating them with process-oriented biophysical models. Provided with Janus is a suite of geospatial data preprocessing tools that can use arbitrary land cover products as an input. Crop choice decisions are based on potential crop prices, these can be created synthetically, or drawn from integrated human-Earth systems models such as the GCAM. Janus is publicly accessible through GitHub and provides an example dataset for testing.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2017

Distributed Spatiotemporal Motion Planning for Spacecraft Swarms in Cluttered Environments

This paper focuses on trajectory planning for spacecraft swarms in cluttered environments, like debris fields or the asteroid belt. Our objective is to reconfigure the spacecraft swarm to a desired formation in a distributed manner while minimizing fuel and avoiding collisions among themselves and with obstacles. In our prior work we proposed a novel distributed guidance algorithm for spacecraft swarms in static environments. In this paper, we present the Multi-Agent Moving-Obstacles Spherical Expansion and Sequential Convex Programming (MAMO SE-SCP) algorithm that extends our prior work to include spatiotemporal constraints such as time-varying, moving obstacles and desired time-varying terminal positions. In the MAMO SE-SCP algorithm, each agent uses a spherical-expansion-based sampling algorithm to cooperatively explore the time-varying environment, a distributed assignment algorithm to agree on the terminal position for each agent, and a sequential-convex-programming-based optimization step to compute the locally-optimal trajectories from the current location to the assigned time-varying terminal position while avoiding collision with other agents and moving obstacles. Simulation results demonstrate that the proposed distributed algorithm can be used by a spacecraft swarm to achieve a time-varying, desired formation around an object of interest in a dynamic environment with many moving and tumbling obstacles.

Bandyopadhyay, Saptarshi↗
DOE OSTI2023

Convex Decreasing Algorithms: Distributed Synthesis and Finite-Time Termination in Higher Dimension

Here we establish finite time termination algorithms for consensus algorithms based on geometric properties that yield finite-time guarantees, suited for use in high dimension and in the absence of a central authority. These pursuits motivate a new peer to peer convex hull algorithm which is utilized for one stopping algorithm. Further an alternative lightweight norm based stopping criteria is also developed. The practical utility of the algorithm is illustrated through MATLAB simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Circular Economy Modeling Efforts at NREL: Circular Economy Lifecycle Analysis and Visualization (CELAVI) and the Circular Economy Agent-Based Model (ABM)

Circular Economy (CE) aims at decoupling human activities from economic growth and resource use. While CE economic and environmental benefits are often touted by its proponents, increased circularity and improved sustainability do not necessarily go hand-in-hand. The Circular Economy Lifecycle Assessment and Visualization (CELAVI) framework simulates the transition of renewable energy technology supply chains towards circularity and quantifies spatially explicit environmental impacts and supply chain costs. Using wind blade end-of-life (EOL) management as a case study, this presentation will highlight CELAVI's features. Furthermore, envisioned extension to the framework could improve the stakeholder decision model by accounting for logistical and other factors. The CELAVI framework could be leveraged to answer crucial questions regarding renewables EOL management.

circular economy↗
DOE OSTI2021

Impact of Blockchain Delay on Grid-Tied Solar Inverter Performance

This paper investigates the impact of the delay resulting from a blockchain, a promising security measure, for a hierarchical control system of inverters connected to the grid. The blockchain communication network is designed at the secondary control layer for resilience against cyberattacks. To represent the latency in the communication channel, a model is developed based on the complexity of the blockchain framework. Taking this model into account, this work evaluates the plant’s performance subject to communication delays, introduced by the blockchain, among the hierarchical control agents. In addition, this article considers an optimal model-based control strategy that performs the system’s internal control loop. The work shows that the blockchain’s delay size influences the convergence of the power supplied by the inverter to the reference at the point of common coupling. In the results section, real-time simulations on OPAL-RT are performed to test the resilience of two parallel inverters with increasing blockchain complexity.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2023

Reaction–Diffusion Coupling Facilitates the Sequential Precipitation of Metal Ions from Battery Feedstock Solutions

Here, the development of new technologies for chemical separations is urgently needed to meet the surging demand for critical materials that has strained resources and caused environmental challenges. Inspired by the classic Liesegang experiment, we demonstrated the separation of critical metal ions based on the coupling of ion diffusion and precipitation kinetics. For this purpose, a model feedstock solution simulating dissolved battery electrodes was placed on top of a hydrogel loaded with a precipitating agent, namely sodium hydroxide. As the lithium, manganese, cobalt, and nickel ions diffused into the gel, a gradient of precipitates formed along the length of the reactor. Elemental analysis of the spatially distributed precipitates showed the enrichment of nickel near the gel-solution interface, followed by the formation of an almost pure (>96%) manganese product further along the reactor. Optimization experiments revealed that a sodium hydroxide concentration of 10 mM and a gel/solution volume ratio of 2:1 favored efficient separations. The robustness of the method was demonstrated in four out of five feedstock compositions of typically used battery cathodes. Our proof-of-concept experiments present a paradigm for critical materials separations that does not require specialty chemicals, binding agents, membranes, or toxic solvents.

25 ENERGY STORAGE↗
NASA NTRS2017

Distributed Spatiotemporal Motion Planning for Spacecraft Swarms in Cluttered Environments

This paper focuses on trajectory planning for spacecraft swarms in cluttered environments, like debris fields or the asteroid belt. Our objective is to reconfigure the spacecraft swarm to a desired formation in a distributed manner while minimizing fuel and avoiding collisions among themselves and with the obstacles. In our prior work we proposed a novel distributed guidance algorithm for spacecraft swarms in static environments.1 In this paper, we present the Multi-Agent Moving-Obstacles Spherical Expansion and Sequential Convex Programming (MAMO SE–SCP) algorithm that extends our prior work to include spatiotemporal constraints such as time-varying, moving obstacles and desired time-varying terminal positions. In the MAMO SE–SCP algorithm, each agent uses a spherical-expansion-based sampling algorithm to cooperatively explore the time-varying environment, a distributed assignment algorithm to agree on the terminal position for each agent, and a sequential-convex-programming-based optimization step to compute the locally-optimal trajectories from the current location to the assigned time-varying terminal position while avoiding collision with other agent and the moving obstacles. Simulations results demonstrate that the proposed distributed algorithm can be used by a spacecraft swarm to achieve a time-varying, desired formation around an object of interest in a dynamic environment with many moving and tumbling obstacles.

Hadaegh, Fred Y.↗