SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Impact of Blockchain Delay on Grid-Tied Solar Inverter Performance

This paper investigates the impact of the delay resulting from a blockchain, a promising security measure, for a hierarchical control system of inverters connected to the grid. The blockchain communication network is designed at the secondary control layer for resilience against cyberattacks. To represent the latency in the communication channel, a model is developed based on the complexity of the blockchain framework. Taking this model into account, this work evaluates the plant’s performance subject to communication delays, introduced by the blockchain, among the hierarchical control agents. In addition, this article considers an optimal model-based control strategy that performs the system’s internal control loop. The work shows that the blockchain’s delay size influences the convergence of the power supplied by the inverter to the reference at the point of common coupling. In the results section, real-time simulations on OPAL-RT are performed to test the resilience of two parallel inverters with increasing blockchain complexity.

41 EE - Solar Energy Technologies Office (EE-4S)↗

DOE OSTI2023

Reaction–Diffusion Coupling Facilitates the Sequential Precipitation of Metal Ions from Battery Feedstock Solutions

Here, the development of new technologies for chemical separations is urgently needed to meet the surging demand for critical materials that has strained resources and caused environmental challenges. Inspired by the classic Liesegang experiment, we demonstrated the separation of critical metal ions based on the coupling of ion diffusion and precipitation kinetics. For this purpose, a model feedstock solution simulating dissolved battery electrodes was placed on top of a hydrogel loaded with a precipitating agent, namely sodium hydroxide. As the lithium, manganese, cobalt, and nickel ions diffused into the gel, a gradient of precipitates formed along the length of the reactor. Elemental analysis of the spatially distributed precipitates showed the enrichment of nickel near the gel-solution interface, followed by the formation of an almost pure (>96%) manganese product further along the reactor. Optimization experiments revealed that a sodium hydroxide concentration of 10 mM and a gel/solution volume ratio of 2:1 favored efficient separations. The robustness of the method was demonstrated in four out of five feedstock compositions of typically used battery cathodes. Our proof-of-concept experiments present a paradigm for critical materials separations that does not require specialty chemicals, binding agents, membranes, or toxic solvents.

25 ENERGY STORAGE↗

NASA NTRS2017

Distributed Spatiotemporal Motion Planning for Spacecraft Swarms in Cluttered Environments

This paper focuses on trajectory planning for spacecraft swarms in cluttered environments, like debris ﬁelds or the asteroid belt. Our objective is to reconﬁgure the spacecraft swarm to a desired formation in a distributed manner while minimizing fuel and avoiding collisions among themselves and with the obstacles. In our prior work we proposed a novel distributed guidance algorithm for spacecraft swarms in static environments.1 In this paper, we present the Multi-Agent Moving-Obstacles Spherical Expansion and Sequential Convex Programming (MAMO SE–SCP) algorithm that extends our prior work to include spatiotemporal constraints such as time-varying, moving obstacles and desired time-varying terminal positions. In the MAMO SE–SCP algorithm, each agent uses a spherical-expansion-based sampling algorithm to cooperatively explore the time-varying environment, a distributed assignment algorithm to agree on the terminal position for each agent, and a sequential-convex-programming-based optimization step to compute the locally-optimal trajectories from the current location to the assigned time-varying terminal position while avoiding collision with other agent and the moving obstacles. Simulations results demonstrate that the proposed distributed algorithm can be used by a spacecraft swarm to achieve a time-varying, desired formation around an object of interest in a dynamic environment with many moving and tumbling obstacles.

Hadaegh, Fred Y.↗

NASA NTRSDate not supplied

Dynamic Spectrum Allocation in Urban Air Transportation System via Deep Reinforcement Learning

The emerging concepts of Urban Air Mobility (UAM) and Advanced Air Mobility (AAM) open a new paradigm for urban air transportation. A big challenge is that these new aerial vehicles will quickly saturate the already crowded aviation spectrum, which is an essential resource to ensure reliable communications for safe operations. In this paper, we consider an air transportation system where multiple aerial vehicles are operated to transport passengers or cargo from different sources to destinations along their pre-defined paths. During the flight, the minimum communication Quality of Service (QoS) requirement must be achieved to ensure flight safety. Our objective is to minimize the average mission completion time by jointly optimizing the velocity selection and spectrum allocation for all aerial vehicles. We formulate the optimization problem as a multi-stage Markov Decision Process (MDP) where the optimization variables are coupled together. A multi-agent Deep Reinforcement Learning (DRL) based solution is proposed where Value Decomposition Networks (VDN) algorithm is utilized to take discrete actions. Additionally, we propose a heuristic greedy algorithm as a baseline solution. Simulation results show that our learning based solution outperforms the heuristic greedy algorithm and another Orthogonal Multiple Access (OMA) solution in minimizing the mission completion time.

Ruixuan Han↗

NASA NTRSDate not supplied

Dynamic Spectrum Allocation in Urban Air Transportation System via Deep Reinforcement Learning

Ruixuan Han↗

NASA NTRS2022

Contribution of Spicules to Solar Coronal Emission

Recent high-resolution imaging and spectroscopic observations have generated renewed interest in spicules' role in explaining the hot corona. Some studies suggest that some spicules, often classified as type II, may provide significant mass and energy to the corona. Here we use numerical simulations to investigate whether such spicules can produce the observed coronal emission without any additional coronal heating agent. Model spicules consisting of a cold body and hot tip are injected into the base of a warm (0.5 MK) equilibrium loop with different tip temperatures and injection velocities. Both piston- and pressure-driven shocks are produced. We find that the hot tip cools rapidly and disappears from coronal emission lines such as Fe xii 195 and Fe xiv 274. Prolonged hot emission is produced by preexisting loop material heated by the shock and by thermal conduction from the shock. However, the shapes and Doppler shifts of synthetic line profiles show significant discrepancies with observations. Furthermore, spatially and temporally averaged intensities are extremely low, suggesting that if the observed intensities from the quiet Sun and active regions were solely due to type II spicules, one to several orders of magnitude more spicules would be required than have been reported in the literature. This conclusion applies strictly to the ejected spicular material. We make no claims about emissions connected with waves or coronal currents that may be generated during the ejection process and heat the surrounding area.

Shanwlee Sow Mondal↗

DOE OSTI2022

Physics Informed Reinforcement Learning for Power Grid Control using Augmented Random Search

Wide adoption of deep reinforcement learning need to overcome several challenges in energy system domain, including scalability, learning from limited samples, and high-dimensional continuous state and action spaces. In this paper, we integrated physics-based information from the normal generator operation state formula in the reinforcement learning agent's neural network loss function, and applied an augmented random search agent to optimize the generator control under dynamic contingency. Simulation results demonstrated the reliability performance improvements in training speed, reward convergence, sampling efficiency, scalability, and transferability.

physics informed ML, Physics Informed Neural Netwo↗

DOE OSTI2021

A population data-driven workflow for COVID-19 modeling and learning

CityCOVID is a detailed agent-based model that represents the behaviors and social interactions of 2.7 million residents of Chicago as they move between and colocate in 1.2 million distinct places, including households, schools, workplaces, and hospitals, as determined by individual hourly activity schedules and dynamic behaviors such as isolating because of symptom onset. Disease progression dynamics incorporated within each agent track transitions between possible COVID-19 disease states, based on heterogeneous agent attributes, exposure through colocation, and effects of protective behaviors of individuals on viral transmissibility. Throughout the COVID-19 epidemic, CityCOVID model outputs have been provided to city, county, and state stakeholders in response to evolving decision-making priorities, while incorporating emerging information on SARS-CoV-2 epidemiology. Here we demonstrate our efforts in integrating our high-performance epidemiological simulation model with large-scale machine learning to develop a generalizable, flexible, and performant analytical platform for planning and crisis response.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Sorption and Desorption Study of a Nerve-Agent Simulant from Office Materials for Forensic Applications

Dimethyl methylphosphonate (DMMP) was used as a simulant to investigate the sorption and desorption of G-series nerve agents such as sarin (GB) released into an office space. DMMP was disseminated systematically as an aerosol and vapor on three common office materials, cardboard, polyurethane foam, and cured acrylic-based paint. The amount of DMMP initially captured on the office materials and the amount retained over a period of 10 hours was tracked gravimetrically and by solvent extraction and gas chromatography/mass spectrometry (GC/MS). Physicochemical properties of the office media including the specific surface area per mass, polarity, and Henry’s constants were determined to help explain the capture and release of DMMP that was observed. Polyurethane foam was found to have the largest sorption capacity for DMMP in a low-humidity setting, however, cardboard was found to have the largest sorption capacity at higher humidity. A fraction of the DMMP collected desorbed with first-order kinetics from all of the office materials in an open-air atmosphere over 8 hours. However, importantly for forensic purposes relevant to an indoor attack, a significant quantity (most notably on cardboard) of the nerve agent simulant was retained after 6 weeks and could be analyzed efficiently using solvent extraction followed by GC/MS.

chemical forensics↗

DOE OSTI2023

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2010

Route Generation for a Synthetic Character (BOT) Using a Partial or Incomplete Knowledge Route Generation Algorithm in UT2004 Virtual Environment

This paper presents a new Route Generation Algorithm that accurately and realistically represents human route planning and navigation for Military Operations in Urban Terrain (MOUT). The accuracy of this algorithm in representing human behavior is measured using the Unreal Tournament(Trademark) 2004 (UT2004) Game Engine to provide the simulation environment in which the differences between the routes taken by the human player and those of a Synthetic Agent (BOT) executing the A-star algorithm and the new Route Generation Algorithm can be compared. The new Route Generation Algorithm computes the BOT route based on partial or incomplete knowledge received from the UT2004 game engine during game play. To allow BOT navigation to occur continuously throughout the game play with incomplete knowledge of the terrain, a spatial network model of the UT2004 MOUT terrain is captured and stored in an Oracle 11 9 Spatial Data Object (SOO). The SOO allows a partial data query to be executed to generate continuous route updates based on the terrain knowledge, and stored dynamic BOT, Player and environmental parameters returned by the query. The partial data query permits the dynamic adjustment of the planned routes by the Route Generation Algorithm based on the current state of the environment during a simulation. The dynamic nature of this algorithm more accurately allows the BOT to mimic the routes taken by the human executing under the same conditions thereby improving the realism of the BOT in a MOUT simulation environment.

Hanold, Gregg T.↗

DOE OSTI2023

A subcellular biochemical model for T6SS dynamics reveals winning competitive strategies

Abstract The type VI secretion system (T6SS) is a broadly distributed interbacterial weapon that can be used to eliminate competing bacterial populations. Although unarmed target populations are typically used to study T6SS function in vitro, bacteria most likely encounter other T6SS-armed competitors in nature. However, the connection between subcellular details of the T6SS and the outcomes of such mutually lethal battles is not well understood. Here, we incorporate biological data derived from natural competitors of Vibrio fischeri light organ symbionts to build a biochemical model for T6SS at the single-cell level, which we then integrate into an agent-based model (ABM). Using the ABM, we isolate and experiment with strain-specific physiological differences between competitors in ways not possible with biological samples to identify winning strategies for T6SS-armed populations. Through in vitro experiments, we discover that strain-specific differences exist in T6SS activation speed. ABM simulations corroborate that faster activation is dominant in determining survival during competition. Once competitors are fully activated, the energy required for T6SS creates a tipping point where increased weapon building and firing becomes too costly to be advantageous. Through ABM simulations, we identify the threshold where this transition occurs in the T6SS parameter space. We also find that competitive outcomes depend on the geometry of the battlefield: unarmed target cells survive at the edges of a range expansion where unlimited territory can be claimed. Alternatively, competitions within a confined space, much like the light organ crypts where natural V. fischeri compete, result in the rapid elimination of the unarmed population.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Optimized purification methods for metallic contaminant removal from directly recycled Li-ion battery cathodes

Metallic contaminants pose a significant challenge to the viability of directly recycling Li-ion batteries. To date, few strategies exist to selectively remove metallic impurities from mixtures of shredded end-of-life material (black mass; BM) without concurrently damaging the structure and electrochemical performance of the target active material. We herein present tailored methods to selectively ionize two major contaminants—Al and Cu—while retaining a representative cathode (LiNi 0.33 Mn 0.33 Co 0.33 O 2 ; NMC-111) intact. This BM purification process is conducted at moderate temperatures in a KOH-based solution matrix. We rationally evaluate approaches to increase both the kinetic corrosion rate and the thermodynamic solubility of Al 0 and Cu 0 , and evaluate the impact of these treatment conditions on the structure, chemistry, and electrochemical performance of NMC. Specifically, we explore the impacts of chloride-based salts, a strong chelating agent, elevated temperature, and sonication on the rate and extent of contaminant corrosion, while concurrently evaluating the effects on NMC. The reported BM purification process is then demonstrated on samples of “simulated BM” containing a practically relevant 1 wt% concentration of Al or Cu. Increasing the kinetic energy of the purifying solution matrix through elevated temperature and sonication accelerates the corrosion of metallic Al and Cu, such that ∼100% corrosion of 75 μm Al and Cu particles is achieved within 2.5 hr. Further, we determine that effective mass transport of ionized species critically impacts the efficacy of Cu corrosion, and that saturated Cl – hinders rather than accelerates Cu corrosion by increasing solution viscosity and introducing competitive pathways for Cu surface passivation. The purification conditions do not induce bulk structural damage to NMC, and electrochemical capacity is maintained in half-cell format. Testing in full cells suggests that a limited quantity of residual surface species are present after treatment, which initially disrupt electrochemical behavior at the graphite anode but are subsequently consumed. Process demonstration on simulated BM suggests that contaminated samples—which prior to treatment show catastrophic electrochemical performance—can be recovered to pristine electrochemical capacity. The reported BM purification method offers a compelling and commercially viable solution to address contamination, particularly in the “fine” fraction of BM where contaminant sizes are on the same order of magnitude as NMC and where traditional separation approaches are unfeasible. Thus, this optimized BM purification technique offers a pathway towards viable direct recycling of BM feedstocks that would otherwise be unusable.

25 ENERGY STORAGE↗

NASA NTRSDate not supplied

Bootstrapping Multi-Agent Unmanned Aerial Vehicle (UAV) System Integration Using Ground-Based Assets: Lessons Learned

The highly dynamic nature of UAVs imposes significant challenges when conducting initial testing ranging from safety risks posed by high-capacity lithium batteries and spinning propellers to rigorous timing demands on controllers and the consequences of failures mid-air. Flight testing of a single vehicle is time and labor intensive due to these challenges and more, and the complexity increases exponentially with the number of vehicles. While simulations and hardware-in-the-loop bench testing can provide adequate environments for preliminary validation, differences in system deployment architecture, software interfaces, and hardware infrastructure between simulation and a fleet of real UAVs create a sizable gap that must be navigated carefully during system integration. In support of the Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project, which had the goal of establishing a basis of certification of trust and trustworthiness in multi-agent autonomous systems, this gap was tackled from two directions. First, a novel mixed-reality simulation environment was engineered to blur the transition from simulation to flight hardware. Second, a fleet of Unmanned Surface Vehicles (USVs) was developed as a test and evaluation platform that more closely represented the final aerial fleet while eliminating many of the risks associated with air vehicles. This paper delves into the second element, analyzing the efficacy of the USV platform in performing system integration testing for the UAV system. In this paper we present the USV fleet and its role in reducing the aforementioned gaps in deployment architecture, software interfaces, and hardware infrastructure when moving from simulation to flight. An overview of the hardware and software onboard the vehicles will be provided along with supporting infrastructure. The system integration process will be documented including results in supporting both the overarching design reference mission (DRM) of ATTRACTOR and individual research efforts conducted during the project. Finally, we will discuss some of the practical lessons learned regarding the testing, deployment, and operation of multi-agent autonomous systems.

Matthew P Vaughan↗

NASA NTRS1998

Integration of the Remote Agent for the NASA Deep Space One Autonomy Experiment

This paper describes the integration of the Remote Agent (RA), a spacecraft autonomy system which is scheduled to control the Deep Space 1 spacecraft during a flight experiment in 1999. The RA is a reusable, model-based autonomy system that is quite different from software typically used to control an aerospace system. We describe the integration challenges we faced, how we addressed them, and the lessons learned. We focus on those aspects of integrating the RA that were either easier or more difficult than integrating a more traditional large software application because the RA is a model-based autonomous system. A number of characteristics of the RA made integration process easier. One example is the model-based nature of RA. Since the RA is model-based, most of its behavior is not hard coded into procedural program code. Instead, engineers specify high level models of the spacecraft's components from which the Remote Agent automatically derives correct system-wide behavior on the fly. This high level, modular, and declarative software description allowed some interfaces between RA components and between RA and the flight software to be automatically generated and tested for completeness against the Remote Agent's models. In addition, the Remote Agent's model-based diagnosis system automatically diagnoses when the RA models are not consistent with the behavior of the spacecraft. In flight, this feature is used to diagnose failures in the spacecraft hardware. During integration, it proved valuable in finding problems in the spacecraft simulator or flight software. In addition, when modifications are made to the spacecraft hardware or flight software, the RA models are easily changed because they only capture a description of the spacecraft. one does not have to maintain procedural code that implements the correct behavior for every expected situation. On the other hand, several features of the RA made it more difficult to integrate than typical flight software. For example, the definition of correct behavior is more difficult to specify for a system that is expected to reason about and flexibly react to its environment than for a traditional flight software system. Consequently, whenever a change is made to the RA it is more time consuming to determine if the resulting behavior is correct. We conclude the paper with a discussion of future work on the Remote Agent as well as recommendations to ease integration of similar autonomy projects.

Dorais, Gregory A.↗

DOE OSTI2025

Holistic Microstructure Control Strategies in Photopolymerization‐Induced Phase Separation of Acrylate Systems

Open porous materials, known for their large surface area and interconnected structures, are essential in various applications, including batteries, ion exchange, catalysis, filtration, and electronic waste recycling. A critical aspect of the functionality of porous membranes is the precise control of pore size and morphology. Photopolymerization-induced phase separation (photo-PIPS) offers a convenient and versatile methods for creating porous structures. However, controlling the porous morphology remains challenging due to the complex interplay between thermodynamics, polymerization kinetics, and monomer structures, which makes it difficult to establish the relationship between processing conditions and resulting morphology in photo-PIPS. Herein, a physics-based phase-field model capable of generating and characterizing the microstructures of porous materials based on both average and localized features is developed. Using the phase-field simulations as test bed, the effects of polarity, light intensity, and curing temperature, as well as the previously unexplored roles of chain transfer agents and substrates, on the morphology of the resulting porous microstructure are investigated. Experiments are performed to verify the results predicted by the simulations. This work lays out a comprehensive guide for designing PIPS-derived porous microstructures and offers practical engineering strategies for tailoring microstructure-level topology and size of pores for application-specific needs.

36 MATERIALS SCIENCE↗

DOE OSTI2024

Structure Development in Cross-Linked, Soybean Oil-based Waterborne Polyurethanes

Development of waterborne polyurethanes (WPU) using bio-based sources represents a step towards sustainable materials science and industry. We synthesized bio-based cationic water-dispersed crosslinked polyurethanes from high oleic soybean oil (HOSO) polyol, isophorone diisocyanate, and methyldiethanol amine, with varying ionic group contents after neutralization with acetic acid. Our primary objective was to analyze how crosslinking affects the dispersion process and film properties in multifunctional systems. The synthesis-structure-property relationship is elucidated through comprehensive analyses of the products at different stages of the synthesis. The dispersion of the WPU particles in water must occur prior to gelation during the final preparation, leading to incomplete conversion and the formation of imperfect networks. Insight into the synthesis process and polymer structure was gained by simulating polymer network parameters. Morphological analyses using synchrotron-based X-ray scattering and atomic force microscopy revealed a hierarchical structure within the WPU films. Importantly, all the films prepared in this study, without using coalescence agents, have low water absorption and high water contact angles, demonstrating their potential for textile and leather coatings and other applications.

bio-based polymers↗

DOE OSTI2025

Virtual Agents-Based Attack-Resilient Distributed Control for Islanded AC Microgrid

Due to its dependence on a communication network, distributed secondary control of microgrids is susceptible to denial-of-service (DoS) attacks in channel shutdown mode, which may negatively impact the network connectivity and thus deteriorate the coordination and power sharing among distributed generators (DGs). Honeypot is a common method for cyber deception by introducing fake targets. However, in the context of microgrid, the misleading information spread by honeypots will also impact the system performance. This paper proposes an attack-resilient distributed control for AC microgrids utilizing virtual agents (VAs) to counteract both DoS edge and node attacks. The VAs are designed to not impact the system’s steady state during normal operation but to share information among neighboring real agents and serve as dummy targets for DoS attacks. The control with VAs is implemented by a primal-dual gradient based distributed algorithm to efficiently obtain a practical solution for voltage/frequency regulation and power sharing. The simulation results on a 4-DG test system and a modified IEEE 34-bus system show that 1) VAs do not impact the normal functionality of the test system, and 2) deploying VAs can enhance the resilience of the microgrid control against DoS edge and node attacks.

24 POWER TRANSMISSION AND DISTRIBUTION↗