Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Impact of Blockchain Delay on Grid-Tied Solar Inverter Performance

This paper investigates the impact of the delay resulting from a blockchain, a promising security measure, for a hierarchical control system of inverters connected to the grid. The blockchain communication network is designed at the secondary control layer for resilience against cyberattacks. To represent the latency in the communication channel, a model is developed based on the complexity of the blockchain framework. Taking this model into account, this work evaluates the plant’s performance subject to communication delays, introduced by the blockchain, among the hierarchical control agents. In addition, this article considers an optimal model-based control strategy that performs the system’s internal control loop. The work shows that the blockchain’s delay size influences the convergence of the power supplied by the inverter to the reference at the point of common coupling. In the results section, real-time simulations on OPAL-RT are performed to test the resilience of two parallel inverters with increasing blockchain complexity.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2023

Reaction–Diffusion Coupling Facilitates the Sequential Precipitation of Metal Ions from Battery Feedstock Solutions

Here, the development of new technologies for chemical separations is urgently needed to meet the surging demand for critical materials that has strained resources and caused environmental challenges. Inspired by the classic Liesegang experiment, we demonstrated the separation of critical metal ions based on the coupling of ion diffusion and precipitation kinetics. For this purpose, a model feedstock solution simulating dissolved battery electrodes was placed on top of a hydrogel loaded with a precipitating agent, namely sodium hydroxide. As the lithium, manganese, cobalt, and nickel ions diffused into the gel, a gradient of precipitates formed along the length of the reactor. Elemental analysis of the spatially distributed precipitates showed the enrichment of nickel near the gel-solution interface, followed by the formation of an almost pure (>96%) manganese product further along the reactor. Optimization experiments revealed that a sodium hydroxide concentration of 10 mM and a gel/solution volume ratio of 2:1 favored efficient separations. The robustness of the method was demonstrated in four out of five feedstock compositions of typically used battery cathodes. Our proof-of-concept experiments present a paradigm for critical materials separations that does not require specialty chemicals, binding agents, membranes, or toxic solvents.

25 ENERGY STORAGE↗
DOE OSTI2022

Physics Informed Reinforcement Learning for Power Grid Control using Augmented Random Search

Wide adoption of deep reinforcement learning need to overcome several challenges in energy system domain, including scalability, learning from limited samples, and high-dimensional continuous state and action spaces. In this paper, we integrated physics-based information from the normal generator operation state formula in the reinforcement learning agent's neural network loss function, and applied an augmented random search agent to optimize the generator control under dynamic contingency. Simulation results demonstrated the reliability performance improvements in training speed, reward convergence, sampling efficiency, scalability, and transferability.

physics informed ML, Physics Informed Neural Netwo↗
DOE OSTI2021

A population data-driven workflow for COVID-19 modeling and learning

CityCOVID is a detailed agent-based model that represents the behaviors and social interactions of 2.7 million residents of Chicago as they move between and colocate in 1.2 million distinct places, including households, schools, workplaces, and hospitals, as determined by individual hourly activity schedules and dynamic behaviors such as isolating because of symptom onset. Disease progression dynamics incorporated within each agent track transitions between possible COVID-19 disease states, based on heterogeneous agent attributes, exposure through colocation, and effects of protective behaviors of individuals on viral transmissibility. Throughout the COVID-19 epidemic, CityCOVID model outputs have been provided to city, county, and state stakeholders in response to evolving decision-making priorities, while incorporating emerging information on SARS-CoV-2 epidemiology. Here we demonstrate our efforts in integrating our high-performance epidemiological simulation model with large-scale machine learning to develop a generalizable, flexible, and performant analytical platform for planning and crisis response.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Sorption and Desorption Study of a Nerve-Agent Simulant from Office Materials for Forensic Applications

Dimethyl methylphosphonate (DMMP) was used as a simulant to investigate the sorption and desorption of G-series nerve agents such as sarin (GB) released into an office space. DMMP was disseminated systematically as an aerosol and vapor on three common office materials, cardboard, polyurethane foam, and cured acrylic-based paint. The amount of DMMP initially captured on the office materials and the amount retained over a period of 10 hours was tracked gravimetrically and by solvent extraction and gas chromatography/mass spectrometry (GC/MS). Physicochemical properties of the office media including the specific surface area per mass, polarity, and Henry’s constants were determined to help explain the capture and release of DMMP that was observed. Polyurethane foam was found to have the largest sorption capacity for DMMP in a low-humidity setting, however, cardboard was found to have the largest sorption capacity at higher humidity. A fraction of the DMMP collected desorbed with first-order kinetics from all of the office materials in an open-air atmosphere over 8 hours. However, importantly for forensic purposes relevant to an indoor attack, a significant quantity (most notably on cardboard) of the nerve agent simulant was retained after 6 weeks and could be analyzed efficiently using solvent extraction followed by GC/MS.

chemical forensics↗
DOE OSTI2023

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

A subcellular biochemical model for T6SS dynamics reveals winning competitive strategies

Abstract The type VI secretion system (T6SS) is a broadly distributed interbacterial weapon that can be used to eliminate competing bacterial populations. Although unarmed target populations are typically used to study T6SS function in vitro, bacteria most likely encounter other T6SS-armed competitors in nature. However, the connection between subcellular details of the T6SS and the outcomes of such mutually lethal battles is not well understood. Here, we incorporate biological data derived from natural competitors of Vibrio fischeri light organ symbionts to build a biochemical model for T6SS at the single-cell level, which we then integrate into an agent-based model (ABM). Using the ABM, we isolate and experiment with strain-specific physiological differences between competitors in ways not possible with biological samples to identify winning strategies for T6SS-armed populations. Through in vitro experiments, we discover that strain-specific differences exist in T6SS activation speed. ABM simulations corroborate that faster activation is dominant in determining survival during competition. Once competitors are fully activated, the energy required for T6SS creates a tipping point where increased weapon building and firing becomes too costly to be advantageous. Through ABM simulations, we identify the threshold where this transition occurs in the T6SS parameter space. We also find that competitive outcomes depend on the geometry of the battlefield: unarmed target cells survive at the edges of a range expansion where unlimited territory can be claimed. Alternatively, competitions within a confined space, much like the light organ crypts where natural V. fischeri compete, result in the rapid elimination of the unarmed population.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Optimized purification methods for metallic contaminant removal from directly recycled Li-ion battery cathodes

Metallic contaminants pose a significant challenge to the viability of directly recycling Li-ion batteries. To date, few strategies exist to selectively remove metallic impurities from mixtures of shredded end-of-life material (black mass; BM) without concurrently damaging the structure and electrochemical performance of the target active material. We herein present tailored methods to selectively ionize two major contaminants—Al and Cu—while retaining a representative cathode (LiNi 0.33 Mn 0.33 Co 0.33 O 2 ; NMC-111) intact. This BM purification process is conducted at moderate temperatures in a KOH-based solution matrix. We rationally evaluate approaches to increase both the kinetic corrosion rate and the thermodynamic solubility of Al 0 and Cu 0 , and evaluate the impact of these treatment conditions on the structure, chemistry, and electrochemical performance of NMC. Specifically, we explore the impacts of chloride-based salts, a strong chelating agent, elevated temperature, and sonication on the rate and extent of contaminant corrosion, while concurrently evaluating the effects on NMC. The reported BM purification process is then demonstrated on samples of “simulated BM” containing a practically relevant 1 wt% concentration of Al or Cu. Increasing the kinetic energy of the purifying solution matrix through elevated temperature and sonication accelerates the corrosion of metallic Al and Cu, such that ∼100% corrosion of 75 μm Al and Cu particles is achieved within 2.5 hr. Further, we determine that effective mass transport of ionized species critically impacts the efficacy of Cu corrosion, and that saturated Cl – hinders rather than accelerates Cu corrosion by increasing solution viscosity and introducing competitive pathways for Cu surface passivation. The purification conditions do not induce bulk structural damage to NMC, and electrochemical capacity is maintained in half-cell format. Testing in full cells suggests that a limited quantity of residual surface species are present after treatment, which initially disrupt electrochemical behavior at the graphite anode but are subsequently consumed. Process demonstration on simulated BM suggests that contaminated samples—which prior to treatment show catastrophic electrochemical performance—can be recovered to pristine electrochemical capacity. The reported BM purification method offers a compelling and commercially viable solution to address contamination, particularly in the “fine” fraction of BM where contaminant sizes are on the same order of magnitude as NMC and where traditional separation approaches are unfeasible. Thus, this optimized BM purification technique offers a pathway towards viable direct recycling of BM feedstocks that would otherwise be unusable.

25 ENERGY STORAGE↗
DOE OSTI2025

Holistic Microstructure Control Strategies in Photopolymerization‐Induced Phase Separation of Acrylate Systems

Open porous materials, known for their large surface area and interconnected structures, are essential in various applications, including batteries, ion exchange, catalysis, filtration, and electronic waste recycling. A critical aspect of the functionality of porous membranes is the precise control of pore size and morphology. Photopolymerization-induced phase separation (photo-PIPS) offers a convenient and versatile methods for creating porous structures. However, controlling the porous morphology remains challenging due to the complex interplay between thermodynamics, polymerization kinetics, and monomer structures, which makes it difficult to establish the relationship between processing conditions and resulting morphology in photo-PIPS. Herein, a physics-based phase-field model capable of generating and characterizing the microstructures of porous materials based on both average and localized features is developed. Using the phase-field simulations as test bed, the effects of polarity, light intensity, and curing temperature, as well as the previously unexplored roles of chain transfer agents and substrates, on the morphology of the resulting porous microstructure are investigated. Experiments are performed to verify the results predicted by the simulations. This work lays out a comprehensive guide for designing PIPS-derived porous microstructures and offers practical engineering strategies for tailoring microstructure-level topology and size of pores for application-specific needs.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Structure Development in Cross-Linked, Soybean Oil-based Waterborne Polyurethanes

Development of waterborne polyurethanes (WPU) using bio-based sources represents a step towards sustainable materials science and industry. We synthesized bio-based cationic water-dispersed crosslinked polyurethanes from high oleic soybean oil (HOSO) polyol, isophorone diisocyanate, and methyldiethanol amine, with varying ionic group contents after neutralization with acetic acid. Our primary objective was to analyze how crosslinking affects the dispersion process and film properties in multifunctional systems. The synthesis-structure-property relationship is elucidated through comprehensive analyses of the products at different stages of the synthesis. The dispersion of the WPU particles in water must occur prior to gelation during the final preparation, leading to incomplete conversion and the formation of imperfect networks. Insight into the synthesis process and polymer structure was gained by simulating polymer network parameters. Morphological analyses using synchrotron-based X-ray scattering and atomic force microscopy revealed a hierarchical structure within the WPU films. Importantly, all the films prepared in this study, without using coalescence agents, have low water absorption and high water contact angles, demonstrating their potential for textile and leather coatings and other applications.

bio-based polymers↗
DOE OSTI2025

Virtual Agents-Based Attack-Resilient Distributed Control for Islanded AC Microgrid

Due to its dependence on a communication network, distributed secondary control of microgrids is susceptible to denial-of-service (DoS) attacks in channel shutdown mode, which may negatively impact the network connectivity and thus deteriorate the coordination and power sharing among distributed generators (DGs). Honeypot is a common method for cyber deception by introducing fake targets. However, in the context of microgrid, the misleading information spread by honeypots will also impact the system performance. This paper proposes an attack-resilient distributed control for AC microgrids utilizing virtual agents (VAs) to counteract both DoS edge and node attacks. The VAs are designed to not impact the system’s steady state during normal operation but to share information among neighboring real agents and serve as dummy targets for DoS attacks. The control with VAs is implemented by a primal-dual gradient based distributed algorithm to efficiently obtain a practical solution for voltage/frequency regulation and power sharing. The simulation results on a 4-DG test system and a modified IEEE 34-bus system show that 1) VAs do not impact the normal functionality of the test system, and 2) deploying VAs can enhance the resilience of the microgrid control against DoS edge and node attacks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Electronic Coherences in Molecules: The Projected Nuclear Quantum Momentum as a Hidden Agent

Electronic coherences are key to understanding and controlling photoinduced molecular transformations. Here, we identify a crucial quantum-mechanical feature of electron-nuclear correlation, the projected nuclear quantum momenta, essential to capture the correct coherence behavior. For simulations, we show that, unlike traditional trajectory-based schemes, exact-factorization-based methods approximate these correlation terms and correctly capture electronic coherences in a range of situations, including their spatial dependence, an important aspect that influences subsequent electron dynamics and that is becoming accessible in more experiments.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2025

Integrated Routing and Traffic Signal Control for CAVs via Reinforcement Learning Approach

Incorporating Connected and Automated Vehicles (CAVs) into urban traffic networks presents opportunities and challenges for traffic management systems. This paper aims to develop an integrated routing and traffic signal control system designed explicitly for CAVs, utilizing a Reinforcement Learning (RL) approach. The objective is to enhance traffic flow and improve overall transportation efficiency in the controlled areas. We propose an innovative framework that employs the Deep Reinforcement Learning (DRL) algorithm, especially the Deep Q-network (DQN), to dynamically adjust the number of vehicles in the routes and the duration of traffic signals. Our simulation results demonstrate that a DQN agent successfully optimizes the number of vehicles in the routes and traffic signal timings of traffic signal controllers, eventually reducing total travel time. The study illustrates the potential usage of RL-based systems in managing routing and traffic signals for CAVs, offering a promising opportunity for future urban traffic management strategies.

Park, Jiho [New York University]↗
DOE OSTI2022

Reinforcement Learning for Volt- Var Control: A Novel Two-stage Progressive Training Strategy

This paper develops a reinforcement learning (RL) approach to solve a cooperative, multi-agent Volt-Var Control (VVC) problem for high solar penetration distribution systems. The ingenuity of our RL method lies in a novel two-stage progressive training strategy that can effectively improve training speed and convergence of the machine learning algorithm. In Stage 1 (individual training), while holding all the other agents inactive, we separately train each agent to obtain its own optimal VVC actions in the action space: fconsume, generate, do-nothingg. In Stage 2 (cooperative training), all agents are trained again coordinatively to share VVC responsibility. Rewards and costs in our RL scheme include (i) a system-level reward (for taking an action), (ii) an agent-level reward (for doing-nothing), and (iii) an agent-level action cost function. This new framework allows rewards to be dynamically allocated to each agent based on their contribution while accounting for the trade-off between control effectiveness and action cost. The proposed methodology is tested and validated in a modified IEEE 123-bus system using realistic PV and load profiles. Simulation results confirm that the proposed approach is robust and computationally efficient; and it achieves desirable volt-var control performance under a wide range of operation conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Load Shedding for Voltage Regulation With Probabilistic Agent Compliance

With the increased observability and controllability of distribution systems, the share of behind-the-meter systems is trending upwards rapidly. As a consequence, the impact of human behaviors on system performance can no longer be ignored and should be reflected in the energy management system models. In this paper, we discuss the problem of distribution system voltage control by active power curtailment where the agent compliance of the load curtailment signal is probabilistic. We discuss the modeling of the optimal voltage control problem with probabilistic agent compliance as a chance-constrained optimization problem, its tractable safe approximation using convex restriction, and a scenario-based mixed-integer reformulation as well as the associated solution method based on augmented Lagrangian method. The numerical simulation on IEEE test system validates the effectiveness of the proposed approach in obtaining high-quality feasible load curtailment signal with low computational cost, which makes it a viable tool for real time decision making.

augmented Lagrangian method↗
DOE OSTI2026

From Rules to Reasoning: A Survey of Large Language Model-Based Approaches to Scientific Hypothesis and Idea Generation

Scientific hypothesis generation represents a fundamental challenge in contemporary research due to exponentially expanding literature volumes and increasing disciplinary specialization. Large language models (LLMs) have emerged as transformative tools for automated scientific discovery, moving beyond traditional rule-based and literature-mining approaches. Four paradigmatic approaches define current LLM-driven hypothesis generation: direct prompting and fine-tuning methods, knowledge-enhanced frameworks integrating retrieval-augmented generation (RAG), multi-agent collaborative systems simulating research teams, and reasoning-focused approaches implementing cognitive architectures. Domain-specific applications demonstrate statistical equivalence to human expert performance in social psychology, experimental validation in biomedical research, and near-expert quality in astronomy. Evaluation methodologies encompass human expert assessment, LLM-as-judge frameworks, and comprehensive benchmarking systems. Technical challenges include hallucination management, knowledge integration limitations, and balancing novelty with feasibility. Future directions emphasize hybrid neural-symbolic architectures and sophisticated human-AI collaboration models for responsible scientific discovery acceleration.

AI-driven discovery↗
DOE OSTI2021

Building and experimenting with an agent-based model to study the population-level impact of CommunityRx, a clinic-based community resource referral intervention

CommunityRx (CRx), an information technology intervention, provides patients with a personalized list of healthful community resources (HealtheRx). In repeated clinical studies, nearly half of those who received clinical “doses” of the HealtheRx shared their information with others (“social doses”). Clinical trial design cannot fully capture the impact of information diffusion, which can act as a force multiplier for the intervention. Furthermore, experimentation is needed to understand how intervention delivery can optimize social spread under varying circumstances. To study information diffusion from CRx under varying conditions, we built an agent-based model (ABM). This study describes the model building process and illustrates how an ABM provides insight about information diffusion through in silico experimentation. To build the ABM, we constructed a synthetic population (“agents”) using publicly-available data sources. Using clinical trial data, we developed empirically-informed processes simulating agent activities, resource knowledge evolution and information sharing. Using RepastHPC and chiSIM software, we replicated the intervention in silico, simulated information diffusion processes, and generated emergent information diffusion networks. The CRx ABM was calibrated using empirical data to replicate the CRx intervention in silico. We used the ABM to quantify information spread via social versus clinical dosing then conducted information diffusion experiments, comparing the social dosing effect of the intervention when delivered by physicians, nurses or clinical clerks. The synthetic population (N = 802,191) exhibited diverse behavioral characteristics, including activity and knowledge evolution patterns. In silico delivery of the intervention was replicated with high fidelity. Large-scale information diffusion networks emerged among agents exchanging resource information. Varying the propensity for information exchange resulted in networks with different topological characteristics. Community resource information spread via social dosing was nearly 4 fold that from clinical dosing alone and did not vary by delivery mode. This study, using CRx as an example, demonstrates the process of building and experimenting with an ABM to study information diffusion from, and the population-level impact of, a clinical information-based intervention. While the focus of the CRx ABM is to recreate the CRx intervention in silico, the general process of model building, and computational experimentation presented is generalizable to other large-scale ABMs of information diffusion.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Large-Eddy Simulation of Flow Over Boeing Gaussian Bump Using Multiagent Reinforcement Learning Wall Model: Preprint

We develop a wall model for large-eddy simulation (LES) that takes into account various pressure-gradient effects using multi-agent reinforcement learning. The model is trained using low-Reynolds-number flow over periodic hills with agents distributed on the wall at various computational grid points. It utilizes a wall eddy-viscosity formulation as the boundary condition to apply the modeled wall shear stress. Each agent receives states based on local instantaneous flow quantities at an off-wall location, computes a reward based on the estimated wall-shear stress, and provides an action to update the wall eddy viscosity at each time step. The trained wall model is validated in wall-modeled LES of flow over periodic hills at higher Reynolds numbers, and the results show the effectiveness of the model on flow with pressure gradients. The analysis of the trained model indicates that the model is capable of distinguishing between the various pressure gradient regimes present in the flow. To further assess the robustness of the developed wall model, simulations of flow over the Boeing Gaussian bump are conducted at a Reynolds number of 2 x 10^6, based on the free-stream velocity and the bump width. The results of mean skin friction and pressure on the bump surface, as well as the velocity statistics of the flow field, are compared to those obtained from equilibrium wall model (EQWM) simulations and published experimental data sets. The developed wall model is found to successfully capture the acceleration and deceleration of the turbulent boundary layer on the bump surface, providing better predictions of skin friction near the bump peak and exhibiting comparable performance to the EQWM with respect to the wall pressure and velocity field. We also conclude that the subgrid-scale model is crucial to the accurate prediction of the flow field, in particular the prediction of separation.

boundary layer↗