SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

All sources NASA NTRS DOE OSTI

DOE OSTI2024

National Modeling of Geothermal District Energy Systems with Ambient-Temperature Loops Using dGeo: Preprint

Geothermal district energy systems (DES) with ambient-temperature loops, also known as thermal energy networks, are one option for decarbonizing space heating and cooling loads. Geothermal fifth-generation DES include an "ambient" temperature thermal loop that connects heat pumps at each building with thermal balancing sources such as geothermal borehole fields. Heating and cooling are provided via a water-source heat pump at each end-user. This project seeks to analyze the nationwide potential for ambient-temperature loop districts by creating a new module within the Distributed Geothermal Market Demand Model (dGeo). dGeo is an agent-based modeling tool for distributed geothermal resources; it can investigate potential on a nationwide or statewide scale using geospatial data for all 50 states and thermal demands for existing buildings. This process allows for high-level estimates of technical and economic potential for ambient-temperature loop districts across the United States. Using GHEDesigner, a lookup table was created to size borehole fields for different thermal loads and ground conditions experienced across the country. A cost and financing structure, along with incentives, were applied. Cost estimates include costs for the distribution network, borehole field installation and operation, and circulation pump operation, while savings are calculated based on agent energy bills. This newly developed module can be used for assessing which areas of the country have the highest potential for agent benefits from ambient-temperature loop installation and assess the impact of different costing and pricing future scenarios. While the code is still under development and nationwide simulations are ongoing, initial results for two states are provided. Future work includes expanding the module to consider mixed residential and commercial districts and considering multiple costing scenarios.

ambient temperature loop↗

DOE OSTI2025

Replace Human Intelligence with Fast and Smart Geometric Reasoning and Graph Neural Network to Accelerate Next Gen ModSim Workflows

We present an agent-guided approach to CAD geometry decomposition that automates hex/hybrid meshing with graph neural networks (GNNs) to accelerate next-generation ModSim workflows. Our end-to-end pipeline (i) reduces 3D boundary-representation (B-Rep) models to a 2D chordal axis skeleton (CAT) and then to a 1D bipartite graph of surface and curve nodes, (ii) assigns per node labels as Cubit® WebCut actions, (iii) trains a multi-action GNN under supervised learning, and (iv) predicts five surface-node and three curve-node actions on out-of-distribution test geometries. Each graph node carries geometric, topological, and meshing attributes drawn from the B-Rep “skin” and CAT “skeleton,” with two-way mappings across 3D↔2D↔1D representations to maintain traceability back to 3D CAD. The supervised learning model exhibits stable convergence of the binary cross-entropy loss and achieves 98.7% accuracy on unseen lattice models. To operationalize decision-making, we rank predicted commands by geometric significance and prototyped the agent-guided workflow through the Cubit® Meshing PowerTool GUI. As a stretch goal, we explore reinforcement learning (RL) to reduce or remove label requirements and to learn policies for action sequences that maximize total reward (e.g., size of hex-meshable regions and resulting hex mesh quality). When all-hex meshing is not feasible, the agent assists in producing hybrid meshes—prioritizing hex in critical regions and transitioning to tetrahedral elements (tets) elsewhere—maintaining fidelity while ensuring robustness. The overarching objective is to replace manual, heuristics-based decomposition with data-driven, reproducible automation, cutting meshing turnaround time by orders of magnitude. We anticipate direct impact on simulation workflows through intelligent, scalable decomposition of complex CAD models into hex-meshable subdomains.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Closing the Loop between In Situ Stress Complexity and EGS Fracture Complexity

42 ENGINEERING↗

DOE OSTI2026

A Physics-Informed Reinforcement Learning Framework for Economic-Thermal Co-Optimization of Crypto Mining Data Centers: Preprint

The rapid expansion of cryptocurrency mining has created a new class of high-density data centers characterized by extreme thermal flux and high sensitivity to volatile economic markets. Traditional thermal management strategies, typically reliant on rule-based control, maintain static setpoints that fail to account for fluctuating electricity prices and cryptocurrency values - factors critical to mining profitability. To address this, we present a physics-informed reinforcement learning (PIRL) framework for economic-thermal co-optimization in crypto mining data centers. This framework consists of a proximal policy optimization (PPO) agent, a virtual testbed powered by high-fidelity physics-based models, and an interactive frontend dashboard. The PPO agent is trained using the virtual testbed and strict hardware safety limits. This physics-informed approach allows the agent to learn a stochastic policy that dynamically balances mining revenue against operational costs by co-optimizing HVAC cooling setpoints and IT computational hashrate. The simulation results demonstrate that the integrated framework achieved an 8.62% increase in net operational profit compared to traditional baseline strategies while strictly adhering to safety-critical temperature constraints (coolant supply temperature < 32 degrees C). This work provides a scalable template for the deployment of reinforcement learning in mission critical facilities where economic volatility and physical safety must be managed simultaneously.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2020

Capture and Decomposition of the Nerve Agent Simulant, DMCP, Using the Zeolitic Imidazolate Framework (ZIF-8)

Understanding mechanisms of decontamination of chemical warfare agents (CWA) is an area of intense research aimed at developing new filtration materials to protect soldiers and civilians in case of state-sponsored or terrorist attack. In this study, we employed complementary structural, chemical, and dynamic probes and in situ data collection, to elucidate the complex chemistry, capture, and decomposition of the CWA simulant, dimethyl chlorophosphonate (DMCP). Our work reveals key details of the reactive adsorption of DMCP and demonstrates the versatility of zeolitic imidazolate framework (ZIF-8) as a plausible material for CWA capture and decomposition. Here, the in situ synchrotron-based powder X-ray diffraction (PXRD) and pair distribution function (PDF) studies, combined with Fourier transform infrared (FT-IR) spectroscopy, X-ray photoelectron spectroscopy (XPS), zinc K-edge X-ray absorption near edge structure (XANES), and Raman spectroscopies, showed that the unique structure, chemical state, and topology of ZIF-8 enable accessibility, adsorption, and hydrolysis of DMCP into the pores and revealed the importance of linker chemistry and Zn 2+ sites for nerve agent decomposition. DMCP decontamination and decomposition product(s) formation were observed by thermogravimetric analysis, FT-IR spectroscopy, and phosphorus (P) K-edge XANES studies. Differential PDF analysis indicated that the average structure of ZIF-8 (at the 30 Å scale) remains unchanged after DMCP dosing and provided information on the dynamics of interactions of DMCP with the ZIF-8 framework. Using in situ PXRD and diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS), we showed that nearly 90% regeneration of the ZIF-8 structure and complete liberation of DMCP and decomposition products occur upon heating.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

Decentralised Reinforcement Learning for Dynamic Cyberattack Response in Microgrid Networks

Microgrids rely on communication networks for reliable operation, which makes them inherently vulnerable to cyberattacks. Such attacks can destabilise system dynamics and drive states away from their nominal operating trajectories. Although several physics-informed and machine learning-based strategies have been developed to counter these threats, the rapidly evolving cyber landscape enables adversaries to bypass static defences or rules-based mitigation approaches. This paper proposes a dynamic, online-trained and fully decentralised reinforcement learning (RL)-based cyberattack response framework to protect microgrids from evolving cyberattacks. The proposed framework deploys multiple deep Q-networks (DQNs), each associated with a distributed energy resource (DER), to enable localised and adaptive attack mitigation. In this framework, each DQN processes local voltage and frequency measurements—combined with intrusion detection system (IDS) alerts—as observations and rewards to guide decision-making. Extensive simulation studies demonstrate the robustness of the proposed framework under diverse attack scenarios and varying IDS-induced detection delays. Comparative analysis highlights its superiority over existing static or preexisting rules-based mitigation approaches. Finally, we present an analysis that shows the framework's scalability to real-life microgrids with more interacting agents.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1987

Mission planning and simulation via intelligent agents

A system that can operate from a flight manifest to plan and simulate payload preparation and transport via Shuttle flights is described. The design alternatives and the prototype implementation of the payload hardware and inventory tracking system are discussed. It is shown how intelligent agents can be used to generate mission schedules, and how, through the use of these intelligent agents, knowledge becomes separated into small manageable knowledge bases.

Gargan, Robert A., Jr.↗

DOE OSTI2024

"Reinforcement Learning Based Oscillation Dampening: Scaling up Single-Agent RL algorithms to a 100 AV highway field operational test"

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. We explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

Jang, Kathy↗

DOE OSTI2024

Design and optimization of a modular hydrogen-based integrated energy system to maximize revenue via nuclear-renewable sources

Here, this paper demonstrates a novel modular distributed framework that uses optimal energy-dispatching strategies to enable greater flexibility and profitability in nuclear-renewable integrated energy systems (NR-IES). Hydrogen is used as a commodity in this framework since its production can improve grid stability and system operational flexibility, decarbonize heavy industry, and create an additional revenue stream for electricity generators, particularly nuclear power plants with high operational expenses. The proposed solution addresses the challenges associated with merging multiple software and services from various domains by using functional mock-up units (FMU) to co-simulate diverse subsystems designed in various platforms. The tightly coupled integrated energy system (IES) is optimized to maximize revenue by utilizing the deep reinforcement learning (DRL) technique to make smart dispatching decisions based on variable electricity prices and the availability of renewable energy. Proximal policy optimization (PPO) algorithm is used in training and testing the DRL agent. Over a period of 120 days, the proposed hydrogen-based IES framework showed about 10% revenue boost compared to a non-hydrogen generating baseline IES while also providing an easily-adoptable framework which can help to improve the flexibility of future generation nuclear power plants.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2025

Reinforcement Learning-Based Secondary Control Strategy for Voltage and Frequency Regulation in Islanded Inverter-Based Microgrids

This paper presents a reinforcement learning (RL) approach for secondary voltage and frequency control in islanded inverter-based microgrids. The proposed control strategy aims to restore voltage and frequency deviations caused by the primary droop control while ensuring proper power sharing between distributed generators. The RL agent is designed to provide correction signals to the primary control, considering communication delays and system constraints. The effectiveness of the proposed control strategy is validated through simulation results in MATLAB/Simulink environment, demonstrating superior performance in maintaining voltage and frequency within the nominal values.

Rodriguez Martinez, Omar Felipe [University of Pue↗

DOE OSTI2021

Impact of Blockchain Delay on Grid-Tied Solar Inverter Performance: Preprint

This paper investigates the impact of the delay resulting from a blockchain, a promising security measure, for a hierarchical control system of inverters connected to the grid. The blockchain communication network is designed at the secondary control layer for resilience against cyberattacks. To represent the latency in the communication channel, a model is developed based on the complexity of the blockchain framework. Taking this model into account, this work evaluates the plant’s performance subject to communication delays, introduced by the blockchain, among the hierarchical control agents. In addition, this article considers an optimal model-based control strategy that performs the system’s internal control loop. The work shows that the blockchain’s delay size influences the convergence of the power supplied by the inverter to the reference at the point of common coupling. In the results section, real-time simulations on OPAL-RT are performed to test the resilience of two parallel inverters with increasing blockchain complexity.

41 EE - Solar Energy Technologies Office (EE-4S)↗

DOE OSTI2026

Filling the cellulosic bio-economy gap by utilizing a wedge approach combined with stakeholder collaboration

The price gap between the market and breakeven prices of cellulosic biomass for farmers represents a significant barrier to the development of a low-carbon cellulosic bioeconomy. Using a bottom-up, agent-based modeling tool that replicates the behaviors and interactions of key stakeholders, this study analyzes the emergence of a cellulosic bioeconomy at the local scale through a wedge approach that examines an integrated portfolio of multiple policy options, including subsidies for small-scale bioproducts and environmental credits. Here, the role of collaboration among multiple stakeholders, such as biomass producers (farmers), bio-refinery industry, government, and society, is assessed for filling the price gap. Using the Sangamon River Basin as a case study site, we evaluate the effectiveness of the wedge approach by comparing simulation results from multiple scenarios, each incorporating different combinations of bioeconomy wedges, with and without stakeholder collaboration. Results underscore that active collaboration among stakeholders acts as a catalyst enlarging the effectiveness of bioeconomy wedges. Including the carbon credits and environmental value in the policy portfolio is found to bridge the price gap through collective contributions from diverse stakeholders, where the cellulosic biofuel and bioproduct industry plays a pivotal role. Although this study is conducted at the local watershed scale, the methodology and findings offer valuable insights for market development in other watersheds and the potential scaling of local markets to regional and national levels.

09 BIOMASS FUELS↗

DOE OSTI2025

Reinforcement Learning-Based Oscillation Dampening: Scaling Up Single-Agent Reinforcement Learning Algorithms to a 100-Autonomous-Vehicle Highway Field Operational Test

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. Here, we explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

Reinforcement Learning for In-Spill Optimization of the Mu2e Resonant Extraction: Compensating Non-Stationarity

We present design considerations and challenges for the fast machine learning component of a third-order resonant beam extraction regulation system being commissioned to deliver steady beam rates to the mu2e experiment at Fermilab. Dedicated quadrupoles drive the tune toward the 29/3 resonance each spill, extracting beam at kV multiwire septa. The overall Spill Regulation System consists of (1) a “slow” process using ~100-spill averages to adjust the base quad ramp infrequently, (2) a feedforward harmonic content compensator, and (3) the “fast” ML agent reacting during each ongoing spill with on-the-fly additive corrections to the sum of (1) and (2). We have demonstrated improved beam-rate steadying for a fast ML agent compared to a PID controller using a quasi-physical spill simulation, and demonstrated distillation of that simulation into a predictive surrogate model. Current work includes a data-and-training pipeline to generate data-aware surrogates with real-world dynamics, even as the dynamics shift unpredictably. The surrogates are to act as RL environments against which to train our fast ML control agents before deploying them on FPGA in the live system. Further current efforts focus on modeling and controlling beam loss around the storage ring, understanding additional available hardware inputs to the model, and the interplay of these with beam-steadying performance.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS2004

Fire Suppression in Low Gravity Using a Cup Burner

Longer duration missions to the moon, to Mars, and on the International Space Station increase the likelihood of accidental fires. The goal of the present investigation is to: (1) understand the physical and chemical processes of fire suppression in various gravity and O2 levels simulating spacecraft, Mars, and moon missions; (2) provide rigorous testing of numerical models, which include detailed combustion-suppression chemistry and radiation sub-models; and (3) provide basic research results useful for advances in space fire safety technology, including new fire-extinguishing agents and approaches.The structure and extinguishment of enclosed, laminar, methane-air co-flow diffusion flames formed on a cup burner have been studied experimentally and numerically using various fire-extinguishing agents (CO2, N2, He, Ar, CF3H, and Fe(CO)5). The experiments involve both 1g laboratory testing and low-g testing (in drop towers and the KC-135 aircraft). The computation uses a direct numerical simulation with detailed chemistry and radiative heat-loss models. An agent was introduced into a low-speed coflowing oxidizing stream until extinguishment occurred under a fixed minimal fuel velocity, and thus, the extinguishing agent concentrations were determined. The extinguishment of cup-burner flames, which resemble real fires, occurred via a blowoff process (in which the flame base drifted downstream) rather than the global extinction phenomenon typical of counterflow diffusion flames. The computation revealed that the peak reactivity spot (the reaction kernel) formed in the flame base was responsible for attachment and blowoff of the trailing diffusion flame. Furthermore, the buoyancy-induced flame flickering in 1g and thermal and transport properties of the agents affected the flame extinguishment limits.

Takahashi, Fumiaki↗

NASA NTRS2004

Fire Suppression in Low Gravity Using a Cup Burner

Longer duration missions to the moon, to Mars, and on the International Space Station increase the likelihood of accidental fires. The goal of the present investigation is to: (1) understand the physical and chemical processes of fire suppression in various gravity and O2 levels simulating spacecraft, Mars, and moon missions; (2) provide rigorous testing of numerical models, which include detailed combustion suppression chemistry and radiation sub-models; and (3) provide basic research results useful for advances in space fire safety technology, including new fire-extinguishing agents and approaches. The structure and extinguishment of enclosed, laminar, methane-air co-flow diffusion flames formed on a cup burner have been studied experimentally and numerically using various fire-extinguishing agents (CO2, N2, He, Ar, CF3H, and Fe(CO)5). The experiments involve both 1g laboratory testing and low-g testing (in drop towers and the KC-135 aircraft). The computation uses a direct numerical simulation with detailed chemistry and radiative heat-loss models. An agent was introduced into a low-speed coflowing oxidizing stream until extinguishment occurred under a fixed minimal fuel velocity, and thus, the extinguishing agent concentrations were determined. The extinguishment of cup-burner flames, which resemble real fires, occurred via a blowoff process (in which the flame base drifted downstream) rather than the global extinction phenomenon typical of counterflow diffusion flames. The computation revealed that the peak reactivity spot (the reaction kernel) formed in the flame base was responsible for attachment and blowoff of the trailing diffusion flame. Furthermore, the buoyancy-induced flame flickering in 1g and thermal and transport properties of the agents affected the flame extinguishment limits.

Takahashi, Fumiaki↗

DOE OSTI2023

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration

Grid resilience has become a critical topic recently because of the increasing occurrence of extreme events and the growing integration of intermittent renewable energy sources. To build a resilient distribution system, this paper develops a multiagent reinforcement learning-based (MARL) method to coordinate distribution energy resources (DERs) dispatch, load pickup, and network reconfiguration for load restoration after a system outage. With the help of two types of control agents, namely critical load restoration (CLR) and coordination (COR) agents, system loads can be restored efficiently, given available resources. The effectiveness and superiority of the proposed algorithm are demonstrated through simulations and comparative studies on a real distribution feeder in Western Colorado.

distribution system↗

NASA NTRS2003

Advantages of Brahms for Specifying and Implementing a Multiagent Human-Robotic Exploration System

We have developed a model-based, distributed architecture that integrates diverse components in a system designed for lunar and planetary surface operations: an astronaut's space suit, cameras, all-terrain vehicles, robotic assistant, crew in a local habitat, and mission support team. Software processes ('agents') implemented in the Brahms language, run on multiple, mobile platforms. These mobile agents interpret and transform available data to help people and robotic systems coordinate their actions to make operations more safe and efficient. The Brahms-based mobile agent architecture (MAA) uses a novel combination of agent types so the software agents may understand and facilitate communications between people and between system components. A state-of-the-art spoken dialogue interface is integrated with Brahms models, supporting a speech-driven field observation record and rover command system. An important aspect of the methodology involves first simulating the entire system in Brahms, then configuring the agents into a runtime system Thus, Brahms provides a language, engine, and system builder's toolkit for specifying and implementing multiagent systems.

Clancey, William J.↗