Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Reinforcement Learning for Load-balanced Parallel Particle Tracing

We explore an online reinforcement learning (RL) paradigm to dynamically optimize parallel particle tracing performance in distributed-memory systems. Our method combines three novel components: (1) a work donation algorithm, (2) a high-order workload estimation model, and (3) a communication cost model. First, we design an RL-based work donation algorithm. Our algorithm monitors workloads of processes and creates RL agents to donate data blocks and particles from high-workload processes to low-workload processes to minimize program execution time. The agents learn the donation strategy on the fly based on reward and cost functions designed to consider processes' workload changes and data transfer costs of donation actions. Second, we propose a workload estimation model, helping RL agents estimate the workload distribution of processes in future computations. Third, we design a communication cost model that considers both block and particle data exchange costs, helping RL agents make effective decisions with minimized communication costs. We demonstrate that our algorithm adapts to different flow behaviors in large-scale fluid dynamics, ocean, and weather simulation data. Our algorithm improves parallel particle tracing performance in terms of parallel efficiency, load balance, and costs of I/O and communication for evaluations with up to 16,384 processors.

Distributed and parallel particle tracing↗
DOE OSTI2020

Formal methods approach to the charging facility location problem for battery electric vehicles

Battery electric vehicles (BEVs) are becoming more prevalent as improvements in battery technology and energy management continue to be made. As the number of electric vehicles grows, the demand for fast-charging stations is expected to increase dramatically. Thus, building new charging station infrastructure efficiently will be key to reducing upfront costs while meeting consumer demands. In this work, we propose a method for choosing a set of charging station locations that are optimized based on a set of given common vehicle demand points. As part of this solution, we also offer a novel abstraction of the road network on which energy-efficient paths that account for charge-time delays may be found. The current algorithm chooses the optimal charging locations for a single agent which has a route objective specified using temporal logic. To demonstrate the proposed method, the running example shows how a charging station could be chosen for an electric delivery vehicle. Simulations were run on sample road networks with a given set of demand points to service and potential charging station locations to compare. The method is shown to successfully rank potential charging stations in terms of their expected average charging time cost.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

RLC4CLR (Reinforcement Learning Controller for Critical Load Restoration Problems)

RLC4CLR demonstrates using a reinforcement learning controller (RLC) to solve a critical load restoration (CLR) problem, which improves the grid resilience after a substation outage event. RLC4CLR consists of two parts. (1) RL environment: This environment encapsulates the CLR problem to be solved and provides interfacing functions to follow the standard OpenAI Gym format. A power system simulator, i.e., OpenDSS, is included to provide the power flow solution. Controller inputs and outputs (RL state and action) as well as the reward are defined in this environment as well. In summary, the RL environment is the problem formulation from which the RL agent can learn. (2) RL training script: The training script enables the RL agent to learn its control policy by interacting with the RL environment. For RL training, an open-sourced RL library, i.e., RLlib, is leveraged which is based on a distributed computing framework (Ray). The training script is designed to be able to be run on both local machine or the NREL HPC system. Other components of RLC4CLR include input data, e.g., grid model (standard IEEE test feeders), and other files used for results analysis.

Zhang, Xiangyu↗
DOE OSTI2023

Sensor Development for Liquid Water Detection in Dry Storage Casks: FY 2023 Status

Modeling efforts were undertaken in fiscal year (FY) 2023 to evaluate the feasibility and capability for sensing of liquid water inside canister-based dry cask storage systems (DCSSs). The focus was on the development of full-scale finite element models (FEMs) of ultrasound propagation through DCSS canister components. For these initial investigations, simulation of the baseplate component was targeted, envisioning water collection at the bottom of a vertically oriented canister. The effects of internal canister components (i.e., fuel basket, fuel assemblies) on ultrasound propagation in the baseplate component were evaluated in these efforts. The environment inside a DCSS confinement is intended to be inert and free of water to prevent potential corrosion of used fuel cladding or other internal hardware. Spent fuel assemblies are dried, after storage in water pools, to make sure water has been removed from assembly cavities. However, there is some uncertainty about the amount of residual water potentially left behind in a DCSS after drying processes, because water can become trapped in cavities or other small crevices in the surfaces formed by the fuel cladding, fuel assemblies, and other internal hardware components. Considering the complex space- and time-dependent temperature profiles in DCSSs, water may be in a liquid or gas phase depending on its location in the cask and how long the cask has been in storage. Evacuating most water and oxidizing agents contained within a canister is recommended by NUREG-1536 (NRC 2010), which covers DCSSs. As summarized by Salazar et al. (2020), existing guidance typically relies on achievable vacuum pumping pressures sustained over a hold period as a signal of dryness and water removal. However, time series data about water removal from full-scale commercial drying procedures are lacking (Hanson and Alsaed 2019). A review of drying specifications from several vendors led to the conclusion that if the specifications are followed correctly, the residual moisture in DCSSs should present an insignificant risk of cladding degradation (Knoll and Gilbert 1987). A more recent analysis concluded that much larger quantities of residual water could remain in DCSSs, but the amount would still not be expected to lead to significant corrosion of fuel cladding or other internal components (Jung et al. 2013). Industry drying procedures are mostly prescriptive in nature, and operational issues arising during the process could result in incomplete drying. A summary of operational issues and potential negative effects of residual water is provided by Salazar et al. (2019). Experimental efforts are ongoing to validate the extent of water removal in a DCSS based on drying procedures used at nuclear power plants through well-designed investigations of drying process efficacy and water retention (Durbin et al. 2021; Pulido et al. 2022a, Pulido et al. 2022b). This has been approached by simulating limited portions of a DCSS internal volume and fuel assemblies. So far, these efforts have focused on the effects of potential water trapping in the dashpot region of control rod guide tubes but are expected to be expanded to include the effects of other internal hardware features and failed fuel rod cladding. A method for detection and measurement of liquid water (Meyer et al. 2022; Meyer et al. 2021), in tandem with the drying information gained through experimental investigations, provides comprehensive bases for understanding the internal conditions of DCSSs and provides technical information to support licensing decisions.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Antifoam Development for Eliminating Flammability Hazards and Decreasing Cycle Time in the Defense Waste Processing Facility

The Savannah River National Laboratory (SRNL) was requested to develop a new antifoam control method for the Defense Waste Processing Facility’s (DWPF) Chemical Process Cell (CPC). SRNL completed testing of both chemical and nonchemical foam controls. The nonchemical foam controls were either ineffective (or worse, created more foam) or impractical (a water spray can control foam, but excessive water is needed). As a result, the focus of this study was on finding a superwetter or commercial antifoam for controlling foam. Thirty potential antifoams were tested as part of this study. A series of tests were developed to help screen out ineffective alternatives including: 1. Spreading testing of superspreaders, 2. Foam column testing with physical simulants, 3. Boiling testing with physical and chemical simulants, 4. Days-only Sludge Receipt and Adjustment Tank (SRAT) process simulations with sludge (containing noble metals and mercury), Precipitate Reactor Feed Tank (PRFT), and Slurry Mix Evaporator Feed Tank (SEFT) simulants in the RC1 Reaction Calorimeter (purchased for antifoam testing), and 5. Around-the-clock SRAT and Slurry Mix Evaporator (SME) process simulations with sludge(containing noble metals and mercury), PRFT, and SEFT simulants in the RC1 Reaction Calorimeter. Evonik Surfynol® MD20, a commercially available defoamer, was relatively effective in controlling foam, while remaining chemically stable in SRAT and SME processing across the pH range of 4 to 13. No degradation products were detected in the offgas, in the condensate or in the SRAT and SME products. In nitric-glycolic acid flowsheet testing, 250 mg/kg Evonik Surfynol® MD20 was needed for foam control compared to 1,625 mg/kg for Antifoam 747, DWPF’s current antifoam. In nitric-formic acid flowsheet testing, 1,125 mg/kg of Evonik Surfynol® MD20 was needed to control foam throughout the SRAT and SME cycles. The commercially available superspreader Momentive™ Y-17112 was even more effective than Evonik Surfynol® MD20 as both a defoamer and an antifoam. Not only was the foam destroyed upon addition but also was less persistent between additions. It was the most effective antifoam in testing using both the nitric-glycolic acid flowsheet and the nitric-formic acid flowsheet. In nitric-glycolic acid flowsheet testing, only 100 mg/kg Momentive™ Y-17112 was needed to control foam throughout the SRAT and SME cycles. In nitric-formic acid flowsheet testing, 300 mg/kg Momentive™ Y-17112 was needed to control foam throughout the SRAT and SME cycles. Momentive™ Y-17112 is also resistant to hydrolysis as demonstrated by its chemical stability in SRAT and SME processing across the pH range of 4 to 13 and lack of degradation products in offgas or condensate. Both candidates were effective as potential replacements for Antifoam 747, with Y-17112 demonstrating superior foam control. During nitric-glycolic flowsheet testing 50% less antifoam was needed when using Momentive™ Y-17112 compared to MD20. During nitric-formic flowsheet testing 75% less antifoam was needed when using Momentive™ Y-17112 compared to MD20. Foam remediated with Momentive™ Y-17112 was less persistent throughout testing. In addition, no degradation products were detected in the offgas, in the condensate or in the SRAT and SME products. Based on this testing, Momentive™ Y-17112 is clearly superior to Evonik Surfynol® MD20 and Antifoam 747, especially for the nitric-formic acid flowsheet processing; it is recommended that Momentive™ Y-17112 replace Antifoam 747 in DWPF. An antifoam addition strategy is recommended for both the nitric-glycolic acid flowsheet and the nitric-formic acid flowsheet. Implementation of Momentive™ Y-17112 is expected to decrease SRAT and SME boiling times by up to 50%, eliminate the issues resulting from Antifoam 747 degradation products, and minimize foamovers. To validate the effectiveness of these defoaming agents, SRNL recommends irradiation of a SRAT or SME product simulant containing fresh antifoam. The goal of this testing is to determine whether the irradiation causes decomposition of the antifoam that would make it less effective or produce new species in the offgas or slurry. This testing began in April 2020. An evaluation should be completed to determine the thermolytic hydrogen and methane generation rate in downstream equipment, including the High-Level Waste evaporators.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

A Sequential Model Predictive and Deep Reinforcement Learning-Based Controller for Distribution System Outage Mitigation under Hurricane Events

This paper proposes a proactive outage mitigation framework for power distribution networks to withstand hurricane-induced disruptions. It leverages Model Predictive Control (MPC) to identify safe lines for proactive switching during hurricanes, minimizing the risk of cascading failures and voltage violations. The switching strategies optimized by MPC are sequentially integrated with a Deep Reinforcement Learning agent using the Advantage Actor-Critic algorithm, enabling dynamic line switching to maximize connected buses and minimize voltage violations in real time. Using a probabilistic hurricane model, the framework predicts line failures and adapts to varying conditions to enhance grid resilience. Simulations on the IEEE 123-bus system demonstrate its effectiveness in maintaining high connectivity and minimizing disruptions. Real-time testing with an RTDS confirms the practicality and reliability of the proposed approach.

Selim, Alaa [University of Connecticut]↗
DOE OSTI2026

Small-molecule modulation of β-arrestins

β-Arrestins are multifunctional regulators of G-protein-coupled receptor (GPCR) signalling and orchestrate diverse downstream signalling events and physiological responses across the GPCR superfamily. Although GPCR pharmacology has advanced to target orthosteric and allosteric sites, as well as G proteins and GPCR kinases, direct chemical tools to modulate β-arrestin activities have remained conspicuously absent. Here we report the identification of small-molecule inhibitors that selectively target β-arrestins and delineate their mechanism of action through integrated pharmacological, biochemical, biophysical and structural analyses. These inhibitors disrupt β-arrestin engagement with agonist-activated GPCRs, impairing desensitization, internalization and β-arrestin-dependent physiological functions while sparing G protein–receptor coupling. Cryo-electron microscopy, molecular dynamics simulations and structure-guided mutagenesis reveal that one modulator, Cmpd-5, engages a pocket within the central crest of β-arrestin1 formed by the middle, C and lariat loops, a critical receptor-binding interface, stabilizing a distinct conformation that is incompatible with full β-arrestin–receptor engagement. Together, these findings establish a mechanistic framework for β-arrestin modulation, reveal a novel allosteric site for structure-based drug design, and open new avenues for transducer-targeted, pathway-specific GPCR therapeutic agents.

Kahsai, Alem W. [Duke University, Durham, NC (Unit↗
DOE OSTI2024

Reinforcement learning pulses for transmon qubit entangling gates

The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Distributed deep reinforcement learning for simulation control

Abstract Several applications in the scientific simulation of physical systems can be formulated as control/optimization problems. The computational models for such systems generally contain hyperparameters, which control solution fidelity and computational expense. The tuning of these parameters is non-trivial and the general approach is to manually ‘spot-check’ for good combinations. This is because optimal hyperparameter configuration search becomes intractable when the parameter space is large and when they may vary dynamically. To address this issue, we present a framework based on deep reinforcement learning (RL) to train a deep neural network agent that controls a model solve by varying parameters dynamically. First, we validate our RL framework for the problem of controlling chaos in chaotic systems by dynamically changing the parameters of the system. Subsequently, we illustrate the capabilities of our framework for accelerating the convergence of a steady-state computational fluid dynamics solver by automatically adjusting the relaxation factors of the discretized Navier–Stokes equations during run-time. The results indicate that the run-time control of the relaxation factors by the learned policy leads to a significant reduction in the number of iterations for convergence compared to the random selection of the relaxation factors. Our results point to potential benefits for learning adaptive hyperparameter learning strategies across different geometries and boundary conditions with implications for reduced computational campaign expenses 4 4 Data and codes available at https://github.com/Romit-Maulik/PAR-RL . .

42 ENGINEERING↗
DOE OSTI2021

Reduced Order Model of Transactive Bidding Loads

Transactive energy (TE) has been identified to provide better grid efficiency and reliability by market-based transactive exchanges between energy producers and energy consumers. Simulations of TE systems are crucial to evaluate the benefits and impacts of different transactive mechanisms. However, such simulations can be time consuming due to the information exchange between various participants and complex co-simulation environments. In this paper, we develop a reduced order model to speed up the simulation of transactive systems in TE simulation platform (TESP) while achieving very low error between the reduced order and full model results. Specifically, the developed reduced order model consists of an aggregate responsive load agent which utilizes two Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTMs) to enable transactive elements to collectively participate in the TE system. The proposed aggregate responsive load (ARL) agent is able to produce similar transactive behaviors to the full simulation model while achieving significant simulation time reduction. Finally, we also show that the developed model enables generalization of simulation results across different dates and across different number of loads included in the simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Autonomous Cyber Defense Against Dynamic Multi-strategy Infrastructural DDoS Attacks

Dynamic Infrastructural Distributed Denial of Service (I-DDoS) attacks constantly change attack vectors to congest core backhaul links and disrupt critical network availability while evading end-system defenses. To effectively counter these highly dynamic attacks, defense mechanisms need to exhibit adaptive decision strategies for real-time mitigation. This paper presents a novel Autonomous DDoS Defense framework that employs model-based reinforcement agents. The framework continuously learns attack strategies, predicts attack actions, and dynamically determines the optimal composition of defense tactics such as filtering, limiting, and rerouting for flow diversion. Our contributions include extending the underlying formulation of the Markov Decision Process (MDP) to address simultaneous DDoS attack and defense behavior, and accounting for environmental uncertainties. We also propose a fine-grained action mitigation approach robust to classification inaccuracies in Intrusion Detection Systems (IDS). Additionally, our reinforcement learning model demonstrates resilience against evasion and deceptive attacks. Evaluation experiments using real-world and simulated DDoS traces demonstrate that our autonomous defense framework ensures the delivery of approximately 96 - 98% of benign traffic despite the diverse range of attack strategies.

Dutta, Ashutosh↗
DOE OSTI2023

Reinforcement Learning of Structured Stabilizing Control for Linear Systems With Unknown State Matrix

This paper delves into designing feedback control gains for a continuous-time linear quadratic regulator (LQR) problem that is constrained to certain predefined structure with unknown state matrix. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. Here we first formulate a model-based framework using dynamic programming (DP) to embed the structural constraint to the LQR gain computation in the continuous-time setting, and then subsequently, formulate a policy iteration RL algorithm that can alleviate the requirement of known state matrix in conjunction with maintaining the feedback gain structure. The design enables a distributed learning control design which is necessary for many large-scale cyber-physical systems. Theoretical guarantees are provided for stability and convergence of the structured reinforcement learning (SRL) algorithm. We validate our theoretical results with numerical simulations on a multi-agent networked linear time-invariant (LTI) dynamic system.

42 ENGINEERING↗
DOE OSTI2023

AI-Based Faster-Than-Real-Time Stability Assessment of Large Power Systems with Applications on WECC System

Achieving clean energy goals will require significant advances in regard to addressing the computational needs for next-generation renewable-dominated power grids. One critical obstacle that lies in the way of transitioning today’s power grid to a renewable-dominated power grid is the lack of a faster-than-real-time stability assessment technology for operating a fast-changing power grid. This paper proposes an artificial intelligence (AI) -based method that predicts the system’s stability margin information (e.g., the frequency nadir in the frequency stability assessment and the critical clearing time (CCT) value in the transient stability assessment) directly from the system operating conditions without performing the conventional time-consuming time-domain simulations over detailed dynamic models. Since the AI method shifts the majority of the computational burden to offline training, the online evaluation is extremely fast. This paper has tested the AI-based stability assessment method using multiple dispatch cases that are converted and tuned from actual dispatch cases of the Western Electricity Coordinating Council (WECC) system model with more than 20,000 buses. The results show that the AI-based method could accurately predict the stability margin of such a large power system in less than 0.2 milliseconds using the offline-trained AI agent. Therefore, the proposed method has great potential to achieve faster-than-real-time stability assessment for practical large power systems while preserving sufficient accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A comparison of ceramic and carbon-based reductants for vitrification of low-activity waste

Sucrose is the current baseline additive at the Hanford Waste Treatment and Immobilization Plant in Washington, USA to control foaming during waste feed to glass transitions and the redox state of the glass melt. Alternative reductants are being investigated to alleviate strain on effluent treatment from toxic acetonitrile production from incomplete combustion of sucrose. This study evaluates ceramic additive options including B4C, B6Si, SiC and VB2 in simulated low-activity waste feed, as well as coke dust, probing the feed volume expansion during melting as well as the gas evolution. All alternative reductant options examined significantly reduced acetonitrile production, however there was variability in their effectiveness as foam-reducing agents. VB2 and coke at the appropriate ratios were similarly effective as sucrose in controlling both foam volume and glass redox state, but with considerably less acetonitrile production. B4C, B6Si and SiC showed more promising foam control and very little acetonitrile production, however all of the final glasses were over reduced, i.e., Fe2+/FeT = 0.5. These alternative reductant studies provide operational flexibility to the operation of the vitrification plant, as well as options for alternative raw materials in industrial glass melting.

Rigby, Jessica C. (ORCID:0000000235719977)↗
DOE OSTI2026

Bounded-Confidence Models of Multidimensional Opinions with Topic-Weighted Discordance

People’s opinions on a wide range of topics often evolve over time through their interactions with others. Models of opinion dynamics primarily focus on one-dimensional opinions, which represent opinions on one topic. However, opinions on various topics are rarely isolated; instead, they can be interdependent and correlated. In a bounded-confidence model (BCM) of opinion dynamics, agents are receptive to each other only if their opinions are sufficiently similar. Here, we extend classical agent-based BCMs—namely, the Hegselmann–Krause BCM, which has synchronous interactions, and the Deffuant–Weisbuch BCM, which has asynchronous interactions—to a multidimensional setting, in which the opinions are multidimensional vectors representing opinions of different topics and opinions on different topics are interdependent. To measure opinion differences between agents, we introduce topic-weighted discordance functions that account for opinion differences in all topics. We define regions of receptiveness for our models, and we use them to characterize the steady-state opinion clusters and provide an analytical approach to compute these regions. In addition, we numerically simulate our models on various networks with initial opinions drawn from a variety of distributions. When initial opinions are correlated across different topics, our topic-weighted BCMs yield significantly different results in both transient and steady states compared to baseline models, where the dynamics of each opinion topic are independent.

Mathematics and Computing↗
DOE OSTI2021

Isomer of linker for NU-1000 yields a new she -type, catalytic, and hierarchically porous, Zr-based metal–organic framework

The well-known MOF (metal–organic framework) linker tetrakis(p-benzoate)pyrene (TBAPy 4– ) lacks steric hindrance between its benzoates. Changing the 1,3,6,8-siting of benzoates in TBAPy 4– to 4,5,9,10-siting introduces substantial steric hindrance and, in turn, enables the synthesis of a new hierarchically porous, she-type MOF Zr 6 (μ 3 -O)4(μ 3 -OH) 4 (C 6 H 5 COO) 3 (COO) 3 (TBAPy-2) 3/2 (NU-601), where TBAPy-2 4– is the 4,5,9,10 isomer of TBAPy 4– . In conclusion, NU-601 shows high catalytic activity for degradative hydrolysis of a simulant for G-type fluoro-phosphorus nerve agents.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Stability study of aqueous foams under high-temperature and high-pressure conditions relevant to Enhanced Geothermal Systems (EGS)

In Enhanced Geothermal System (EGS) hydraulic fracturing is carried out by injecting cold water into deep Hot Dry Rocks (HDR) under carefully controlled conditions to create new or reopen existing fractures. Water-based fracturing fluids demonstrate some challenges including immense quantity of water usage, water sensitivity of the formations, water blocking, and lack of proppant carrying capacity and transportation. Thus, an alternative is to use foam-based fracturing fluid which offers potential advantage over conventional water-based fracturing fluid such as minimum water usage, reduced wellbore damage, high proppant carrying capacity, and less environmental damage. However, foams are complex mixture of gaseous phase and liquid phase which are thermodynamically unstable at downhole conditions, and their stability can decrease over time due to liquid drainage, bubble coarsening, and coalescence. This paper shows laboratory experiments executed to study foam stability at high temperature (200°C) and high pressure (6.9MPa) conditions which simulates the geothermal environment. Foam stability was characterized by half-life of foam, which is defined as the time taken by the foam to decreases by 50% of its original height due to drainage. In this paper, two types of gaseous phases, nitrogen (N 2 ) and carbon dioxide (CO 2 ) were investigated. Also, based on successful practice of foam-based fracturing fluid in oil and gas industries, four surfactants, including Alpha olefin sulfonate (AOS), Sodium dodecyl sulfonate (SDS), Tergitol™ (NP – 40), and Cetyltrimethylammonium chloride (CTAC) at optimum concentration of 1wt.% were tested for best stability performance. In addition, different stabilizing agents including guar gum, bentonite clay, crosslinker, silicon dioxide nanoparticles (SiO 2 ), graphene oxide (GO) were also studied. Experimental results showed that N 2 foams were more stable than CO 2 foams. It was observed that foam half-life decreased with the increase in temperature. Among all the surfactants, AOS foams showed the most promising thermal stability at high temperatures. Moreover, with the addition of stabilizing agents, foam's half-life was enhanced. Stabilizing agents such as crosslinker and GO dispersion showed the most stable foams with half-life recorded at 20min and 17min, respectively, at 200°C and 6.9MPa. Finally, pressure also showed a positive effect on foam stability; with increased pressure, foam half-life was increased. Based on the experimental data, analytical models for the effect of temperature and pressure were developed, considering foam degradation is a first-order kinetic reaction that linearly depends on the foam drainage mechanism. The effect of temperature on foam half-life was studied as an exponential decay model. In this model, foam half-life is a function of drainage rate constant (D A ) and activation energy (E a ) of the foam system. The effect of pressure on foam half-life was found to obey a power-law model where an increase in pressure showed an increase in foam half-life. Furthermore, a linear relation was studied for the effect of pressure on foam activation energy and drainage rate. Then the combined effects of temperature and pressure were studied, which yielded an analytical model to predict the foam stabilities in terms of half-life for different foam compositions. In conclusion, this research indicates that with an appropriate selection of surfactants and stabilizing agents, it is possible to obtain stable foams, which could replace conventional water fracturing fluid under EGS conditions.

15 GEOTHERMAL ENERGY↗
DOE OSTI2025

Deep reinforcement learning control for co-optimizing energy consumption, thermal comfort, and indoor air quality in an office building

With the recent demand for decarbonization and energy efficiency, advanced HVAC control using Deep Reinforcement Learning (DRL) becomes a promising solution. Due to its flexible structures, DRL has been successful in energy reduction for many HVAC systems. However, only a few researches applied DRL agents to manage the entire central HVAC system and control multiple components in both the water loop and the air loop, owing to its complex system structures. Moreover, those researches have not extended their applications by incorporating the indoor air quality, especially both CO2 and PM2.5concentrations, on top of energy saving and thermal comfort, as achieving those objectives simultaneously can cause multiple control conflicts. What's more, DRL agents are usually trained on the simulation environment before deployment, so another challenge is to develop an accurate but relatively simple simulator. Therefore, we propose a DRL algorithm for a central HVAC system to co-optimize energy consumption, thermal comfort, indoor CO2 level, and indoor PM2.5 level in an office building. To train the controller, we also developed a hybrid simulator that decoupled the complex system into multiple simulation models, which are calibrated separately using laboratory test data. The hybrid simulator combined the dynamics of the HVAC system, the building envelope, as well as moisture, CO2, and particulate matter transfer. Three control algorithms (rule-based, MPC, and DRL) are developed, and their performances are evaluated on the hybrid simulator environment with a realistic scenario (i.e., with stochastic noises). The test results showed that, the DRL controller can save 21.4 % of energy compared to a rule-based controller, and has improved thermal comfort, reduced indoor CO2 concentration. The MPC controller showed an 18.6 % energy saving compared to the DRL controller, mainly due to savings from comfort and indoor air quality boundary violations caused by unmeasured disturbances, and it also highlights computational challenges in real-time control due to non-linear optimization. Finally, we provide the practical considerations for designing and implementing the DRL and MPC controllers based on their respective pros and cons.

Guo, Fangzhou↗