Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Self-healing capability is a critical factor for a resilient distribution system, which requires intelligent agents to automatically perform service restoration online, including network reconfiguration and reactive power dispatch. Here, the article proposes the imitation learning framework for training such an agent, where the agent will interact with an expert built based on the mixed-integer program to learn its optimal policy, and therefore significantly improve the training efficiency compared with exploration-dominant reinforcement learning (RL) methods. This significantly improved training efficiency makes the training problem under N-k scenarios tractable. A hybrid policy network is proposed to handle tie-line operations and reactive power dispatch simultaneously to further improve the restoration performance. The 33-bus and 119-bus systems with N-k disturbances are employed to conduct the training. The results indicate that the proposed method outperforms traditional RL algorithms such as the deep-Q network.

42 ENGINEERING↗

Microbial Diversity Associated with the Pollen Stores of Captive-Bred Bumble Bee Colonies

The pollen stores of bumble bees host diverse microbiota that influence overall colony fitness. Yet, the taxonomic identity of these symbiotic microbes is relatively unknown. In this descriptive study, we characterized the microbial community of pollen provisions within captive-bred bumble bee hives obtained from two commercial suppliers located in North America. Findings from 16S rRNA and ITS gene-based analyses revealed that pollen provisions from the captive-bred hives shared several microbial taxa that have been previously detected among wild populations. While diverse microbes across phyla Firmicutes, Proteobacteria, Bacteroidetes, Actinobacteria, and Ascomycota were detected in all commercial hives, significant differences were detected at finer-scale taxonomic resolution based on the supplier source. The causative agent of chalkbrood disease in honey bees, Ascosphaera apis, was detected in all hives obtained from one supplier source, although none of the hives showed symptoms of infection. The shared core microbiota across both commercial supplier sources consisted of two ubiquitous bee-associated groups, Lactobacillus and Wickerhamiella/Starmerella clade yeasts that potentially contribute to the beneficial function of the microbiome of bumble bee pollen provisions.

16S rRNA gene↗

Reinforcement Learning to Enhance Optimal Operation of Resilient Community Energy Systems

This paper presents a novel model-free multi-agent Reinforcement Learning (RL) control method to enhance the resilience of community energy systems in island mode, which coordinates multiple objectives without the necessity of identifying system models that require expert knowledge. Specifically, a community-level coordinator agent is designed to allocate renewable energy resources among different buildings, and multiple building-level agents are developed to optimize load schedules based on limited energy resources and requirements of building loads and occupants’ comfort. In a two-day evaluation, our RL approach demonstrated a similar performance against MPC without requiring system models and formulation of optimization problems as required in MPC.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗

Improved Particle Swarm Algorithm Using Rubik’s Cube Topology for Bilevel Building Energy Transaction

Following the rapid growth of distributed energy resources (e.g., renewables, battery), localized peer-to-peer energy transactions are receiving more attention for multiple benefits, such as reducing power loss and stabilizing the main power grid. To promote distributed renewables locally, the local trading price is usually set to be within the external energy purchasing and selling price range. Consequently, building prosumers are motivated to trade energy through a local transaction center. This local energy transaction is modeled in bilevel optimization game. A selfish upper level agent is assumed with the privilege to set the internal energy transaction price with an objective of maximizing its arbitrage profit. Meanwhile, the building prosumers at the lower level will response to this transaction price and make decisions on electricity transaction amount. Therefore, this non-cooperative leader-follower trading game is seeking for equilibrium solutions on the energy transaction amount and prices. Additionally, a uniform local transaction price structure (purchase price equals selling price) is considered here. Aiming at reducing the computational burden from classical Karush–Kuhn–Tucker (KKT) transformation and protecting the private information of each stakeholder (e.g., building), swarm intelligence-based solution approach is employed for upper level agent to generate trading price and coordinate the transactive operations. On one hand, to decrease the chance of premature convergence in global-best topology, Rubik’s Cube topology is proposed in this study based on further improvement of a two-dimensional square lattice model (i.e., one local-best topology-Von Neumann topology). Rotating operation of the cube is introduced to dynamically changing the neighborhood and enhancing information flow at the later searching state. Several groups of experiments are designed to evaluate the performance of proposed Rubik’s Cube topology-based particle swarm algorithm. The results have validated the effectiveness of proposed topology and operators comparing with global-best version PSO and Von Neumann topology-based PSO and its scalability on larger scale applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Identifying Molecular Scale Interactions of Nucleating Agents with Salt Hydrates for Thermal Energy Storage Applications

Latent heat based thermal energy storage is of interest as a method to mitigate and time-shift thermal load peaks and hence reduce energy demands for heating and cooling buildings. Notably, Glauber’s salt (GS) presents a good choice as a phase change material (PCM) for building applications because it has a melting point near room temperature (i.e., 32.4 °C), a large heat of fusion and energy density, and is low cost (~ $100/ton). However, there are several known limitations to using GS as a PCM, such as incongruent melting, high degree of supercooling, and formation of other hydration states, which renders GS unsuitable for practical use. Inorganic crystals acting as nucleating agents can avoid some of these issues with GS [1]. Understanding nucleation behavior through typical calorimetry experiments offers limited insight into the molecular-scale mechanisms. Here, we report isothermal titration calorimetry [2] to investigate the interactions between sodium sulfate salt and an organic nucleating agent - glycine. Our experiments provide qualitative and quantitative understanding of how glycine interacts with sodium sulfate across various temperatures. The findings offer insights into the nucleation onset temperature (24°C) and underscore the pivotal role of glycine's zwitterionic structure in facilitating nucleation. This preliminary work provides a foundation for subsequent exploration of nucleating agents that would be incorporated in different salt hydrate systems. 1. Purohit, B. K., and V. S. Sistla. "Inorganic salt hydrate for thermal energy storage application: A review." Energy Storage 3, no. 2 (2021): e212. 2. Velázquez-Campoy, Adrián, Hiroyasu Ohtaka, Azin Nezami, Salman Muzammil, and Ernesto Freire. "Isothermal titration calorimetry." Current protocols in cell biology 23, no. 1 (2004): 17-8.

calorimetry, nucleation & growth↗

Convex Decreasing Algorithms: Distributed Synthesis and Finite-Time Termination in Higher Dimension

Here we establish finite time termination algorithms for consensus algorithms based on geometric properties that yield finite-time guarantees, suited for use in high dimension and in the absence of a central authority. These pursuits motivate a new peer to peer convex hull algorithm which is utilized for one stopping algorithm. Further an alternative lightweight norm based stopping criteria is also developed. The practical utility of the algorithm is illustrated through MATLAB simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DSS-SimPy-RL (Open-DSS and SimPy based Cyber-Physical RL environment) [SWR-23-29]

Recently, numerous data-driven approaches to control an electric grid using machine learning techniques have been investigated. With the advancement of reinforcement learning (RL) based techniques, gradually the conventional optimization based solvers are being replaced with RL approach where there is uncertainty in the environment such as renewable generation or cyber system emulation. However, to train an agent efficiently, it requires numerous interactions with an environment to learn the best policies. There are numerous RL environments for the power systems based on some well-known simulators, similarly there are environment for communication domains. While majority of the cyber emulators are based in an UNIX environment, the power simulators are based in the Windows-based operating system, the generation of cyber-physical mixed domain RL environment has been challenging. Existing co-simulation methods are efficient but resource and time intensive to generate large scale data set for training RL agents. Hence, this software focuses on development and validation of a mixed domain RL environment using Open DSS for the physical side and leverages a discrete event simulator python package, SimPy, for cyber-side emulation which is Operating Systems agnostic. Further utilizing this software co-simulation and training RL agents for re-routing based resilient control for network reconfiguration and volt-var control in power distribution feeder are performed.

Sahu, Abhijeet↗

Rational design, synthesis, and evaluation of uncharged, “smart” bis-oxime antidotes of organophosphate-inhibited human acetylcholinesterase

Organophosphate (OP) intoxications from nerve agent and OP pesticide exposures are managed with pyridinium aldoxime–based therapies whose success rates are currently limited. The pyridinium cation hampers uptake of OPs into the central nervous system (CNS). Furthermore, it frequently binds to aromatic residues of OP-inhibited acetylcholinesterase (AChE) in orientations that are nonproductive for AChE reactivation, and the structural diversity of OPs impedes efficient reactivation. Improvements of OP antidotes need to include much better access of AChE reactivators to the CNS and optimized orientation of the antidotes' nucleophile within the AChE active-center gorge. On the basis of X-ray structures of a CNS-penetrating reactivator, monoxime RS194B, reversibly bound to native and venomous agent X (VX)–inhibited human AChE, here we created seven uncharged acetamido bis-oximes as candidate antidotes. Both oxime groups in these bis-oximes were attached to the same central, saturated heterocyclic core. Diverse protonation of the heterocyclic amines and oxime groups of the bis-oximes resulted in equilibration among up to 16 distinct ionization forms, including uncharged forms capable of diffusing into the CNS and multiple zwitterionic forms optimal for reactivation reactions. Conformationally diverse zwitterions that could act as structural antidote variants significantly improved in vitro reactivation of diverse OP-human AChE conjugates. Oxime group reorientation of one of the bis-oximes, forcing it to point into the active center for reactivation, was confirmed by X-ray structural analysis. Finally, our findings provide detailed structure-activity properties of several CNS-directed, uncharged aliphatic bis-oximes holding promise for use as protonation-dependent, conformationally adaptive, “smart” accelerated antidotes against OP toxicity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Physics-Informed Reinforcement Learning Framework for Economic-Thermal Co-Optimization of Crypto Mining Data Centers: Preprint

The rapid expansion of cryptocurrency mining has created a new class of high-density data centers characterized by extreme thermal flux and high sensitivity to volatile economic markets. Traditional thermal management strategies, typically reliant on rule-based control, maintain static setpoints that fail to account for fluctuating electricity prices and cryptocurrency values - factors critical to mining profitability. To address this, we present a physics-informed reinforcement learning (PIRL) framework for economic-thermal co-optimization in crypto mining data centers. This framework consists of a proximal policy optimization (PPO) agent, a virtual testbed powered by high-fidelity physics-based models, and an interactive frontend dashboard. The PPO agent is trained using the virtual testbed and strict hardware safety limits. This physics-informed approach allows the agent to learn a stochastic policy that dynamically balances mining revenue against operational costs by co-optimizing HVAC cooling setpoints and IT computational hashrate. The simulation results demonstrate that the integrated framework achieved an 8.62% increase in net operational profit compared to traditional baseline strategies while strictly adhering to safety-critical temperature constraints (coolant supply temperature < 32 degrees C). This work provides a scalable template for the deployment of reinforcement learning in mission critical facilities where economic volatility and physical safety must be managed simultaneously.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)

Hybrid-RL-MPC4CLR was developed as a hybrid controller for active distribution grid critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. The RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while the MPC models grid operations incorporating the RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. The developers formulated the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The software is developed using various software packages in Python. The MPC's optimal power flow (OPF) model is implemented using the Pyomo package, the RL simulation environment is implemented using the MPC simulation with various scenarios of renewable energy and load demand profiles and power outage beginning times, based on the OpenAI Gym framework. The RL agent training is performed using the RLlib Ray package. The RL algorithm is trained offline using historical forecasts of renewable generation and load demand profiles. Simulation analysis and performance tests are conducted using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery.

Eseye, Abinet Tesfaye↗

Design and optimization of a modular hydrogen-based integrated energy system to maximize revenue via nuclear-renewable sources

Here, this paper demonstrates a novel modular distributed framework that uses optimal energy-dispatching strategies to enable greater flexibility and profitability in nuclear-renewable integrated energy systems (NR-IES). Hydrogen is used as a commodity in this framework since its production can improve grid stability and system operational flexibility, decarbonize heavy industry, and create an additional revenue stream for electricity generators, particularly nuclear power plants with high operational expenses. The proposed solution addresses the challenges associated with merging multiple software and services from various domains by using functional mock-up units (FMU) to co-simulate diverse subsystems designed in various platforms. The tightly coupled integrated energy system (IES) is optimized to maximize revenue by utilizing the deep reinforcement learning (DRL) technique to make smart dispatching decisions based on variable electricity prices and the availability of renewable energy. Proximal policy optimization (PPO) algorithm is used in training and testing the DRL agent. Over a period of 120 days, the proposed hydrogen-based IES framework showed about 10% revenue boost compared to a non-hydrogen generating baseline IES while also providing an easily-adoptable framework which can help to improve the flexibility of future generation nuclear power plants.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

The Transactive Energy Network Template Metamodel

While transactive energy, which is defined as an allocation of electricity based on dynamically discovered values or prices, has been extensively studied, its uptake and use has been slow. This report describes a tool, the transactive network template, which should hasten the creation and uptake of transactive energy networks. Some basic principles of transactive energy are familiar from existing wholesale electricity markets. Locational prices are calculated today for zones within bulk electric transmission systems. Locational prices differ while accounting for the locational costs of electricity generation and the losses and constraints incurred when electricity is transmitted from generators and distributed to consumers. A transactive energy network might include these transmission zones. However, current research strives to apply transactive energy also in electricity distribution circuits, buildings, and even for individual generating and consuming devices. At the same time, researchers explore how to apply transactive energy in real time during increasingly shorter time intervals. Automated computational agents become necessary as transactive energy becomes applied to smaller circuit zones and at faster dynamic timescales. A transactive energy network is an example of a multi-agent system. Each zone in the network is represented by its transactive agent, which makes decisions for and acts on behalf of a business entity that is responsible for and manages one of the circuit regions. A transactive energy network is also an example of a decentralized, distributed control system. Control decisions and responsibilities are distributed among the network’s transactive agents. The transactive agents are independent; that is, there typically is no centralized authority or oversight function. Instead, transactive agents exchange transactive signals and thereby negotiate the prices and quantities of electricity that they will exchange. Initially, the circuit regions and responsibilities of transactive agents appear to be very dissimilar. Each circuit region may comprise transmission, distribution, or building-level circuits. Each has a unique position and electrical connectivity within the transactive energy network. Each possesses unique assets that either generate or consume electricity, and these (e.g., renewable energy generator, diesel generator, aggregate utility load, building load, space conditioning, refrigerator, etc.) may further differ in their price flexibility and in their strategies for responding to dynamic electricity prices. Given such diversity, an implementer’s first inclination might be to start from scratch to define all these devices and to engineer their seemingly unique interactions. Given that each implementer’s perspective may be narrow within a transactive energy network, it is unlikely that uniquely engineered systems would interact well. This is where the transactive network template is applicable. The transactive network template is a metamodel that has been developed to guide implementers as they configure their own transactive agent within a network of such agents. The object-oriented design of the transactive network template provides basic code object types that may be used and extended by implementers to represent each of the assets in their circuit region. These objects further facilitate the transactive agent’s necessary computations, which are divided among responsibilities to schedule power usage, balance electric supply and demand, and coordinate the exchange of electricity with the other transactive agents. This report addresses the conceptual transactive network template design. Implementers are directed to more formal design documents and reference implementations. A Python™-based1 reference implementation of the transactive network template has been coded, and three implementations have been configured to represent a national laboratory and two university campuses. Version 2 of the transactive node template generalizes the market class and its methods to facilitate multiple, and more diverse market coordination mechanisms than were facilitated by and demonstrated using Version 1. Version 3 includes new Appendix B, which addresses the designs of methods that would make dynamic prices track approved electricity rates. In the future, the author wishes to make the transactive network template more generally applicable to networks that require more accurate power flow. Development of the transactive network template is jointly funded by the U.S. Department of Energy (DOE) Energy Efficiency and Renewable Energy and the DOE Office of Electricity. In late 2015, one of the first projects to be funded by the DOE Grid Laboratory Modernization Laboratory Consortium was the Clean Energy and Transactive Campus project, led by Pacific Northwest National Laboratory. DOE funds were matched by an investment by the Washington Department of Commerce through its Clean Energy Fund. The transactive network template was developed to guide the implementation of transactive energy networks within this project’s scope.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Impact of Blockchain Delay on Grid-Tied Solar Inverter Performance: Preprint

This paper investigates the impact of the delay resulting from a blockchain, a promising security measure, for a hierarchical control system of inverters connected to the grid. The blockchain communication network is designed at the secondary control layer for resilience against cyberattacks. To represent the latency in the communication channel, a model is developed based on the complexity of the blockchain framework. Taking this model into account, this work evaluates the plant’s performance subject to communication delays, introduced by the blockchain, among the hierarchical control agents. In addition, this article considers an optimal model-based control strategy that performs the system’s internal control loop. The work shows that the blockchain’s delay size influences the convergence of the power supplied by the inverter to the reference at the point of common coupling. In the results section, real-time simulations on OPAL-RT are performed to test the resilience of two parallel inverters with increasing blockchain complexity.

41 EE - Solar Energy Technologies Office (EE-4S)↗

A vision of immuno-oncology: the Siena think tank of the Italian Network for Tumor Biotherapy (NIBIT) foundation

The yearly Think Tank Meeting of the Italian Network for Tumor Biotherapy (NIBIT) Foundation, brings together in Siena, Tuscany (Italy), experts in immuno-oncology to review the learnings from current immunotherapy treatments, and to propose new pre-clinical and clinical investigations in selected research areas. While immunotherapies in non-small cell lung cancer and melanoma led to practice changing therapies, the same therapies had only modest benefit for patients with other malignancies, such as mesothelioma and glioblastoma. One way to improve on current immunotherapies is to alter the sequence of each combination agent. Matching the immunotherapy to the host’s immune response may thus improve the activity of the current treatments. A second approach is to combine current immunotherapies with novel agents targeting complementary mechanisms. Identifying the appropriate novel agents may require different approaches than the traditional laboratory-based discovery work. For example, artificial intelligence-based research may help focusing the search for innovative and most promising combination partners. Novel immunotherapies are needed in cancer patients with resistance to or relapse after current immunotherapeutic drugs. Such new treatments may include targeted agents or monoclonal antibodies to overcome the immune-suppressive tumor microenvironment. The mode of combining the novel treatments, including vaccines, needs to be matched to the patient’s immune status for achieving the maximum benefit. In this scenario, specific attention should be also paid nowadays to the immune intersection between COVID-19 and cancer.

60 APPLIED LIFE SCIENCES↗

Computational methods in solution-based plastics purification

Plastic waste can be recycled into resins with near-virgin properties by solution-based purification processes that selectively dissolve polymers, remove contaminants, or detach printing residues. Here, in this review, we examine computational methods for predicting the behavior governing solution-based plastic purification, motivated by the vast polymer–solvent–contaminant compositional space. We discuss thermodynamic and machine learning methods for predicting polymer–solvent and polymer–contaminant interaction and review physics-based molecular dynamics simulations that resolve molecular-scale phenomena within polymer matrices inaccessible to screening methods. We highlight how these methods have informed experimental design for dissolution-based recycling and solvent-based contaminant removal. Finally, we discuss the prospective role of agentic AI in integrating these computational tools with real-time sorting data to adapt purification conditions to the compositional variability of real post-consumer feedstocks. This review charts a path toward computationally guided solution-based purification workflows that can respond to the complexity inherent in plastic waste streams.

Altamimi, Ali [Univ. of Wisconsin, Madison, WI (Un↗

Graph Convolutional Network-Based Topology Embedded Deep Reinforcement Learning for Voltage Stability Control

Topological variations in power system is a common phenomenon and can impose significant challenges to traditional controllers of power system. Recent study revealed the strength of deep reinforcement learning (DRL) based approaches in power system preventive and corrective control. But topological variations are difficult to capture using classical fully connected neural network (FCN) model and has not been explicitly modeled in previous work. Hence, we develop a Graph Convolutional Network (GCN) based DRL framework to tackle topology changes in control design of power system. The GCN model exploits the graph structure of the power network and helps the DRL agent to embed the topology information during learning process. Our GCN based approach is evaluated using the IEEE-39 bus system and it outperforms the FCN-based DRL scheme in terms of training convergence and control performance considering grid topology changes.

Hossain, Ramij Raja↗

Anion Coordination Improves High-Temperature Performance and Stability of NaPF 6 -Based Electrolytes for Supercapacitors

Electrolyte stability can be improved by incorporating complexing agents that bind key decomposition intermediates and slow down decomposition. We show that hexamethyl-phosphoramide (HMPA) extends both the thermal stability threshold of sodium hexafluorophosphate (NaPF 6 ) in dimethoxyethane (DME) electrolyte and the cycle life of double-layer capacitors. HMPA forms a stable complex with PF 5 , an intermediate in PF 6 anion thermal degradation. Unbound, this intermediate leads to autocatalytic degradation of the electrolyte solution. The results of electrochemical impedance spectroscopy (EIS) and galvanostatic cycling measurements show large changes in the cell without the presence of HMPA at higher temperatures (≥60 °C). Fourier transform infrared spectroscopy (FTIR) on the liquid and gas phase of the electrolyte shows without HMPA the formation of measurable amounts of PF 5 and HF. The complimentary results of these measurements proved the usefulness of using Lewis bases such as HMPA to inhibit the degradation of the electrolyte solution at elevated temperatures and potentially lead to improve cycle life of a nonaqueous capacitor. The results showed a large increase in capacitance retention during cycling (72% retention after 750,000 cycles). The results also provide evidence of major decomposition processes (0% capacitance retention after 100,000 cycles) that take place at higher temperatures without the additive of a thermal stability additive such as HMPA.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers

Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.

Naug, Avisek [Hewlett Packard Enterprise]↗