Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

42 ENGINEERING↗
DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

actuated signal control↗
DOE OSTI2020

Optimal Equilibrium Selection of Price-Maker Agents in Performance-Based Regulation Market

This paper analyzes the oligopolistic equilibrium of multiple price-maker agents in performance-based regulation (PBR) markets. In these markets, there are price-maker agents representing some frequency regulation (FR) providers and a number of independent price-taker FR providers. An equilibrium problem with equilibrium constraints (EPEC) model is employed in this paper to study the equilibria of a PBR market in the presence of price-maker agents and pricetaker FR providers. Due to the incorporation of the FR providers' dynamics, the proposed model is reformulated as a mixed-integer linear programming (MILP) problem over innovative mathematical techniques. An optimal equilibrium point is also selected for the market, where none of the agents is unique deviator and the dynamic performance of power system is improved simultaneously. The effectiveness of proposed optimal equilibrium point is evaluated in the numerical results section by comparing the outputs with conventional optimal dispatches of the FR providers.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Deep reinforcement learning assisted co-optimization of Volt-VAR grid service in distribution networks

With the increasing penetration of distributed energy resources in distribution networks, Volt-VAR control and optimization (VVC/VVO) have become very important to ensure an acceptable quality of service to all customers. System operators can rely on slow-responding utility devices, including capacitor banks and on-load tap changing transformers, along with fast-responding battery and photovoltaic (PV) inverters for the VVC/VVO implementation. Because of variations in response time of these two classes of devices, and different control actions (discrete versus continuous), coordinated and optimal scheduling and operation have become of utmost importance. Here, this paper develops a look-ahead deep reinforcement learning (DRL)-based multi-objective VVO technique to improve the voltage profile of active distribution networks, decrease network and inverter power loss, and save the operational cost of the grid. It proposes a deep deterministic policy gradient (DDPG)-based approach to schedule the optimal reactive and/or active power set-points of fast-responding inverters, and a deep Q-network (DQN)-based DRL agent to schedule the discrete decisions variables of slow-responding assets. The reactive power output of PV and battery smart inverters are scheduled at 30-minute intervals and the capacitors’ commitment status is scheduled with several hour intervals. The proposed framework is validated on the modified IEEE 34-bus and 123-bus test cases with embedded PV and PV-plus-storage. To validate the efficacy of the proposed VVO, it is compared with several scenarios, including the base case without VVO, localized droop control of DERs, DDPG-only, and twin delayed DDPG (TD3) agent-based DRL techniques. The results justify the superior performance of the proposed method to improve the voltage profile, reduce network power loss, and minimize the look-ahead grid operational cost while minimizing the undesirable power losses in inverters as a result of power factor adjustments.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Development and Validation of Home Comfort System for Total Performance Deficiency/Fault Detection and Optimal Comfort Control

In this project, we developed and tested a learning-based home thermal model that facilitates the operation of a model predictive control (MPC)-based optimization agent and an automated fault detection and diagnosis (AFDD) agent. The home thermal model was constructed using a two-node resistor-capacitor model. Moreover, two accompanying parameter identification methods were introduced, least-squares and optimization. Based on the home thermal model, the MPC-based optimization agent was developed to optimize residential HVAC operation. Using two FDD methods, the AFDD agent was constructed to detect and diagnose two prevalent residential AC faults, airflow reduction and refrigerant undercharge. The home thermal model, along with the MPC-based optimization agent and AFDD agent, were tested at the Norman Test House, Miami Test House, Pacific Northwest National Laboratory (PNNL) Test House A, and PNNL Test House B. Finally, they were also field tested in nine demonstration homes with real occupants.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Multi-agent motion planning with sporadic communications for collision avoidance

Here, a novel multi-vehicle motion planning and collision avoidance algorithm is proposed and analyzed. The algorithm aims to reduce the amount of onboard calculations and inter-agent communications needed for each vehicle to successfully navigate through an environment with static obstacles and reach their goals. To this end, each agent first calculates a path to the goal by means of an asymptotically optimal rapidly-exploring random tree (RRT*) with respect to the static obstacles. Then, other agents are treated as dynamic obstacles and potential collisions are determined by means of collision cones. Collision cones depend on the position and velocity from other agents and are grown conservatively between inter-agent communications. Based on the available information, each agent determines if a deconfliction maneuver is needed, if it can continue along its current path, or if communication is needed to make a decision about a conflict. With probability one, our algorithm guarantees that the agents keep from colliding with each other. Under an assumption on the existence of a solution for a vehicle to its goal, this algorithm also solves the planning problem with probability one. Simulations illustrate a group of agents successfully reaching their goal configurations and examine how the uncertainty affects the communication frequency of the multi-agent system.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Comprehensive assessment of deep reinforcement learning approaches for economic dispatch in nuclear-driven microgrids

As the electrical grid integrates more variable renewable energy sources such as wind and solar, the demand for distributed and flexible systems to address this increased variability becomes critical. Nuclear-driven microgrids provide a promising solution by offering stable generation to complement intermittent renewables, ensuring grid reliability and operating efficiency. This paper proposes a recurrent deep reinforcement learning framework for optimal economic dispatch in a nuclear-powered microgrid integrating renewable energy sources, small modular reactors, battery storage systems, and balance-of-plant dynamics. A three-agent control architecture is developed, where demand and renewable energy agents act as forecasters, and a reinforcement learning-based dispatch agent performs real-time energy allocation. A nonlinear programming formulation is first used to generate an optimal baseline for benchmarking. The proposed dispatch controller, based on Proximal Policy Optimization enhanced with Long Short-Term Memory networks, exploits temporal correlations in system dynamics by taking advantage of the time series used as inputs to improve policy robustness under uncertainty. Comparative analysis against established deep reinforcement learning methods, including Proximal Policy Optimization with a feedforward architecture, Soft Actor-Critic, and Twin Delayed Deep Deterministic Policy Gradient, demonstrates superior performance. Numerical results indicate that the proposed controller achieves a 0.39% cost reduction relative to the nonlinear programming benchmark and outperforms other learning-based methods by generating additional revenue of up to 0.35%. All reinforcement learning controllers compute dispatch actions in less than 0.3 s, resulting in a computational speedup of more than three orders of magnitude over the nonlinear programming baseline. The findings of this paper highlight their applicability for real-time operation and control in nuclear-integrated microgrids under volatile operating conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Agentic AI vs ML-Based Autotuning: A Comparative Study for Loop Reordering Optimization

High Performance Computing (HPC) applications rely heavily on code optimizations to achieve good performance on modern CPU and GPU architectures. Traditional Machine Learning auto-tuning approaches have demonstrated success in exploring high-dimensional spaces, but they often require expensive compile-run evaluations and lack adaptability for large HPC applications. The recent advances in Large Language Models (LLMs) and Agentic AI systems raise intriguing questions about the potential of these approaches to address specific optimization methodologies. This work aims to answer an essential question for the HPC community: “How Agentic AI Systems Compare to Traditional ML Autotuning Techniques?” To address this question, we present a comparative analysis between a traditional ML-based optimization approach and an Agentic AI system, evaluating their respective capabilities and limitations for loop-level optimization. In addition, we introduced a new Agentic AI system named LoopGen-AI using three different Large Language Models: GPT-4.1, Claude 4.0, and Gemini 2.5. A key finding is that LoopGen-AI achieves competitive per-formance with only a few program runs, the reasoning logs from the agents revealed that their decisions rely heavily on the combination of semantic understanding of the target kernel with dynamic feedback from the environment, highlighting a promising new dimension in performance tuning. In contrast, ML-based autotuners focus on statistical exploration, and require orders of magnitude more runs to reach peak performance. Additionally, our analysis shows that prompt engineering, particularly using Persona + Context Manager patterns, significantly impacts the effectiveness of Agentic AI. Our results indicate that while Agentic AI systems are not yet a complete replacement for ML-based autotuners, it can effectively complement traditional methods.

Rosas, Miguel Romero↗
DOE OSTI2026

AstraAI v1

AstraAI is an open-source, structure-aware AI coding agent designed for large scientific and DOE-HPC codebases such as AMReX-based applications. Unlike general-purpose coding assistants, AstraAI combines retrieval-augmented generation (RAG) with compiler-level Abstract Syntax Tree (AST) analysis to perform precise, scope-constrained code modifications. It identifies exact function spans, enforces locality of edits, and maintains cross-file invariants, enabling deterministic and build-safe transformations in complex C++/GPU environments. AstraAI is intended for developers working on large, evolving HPC frameworks where correctness, reproducibility, and structural integrity are critical. Typical use cases include modifying physics kernels, updating GPU device lambdas, and performing multi-file refactors without breaking compilation or runtime semantics. Compared to conventional LLM-based coding agents - even those with repository access - AstraAI provides structural guarantees rather than free-form text patches. It minimizes unintended diffs, prevents scope drift, preserves formatting and build stability, and reduces structural hallucinations. By integrating compiler tooling directly into the generation loop, AstraAI transforms AI-assisted coding from probabilistic text editing into deterministic, structure-preserving program transformation suitable for mission-critical scientific software.

Natarajan, Mahesh [Lawrence Berkeley National Labo↗
DOE OSTI2025

Precipitation of gadolinium from magnetic resonance imaging contrast agents may be the Brass tacks of toxicity

The formation of gadolinium-rich nanoparticles in multiple tissues from intravenous magnetic resonance imaging contrast agents may be the initial step in rare earth metallosis. The mechanism of gadolinium-induced diseases is poorly understood, as is how these characteristic nanoparticles are formed. Gadolinium deposition has been observed with all magnetic resonance imaging contrast agent brands. Aside from endogenous metals and acidic conditions, little attention has been paid to the role of the biological milieu in the degradation of magnetic resonance imaging contrast agents into nanoparticles. Herein, we describe the decomposition of the commercial magnetic resonance imaging contrast agents Omniscan and Dotarem in the presence of oxalic acid, a well-known endogenous compound. Omniscan dechelated rapidly and preluded measurement by the means available, while Dotarem underwent a two-step decomposition process. The decomposition of both magnetic resonance imaging contrast agents by oxalic acid formed gadolinium oxalate (Gd 2 [C 2 O 4 ] 3 , Gd 2 Ox 3 ). Furthermore, both observed steps of the Dotarem reaction involved the associative addition of oxalic acid. Adding protein (bovine serum albumin) increased the rate of dechelation. Displacement reactions could occur at lysosomal pH. Through these studies, we have demonstrated that magnetic resonance imaging contrast agents can be dissociated by endogenous molecules, thus illustrating a metric by which gadolinium-based contrast agents (GBCAs) might be destabilized in vivo.

Gadodiamide↗
DOE OSTI2023

Non-Fouling, Low Cost Electrolytic Coagulation & Disinfection for Treating Flowback and Produced Water for Reuse

Executive Summary This Final Report is composed of two major sections. The first section presents experimental results obtained in the laboratory during Budget Period 1. The second section presents results from the field test conducted during Budget Period 2. Laboratory Results This research investigated a novel electrochemical process for producing a ferric iron coagulant for use in treating flowback and produced water from hydraulic fracturing and oil production operations. The treatment system improves the effectiveness and lowers the cost of coagulation processes using Fe3+ as the coagulant. The electrolytic coagulant generation (ECG) system uses an electrochemical cell to produce acid and base from oilfield brine solutions. The acid is used to dissolve scrap iron to provide a Fe3+ coagulating agent. The base is used to neutralize the treated water. Compared to conventional electrocoagulation (EC), the main advantage of the ECG system is an order of magnitude lower cost for the source of iron. The second advantage over conventional EC is that it can deliver Fe3+ doses greater than 1 mM, since it is not limited by the amount of dissolved oxygen in the water required to oxidize ferrous to ferric iron. The capital costs for conventional EC and the ECG system are similar, but the operational costs for the ECG system are an order of magnitude lower than conventional EC. The combined costs for iron and electrical energy for treating 1 m3 of FPW with 1 mM Fe3+ is estimated to be $0.87 for conventional EC, and $0.087 for the ECG system. The estimated all-in cost for treating FPW with a 2 mM Fe3+ dose is $0.73/m3 ($0.12/bbl). Field Test Results The field test was conducted at the Paul Foster Central Tank Battery (CTB) in Lea County, New Mexico from November 10, 2022 through December 15, 2022. The feed water to the system was produced water from the Tatanka 1H formation. After approximately two weeks of testing, the initial batch of produced water had been treated and no untreated produced water was available. Thus, after this time, the feed water to the system consisted of previously treated water (i.e., recycled water). The recycled water had nearly all colloidal particles removed, and had a much lower alkalinity due to precipitation of carbonate minerals during the first pass through the system. Although the recycled water was not an ideal test solution due to its low particulate concentrations, its lower alkalinity did allow us to identify the main problem with the treatment system. The main problem with the treatment system was caused by the high alkalinity of the initial feed water (5.4 meq/L) that consumed a significant fraction of the electrochemically generated acid. This resulted in pH values exiting the iron contact tank that were too high to dissolve enough iron to effectively treat the produced water. Tests performed with recycled water with lower alkalinity did not have this issue, and dissolved iron concentrations greater than 20 mM could be achieved. One consistent observation was that effluent water from the iron contact tank was always free of particulates, even when fed with circumneutral solutions. This suggests that there is no need to dissolve high concentrations of iron if all the water to be treated is passed through the scrap iron canister. In this case, dissolved O2 and hypochlorous acid can promote sufficient iron corrosion to provide an effective coagulating agent – even in neutral pH water. This solves the problem resulting from highly alkaline produced water. Modifications to the design of the treatment system were made based on the field test results. These modifications will add minimal additional cost, and were tested in bench-scale laboratory experiments. In short-term testing, the modified treatment process was able to remove colloidal FeS particulates to levels below detection. Long-term, steady state testing will be required to determine whether the modified process is suitable for commercial treatment systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Enabling Real-Time Communication in Multi-Agent Systems: A Graph Neural Network Based Approach

Global connectivity enables effective coordination in Multi-Agent Systems (MAS). Solving these connection problems under hardware constraints is an NP-hard non-Euclidean Degree Constrained Minimum Spanning Tree (DCMST) problem. Prior MAS controllers coordinate team movement for task completion and collision avoidance; some considering Line-of-Sight (LOS) maintenance but prioritizing flexibility over guarantees. Evolutionary Algorithms (EA) have been shown to find good solutions for DCMST, but their performance degrades with larger populations required to support a large MAS. We present a method based on edge graph attention networks, trained offline to reduce online computation times. Empirical comparisons with greedy polynomial-time solvers and EA show that our method leverages latent graph information to consistently find constraint-satisfying solutions in less time.

connectivity maintenance↗
DOE OSTI2021

Integrated Water-Power System Resilience Analysis in a Southeastern Idaho Irrigation District: Minidoka Case Study

This study investigates the joint water–power system resilience of an irrigation district in southeastern Idaho. Irrigation districts face difficulties in the delivery of water to farmers under drought conditions, during equipment failures, or unplanned infrastructure disruptions. The resilience of interconnected water and power systems can be better analyzed and understood through an integrated approach, using a model that connects the dependencies between the two halves of the system. Using a multi-agent system model capturing both water and power system components, as well as their linkages, we capture the interdependencies of these systems and highlight opportunities for improvement when faced with disruptions. Through simulation scenarios, we examine the system resilience using system performance, quantified as the percentage of met demand of the power and water system, when subjected to drought water year, an unforeseen water demand increase, power outage and dam failure. Scenario results indicate that the effects of low flow years are mostly felt in the power system; unexpected increases in water demand marginally impact irrigation system performance; dams and pumps present vulnerabilities of the system, causing substantial unmet demand during disruptions. Noting the interdependencies between the water–power system halves while leveraging an integrated simulation allows for an insightful analysis of the system impacts during disruptions.

13 HYDRO ENERGY↗