Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Reinforcement Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Reinforcement learning pulses for transmon qubit entangling gates

The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Entanglement engineering of optomechanical systems by reinforcement learning

Entanglement is fundamental to quantum information science and technology, yet controlling and manipulating entanglement—so-called entanglement engineering—for arbitrary quantum systems remains a formidable challenge. There are two difficulties: the fragility of quantum entanglement and its experimental characterization. We develop a model-free deep reinforcement-learning (RL) approach to entanglement engineering, in which feedback control together with weak continuous measurement and partial state observation is exploited to generate and maintain desired entanglement. We employ quantum optomechanical systems with linear or nonlinear photon–phonon interactions to demonstrate the workings of our machine-learning-based entanglement engineering protocol. In particular, the RL agent sequentially interacts with one or multiple parallel quantum optomechanical environments, collects trajectories, and updates the policy to maximize the accumulated reward to create and stabilize quantum entanglement over an arbitrary amount of time. The machine-learning-based model-free control principle is applicable to the entanglement engineering of experimental quantum systems in general.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Safe Deep Reinforcement Learning for Active Distribution System Model Predictive Control with EVs and DERs

The temporal and spatial mismatch between PV generation and electric vehicle (EV) charging and discharging may cause voltage violations in active distribution networks. Despite the widespread use of deep reinforcement learning (DRL) in power system optimization and control, it lacks guarantees on constraint satisfaction during both training and deployment. This paper proposes a Lagrangian-based safe DRL approach for model predictive control (MPC) of active distribution systems with large-scale integration of PVs, EVs, and energy storage systems (ESSs). A Transformer-LSTM time-series model is proposed to forecast EV charging demand, which is then formulated as a constraint to ensure charging requirements are met. Using this prediction, a Lagrangian-based safe soft actor-critic (SAC) framework is developed for real-time control in a three-phase unbalanced distribution system, enforcing voltage safety constraints while optimizing the cumulative net reward. By integrating the forecasting model with multi-period constraints, the proposed framework jointly coordinates PV systems, EV charging and discharging, and ESS scheduling within the MPC horizon. Numerical experiments on a modified IEEE 123-bus system with real-world data show that, under a high PV penetration scenario, the proposed method increases the net reward by 30.74% and reduces average voltage violations from 0.0011 p.u. to 0.0002 p.u. compared with standard SAC. Compared with the optimal power flow (OPF) approach, it achieves similar voltage security while yielding lower line losses. It also maintains real-time control capability, reducing operation latency to 53.21 ms per 15-minute control interval. The proposed method remains effective under varying PV/EV penetrations and load conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Adaptive Reinforcement Learning Control for Power Distribution in Multi-Output Resonant Converters

This paper presents an adaptive reinforcement learning (ARL)-based control framework for efficient power distribution in a multi-output resonant converter for UAV applications. The proposed system is based on a high-frequency isolated resonant architecture, where a single energy source supplies multiple propulsion loads through independently controlled output rectifiers, addressing the need for coordinated multi-motor power management. The ARL framework dynamically allocates output power by learning optimal phase-shift control actions under varying load demands and operating conditions. The agent autonomously determines control parameters that maximize conversion efficiency while ensuring accurate power sharing among multiple outputs. In addition, the proposed approach enables adaptive operation without requiring detailed system modeling or manual tuning. Experimental results demonstrate stable and efficient performance over a wide range of operating conditions, confirming the effectiveness and robustness of the learning-based control strategy for multi-output resonant converter system.

Asa, Erdem [ORNL] (ORCID:0000000190884812)↗
DOE OSTI2025

Optimizing on-ramp merging for connected and automated vehicles: A hierarchical approach using deep reinforcement learning and optimal control

On-ramp merging for Connected and Automated Vehicles (CAVs) presents significant challenges in dynamic traffic environments. Traditional methods and recent learning-based approaches often fail to simultaneously address decision-making complexity and execution precision under fluctuating conditions. This study introduces a novel hierarchical framework that combines: (1) a high-level Deep Reinforcement Learning (DRL) module that coordinates merging sequences through Virtual Traffic Signals (VTS) with Yield/Green phases and (2) a low-level optimal controller generating collision-free speed trajectories via pseudospectral convex optimization. A convolutional autoencoder compresses high-dimensional traffic states to enhance responsiveness. Extensive simulations demonstrate a 12.5% improvement in mainline throughput a 28% reduction in emergency braking events, and 31.66% lower fuel consumption compared to baseline methods. Furthermore, the framework’s effectiveness in coordinating CAV merges highlights its potential for real-world deployment. Future work will extend validation to multi-lane scenarios with mixed traffic and large-scale multiple merging points.

Connected and automated vehicles↗
DOE OSTI2025

Development and assessment of hierarchical multi-reward reinforcement learning based potential for silicene with state-of-the-art models

We develop a new interatomic force field for Silicene, a 2D material with a buckled hexagonal lattice structure with high polymorphism. We introduce new parameterizations of a Tersoff model using a hierarchical multi-reward reinforcement learning (RL) methodology coupled with a continuous Monte Carlo Tree Search optimization. Our model significantly outperforms existing methods by enhancing the accuracy of predictions for the structural and thermodynamic properties of seven silicene polymorphs-including structure, energy, equation of state, elasticity, and phonon dispersion-when compared to established models. We further make a comprehensive comparison of the various models in predicting the mechanical and thermal properties of silicene. We trace the origin of the improved performance to the description of the angular dependence in the bond-order term, suggesting that modifying the angular terms in short-range models is essential to capture the structural diversity in low dimensional systems.

2D materials↗
DOE OSTI2024

A safe reinforcement learning algorithm for supervisory control of power plants

Traditional control theory-based methods require tailored engineering for each system and constant fine-tuning. In power plant control, one often needs to obtain a precise representation of the system dynamics and carefully design the control scheme accordingly. Model-free Reinforcement learning (RL) has emerged as a promising solution for control tasks due to its ability to learn from trial-and-error interactions with the environment. It eliminates the need for explicitly modeling the environment’s dynamics, which is potentially inaccurate. However, the direct imposition of state constraints in power plant control raises challenges for standard RL methods. To address this, we propose a chance-constrained RL algorithm based on Proximal Policy Optimization for supervisory control. Our method employs Lagrangian relaxation to convert the constrained optimization problem into an unconstrained objective, where trainable Lagrange multipliers enforce the state constraints. In conclusion, our approach achieves the smallest distance of violation and violation rate in a load-follow maneuver for an advanced Nuclear Power Plant design.

constrained optimization↗
DOE OSTI2025

Topology-Aware Reinforcement Learning for Voltage Control: Centralized and Decentralized Strategies

Volt-VAR control (VVC) methods based on deep reinforcement learning (DRL) can effectively control distribution grid voltage and minimize power loss by implementing corrective and preventive control measures on the reactive power output of inverter-based distributed energy resources (DERs). However, model-free DRL-based VVC approaches usually cannot capture the important topological feature of the power system since they use a fully-connected network (FCN) to deliver the action. Therefore, this paper proposes a graph convolutional network (GCN)-based DRL approach that can employ the topological information of the network to take better control action for regulating the voltage. Our implementation allows for both centralized and decentralized configurations, utilizing a single agent and multiple agents respectively. Although the centralized GCN-based DRL approach has its advantages of minimizing voltage fluctuation and power loss, it is not suitable for large scale power systems due to its challenges in terms of scalability, computation speed and potential single points of failure. Therefore, these problems can be resolved using the decentralized GCN-based DRL approach. Moreover, to ensure the safe operation of the model, our proposed approach incorporates an exponential barrier function while formulating the reward function for each agent. To validate performance of the proposed approaches, the proposed model is tested on modified IEEE test systems and the performances are measured in terms on voltage fluctuation reduction, minimization of power loss and computational speed. Finally, the results show that the proposed topology-aware approach outperforms the FCN-based DRL approach in terms of reducing voltage fluctuation and minimizing power loss of the network. Moreover, it is shown that the decentralized GCN-based DRL has faster computational speed than other approaches.

42 ENGINEERING↗
DOE OSTI2026

MSD CoP Webinar: Modeling the Operations of Reservoir Systems with LLMs and Inverse Reinforcement Learning

Context: This panel featured three presentations centered on the common theme of applying LLMs and inverse reinforcement learning (IRL) to capture the complex human-environment interactions that are central to the operation of reservoir systems. Dr. Wyatt Arnold will kick off the webinar with a talk on how analyzing LLM chain-of-thought reasoning reveals sophisticated quantitative justification and risk awareness, showing promise as a bridge between quantitative models and value-driven water management decisions. Next, Dr. Matteo Giuliani will build on this with a discussion demonstrating that AI- and IRL-driven approaches can infer the trade-offs between flood control and water supply using historical observations. Finally, Dr. Rohan Singh Wilkho will close the webinar with a talk establishing IRL as a generalizable diagnostic tool for decoding decision-making in managed hydrologic and human-infrastructure systems. Across the three presentations, the application of LLMs and IRL opens new possibilities for the development of adaptive, transparent, and human-aware models supporting water management in an increasingly uncertain future. Presenters: Wyatt Arnold (Politecnico di Milano); Matteo Giuliani (Politecnico di Milano); Rohan Singh Wilkho (Cornell University) Moderator: Patrick M. Reed (MSD CoP Facilitation Team); Stefano Galelli (MSD CoP AI Working Group Co-Chair); David Gold (MSD CoP AI Working Group Co-Chair) This webinar was held on: June 23rd, 2026 from 12-1 PM EST.

Arnold, Wyatt [Politecnico di Milano]↗
DOE OSTI2026

Optimization of the FRIB beam dump: a hybrid genetic algorithm and reinforcement learning approach

The operational envelope of high-power-density systems, such as particle accelerators and advanced nuclear energy systems, is critically constrained by the need to manage extreme thermal loads. To address this, we present a novel hybrid optimization framework combining a genetic algorithm (GA) with a soft actor-critic (SAC) deep reinforcement learning agent. This framework was applied to a practical high-heat-flux problem: redesigning the beam dump at the Facility for Rare Isotope Beams (FRIB) for a power upgrade from 20 kW to 50 kW. The resulting design, validated by three-dimensional conjugate heat transfer simulations, suppresses hazardous hot spots and yields a markedly more uniform temperature distribution. This provides a robust operating margin, increasing the average power-handling capability by 72% relative to the current design, demonstrating the framework’s potential to solve complex thermal management challenges in both accelerator technology and advanced nuclear systems.

Accelerator↗
DOE OSTI2026

Technical Report for Bayesian Optimization and Reinforcement Learning for Beam Polarization Increase in the BNL Hadron Injectors

This project developed and evaluated physics-informed Bayesian learning and machine learning (ML)-based optimization methods for improving beam polarization preservation in the BNL hadron injector chain. The work focused on uncertainty-aware digital twin modeling, Bayesian calibration of accelerator simulations using beam measurements, and data-efficient optimization strategies including Bayesian optimization and reinforcement learning. These methods were applied to injector tuning and RF control problems in realistic accelerator settings to support improved operational robustness and readiness for RHIC operations and future Electron–Ion Collider facilities. No subject inventions were disclosed under this award.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Deep reinforcement learning for optimal control of induction welding process

Optimizing induction welding (IW) process parameters for the application of joining thermoplastic composites is challenging as it requires achieving complex spatiotemporal thermal characteristics along the weld-line to obtain desired weld quality. We formulate an optimal control problem which captures these requirements and seeks to optimize the IW coil speed using a fast-acting dynamic IW process model. We develop a novel Deep Reinforcement Learning (DRL) framework to solve this computationally challenging control problem and demonstrate via simulation study that the learned DRL feedback control policy results in better spatiotemporal thermal characteristics as compared to the current state-of-the-art.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Designing reinforcement learning algorithms for building HVAC control: From experimental observation to simulation comparisons

Advanced supervisory-level control with reinforcement learning (RL) is regarded as a promising solution for HVAC systems to minimize energy consumption while maintaining thermal comfort and indoor air quality. However, most RL applications were conducted in the simulation environment rather than real-world HVAC systems. This paper developed a value-based RL controller termed Deep Q-Network (DQN) for a typical central HVAC system and evaluated its performance in a building test facility. By comparing DQN with a rule-based controller, the study not only demonstrated the cases where DQN could properly maintain indoor comfort but also discussed possible reasons why DQN failed in some other situations. Recognizing the limitations of value-based RL algorithms from the experimental tests, a simulation study was conducted to compare DQN with an alternative RL approach, an actor–critic algorithm termed Deep Deterministic Policy Gradient (DDPG). In scenarios with a relatively large action space, DDPG outperformed DQN by requiring fewer computational resources and achieving better thermal comfort, lower energy consumption, and more stable control actions. The findings suggest that the ability of DDPG to handle continuous control variables more effectively allows for faster convergence in training and more precise control in practice, which enhances the overall efficiency and reliability of the HVAC system.

Guo, Fangzhou↗
DOE OSTI2026

Adaptive Reinforcement Learning (ARL) Control of a Multi-port Resonant Converter in UAV Systems

This study presents an adaptive reinforcement learning (ARL) control framework for a multi-port resonant converter used in hybrid unmanned aerial vehicle (UAV) power systems. The converter integrates high-frequency half-bridge input ports connected to a rectified engine–generator set and a battery energy storage system, along with a semi-bridgeless active rectifier supplying the propulsion load. A deep RL agent is trained to dynamically regulate inter-port phase-shift commands in real time based on flight conditions and load power demand. The ARL controller autonomously identifies phase-shift combinations that maximize conversion efficiency while maintaining stable and coordinated power flow, even under rapidly varying operating scenarios. This data-driven approach eliminates the need for explicit system modeling or extensive manual tuning and enables coordinated control among multiple power ports without inter-port communication. Experimental results validate that the ARL based strategy achieves reliable power sharing and consistently high-efficiency operation across diverse UAV operating conditions.

Asa, Erdem [ORNL] (ORCID:0000000190884812)↗
DOE OSTI2025

CyRRL (Cyber Resilient Reinforcement Learning for grid voltage control) [SWR-24-115]

This codebase contains a multi-agent, actor-critic reinforcement learning implementation for cyber-resilient grid voltage control. It uses a 123-bus OpenDSS system as the environment, with three-phase power flow translating nodal power injections into solved nodal voltages. The reward function penalizes deviations from nominal voltage as well as reactive power dispatch, while encouraging agents to take actions that result in fast convergence to nominal conditions. The codebase models false data injection attacks and includes functionality for training, testing, hyper-parameter tuning, and visualization.

Murphy, Sinnott [National Renewable Energy Laborat↗
DOE OSTI2024

Visibility-enhanced model-free deep reinforcement learning algorithm for voltage control in realistic distribution systems using smart inverters

Increasing integration of distributed solar photovoltaic (PV) into distribution networks could result in adverse effects on grid operation. Traditional model-based control algorithms require accurate model information that is difficult to acquire and thus are challenging to implement in practice. Here, this paper proposes a surrogate model-enabled grid visibility scheme to empower deep reinforcement learning (DRL) approach for distribution network voltage regulation using PV inverters with minimal system knowledge. In contrast to existing DRL methods, this paper presents and corroborates the adverse impact of missing load information on DRL performance and, based on this finding, proposes a surrogate model methodology to impute load information utilizing observable data. Additionally, a multi-fidelity neural network is utilized to construct the DRL training environment, chosen for its efficient data utilization and enhanced robustness to data uncertainty. The feasibility and effectiveness of the proposed algorithm are assessed by considering DRL testing across varying degrees of observable load information and diverse training environments on a realistic power system.

14 SOLAR ENERGY↗
DOE OSTI2026

Hierarchical Reinforcement Learning of a Short-Range Bond-Order Potential for Silica: Analytic Embedding of Coordination with Classical Efficiency

Reinforcement learning (RL) has recently emerged as a data-efficient strategy to parametrize short-range interatomic potentials. Building on our past RL optimization of pairwise silica models, we extend the framework to a bond-order (Tersoff-type) potential that provides an analytic embedding of local coordination through a three-body term. A hierarchical RL workflow combining continuous-action Monte Carlo Tree Search and property-based rewards efficiently explores the 26-dimensional parameter space, sequentially optimizing lattice parameters, densities, angles, and cohesive energies of 21 silica polymorphs. The resulting models, Q-Tersoff and ML-Tersoff, reproduce the energetic ordering of low-energy phases and capture the angular correlations and amorphous structure factors of silica with improved fidelity over pairwise force fields, while remaining orders of magnitude faster than high-dimensional machine-learned potentials. Both models underperform for elastic constants and high-energy frameworks, delineating the limits of the current analytic form. The approach establishes a general and interpretable route to angle-aware, short-range potentials that bridge physics-based and machine-learned descriptions of silicate materials.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Deep Multi-Agent Reinforcement Learning for Real-World Signalized Traffic Corridor Control

Signalized traffic control problem has been addressed recently with deep Reinforcement Learning (RL) approaches involving diverse state, action, and reward structures. While significant progress has been noted in the literature, open challenges still remain in the areas of adaptive signal phase timing, coordination in a multi-intersection corridor setting, and consideration of real-world traffic conditions. In the context of deep RL-based problem framing, extensions are needed that enable adaptive signal phase timings in an intersection agent's action space, computationally efficient information sharing among neighboring signalized intersection agents along a corridor, and experimentation in realistic simulation environments. In this paper, we develop a deep Advantage Actor Critic (A2C) multi-agent RL (MARL) approach capturing the research extensions above and apply it within a real-world calibrated Aimsun Next traffic corridor simulation model based on traffic data from the City of Coral Gables, Florida. For a multi-intersection corridor control setting, our numerical simulation experiments with a decentralized A2C MARL algorithm applied at different time periods led to a total average corridor travel delay reduction (expressed in seconds/mile averaged over vehicles) from 4.9% to 19.9% compared to state-of-the-art actuated control.

Shuvo, Salman S. [BATTELLE (PACIFIC NW LAB)]↗