Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Improving Primary Frequency Response in Networked Microgrid Operations using Multilayer Perceptron-Driven Reinforcement Learning

Individual microgrids can improve the reliability of power grids during extreme events, and networked microgrids can further improve efficiency through resource sharing and increase the resilience of critical end-use loads. However, networked microgrid operations are subject to large switching transients, which can cause dynamic instability and lead to system collapse. These transients are especially prevalent in microgrids with high penetrations of inverter-connected renewable energy resources, which do not provide the system inertia needed to mitigate the transients. Existing generator controls can be modified to invoke a drop in terminal voltage in response to a frequency deviation, thereby reducing load and improving frequency response. This paper investigates the use of a reinforcement-learning--based controller trained over several switching transient scenarios to modify generator controls during large frequency deviations. Compared to previously used proportional integral controllers, the proposed controller can improve primary frequency response while adapting to changes in system topologies and events.

Radhakrishnan, Nikitha↗
DOE OSTI2025

Reinforcement Learning‐Based Adaptation of Grid Following Inverter's Internal Controller to Networked Microgrids' Strengths

The varying topological configurations, generator commitments and dispatches, and dynamic load demand lead to changing system's strengths during the operations of networked microgrids. When the system's strengths significantly change, the fixed control gains at large devices may result in unsatisfactory system performance; this necessitates the tuning of the control gains at large devices to adapt to the changing system's strengths. In this paper, observer-based reinforcement learning (RL) is utilised to automatically tune the proportional-integral (PI) gains of phase lock loop (PLL) controller of grid-following (GFL) inverters to adapt to the changing strengths of microgrids and networked microgrids. The RL agent in this framework augments an observer predicting system's strengths, from which the RL control policy will adjust accordingly to tune the PLL controller's gains towards the system's strengths. Also, to enhance the control performance, the recently introduced Barrier function-based RL framework is leveraged for the design of reward function to prevent the high frequency nadir. An operational 26 kV electric distribution system, which is modelled as networked microgrids, is used to illustrate the need and effectiveness of the proposed RL-tuned control.

frequency response↗
DOE OSTI2016

On the nature of high field charge transport in reinforced silicone dielectrics: Experiment and simulation

The high field charge injection and transport properties in reinforced silicone dielectrics were investigated by measuring the time-dependent space charge distribution and the current under dc conditions up to the breakdown field and were compared with the properties of other dielectric polymers. It is argued that the energy and spatial distribution of localized electronic states are crucial in determining these properties for polymer dielectrics. Tunneling to localized states likely dominates the charge injection process. A transient transport regime arises due to the relaxation of charge carriers into deep traps at the energy band tails and is successfully verified by a Monte Carlo simulation using the multiple-hopping model. The charge carrier mobility is found to be highly heterogeneous due to the non-uniform trapping. The slow moving electron packet exhibits a negative field dependent drift velocity possibly due to the spatial disorder of traps.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

Tailoring oxidation of aluminum nanoparticles reinforced with carbon nanotubes

In this report, the oxidation temperature and reaction enthalpy of Aluminum (Al) nanoparticles has been controlled by reinforcing with carbon nanotubes. The physical mixing method with ultrasonication was employed to synthesize CNT/Al nanocomposite powders. The micro-morphology of nanoconmposite powders has been analysed by scanning electron microscopy, energy dispersive spectroscopy, Raman spectroscopy and X-ray diffraction techniques. The oxidation behavior of nanocomposite powders analyzed by thermogravimetry/differential scanning calorimertry showed improvement in the exothermic enthalpy. Largest exothermic enthalpy of-1251J/g was observed for CNT (4 wt%)/Al nanocomposite.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2019

A novel use of frequency-banded synthetic aperture focusing technique for reconstructions of alkali-silica reaction in thick-reinforced concrete structures

As they degrade and age, concrete structures are susceptible to the development of multiple defects, such as alkali-silica reaction (ASR). ASR forms a gel that occurs over time between alkaline cement paste and reactive, noncrystalline silica in aggregates [1]. With the continuing extension of the life span of nuclear power plants, it is important to monitor the structural integrity of concrete structures after 40+ years. Compression strength, modulus of elasticity, flexural stiffness, shear strength, and tensile strength are performance characteristics susceptible to the development of ASR. Current methods for monitoring for defects and degradation within thick concrete structures require damaging the concrete. Oak Ridge National Laboratory, in collaboration with the University of Pittsburgh, has developed a nondestructive evaluation (NDE) and reconstruction technique that enables deep-image reconstruction of damaged concrete. The frequency-banded synthetic aperture focusing technique (FB-SAFT) is an augmented SAFT reconstruction that segments the time-series data into different frequency bands using wavelets immediately before performing SAFT reconstructions [2]. This paper presents the NDE instrumentation and reconstruction techniques applied to thick-reinforced concrete structures and concludes with preliminary results.

Ezell, N Dianne↗
DOE OSTI2020

Evolutionary reinforcement learning of dynamical large deviations

In this work, we show how to bound and calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, potentially allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces, the evolutionary process acts directly on rates, and for models with large state spaces, the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Dynamically reinforced heterogeneous grain structure prolongs ductility in a medium-entropy alloy with gigapascal yield strength

Ductility, i.e., uniform strain achievable in uniaxial tension, diminishes for materials with very high yield strength. Even for the CrCoNi medium-entropy alloy (MEA), which has a simple face-centered cubic (FCC) structure that would bode well for high ductility, the fine grains processed to achieve gigapascal strength exhaust the strain hardening ability such that, after yielding, the uniform tensile strain is as low as ~2%. Here we purposely deploy, in this MEA, a three-level heterogeneous grain structure (HGS) with grain sizes spanning the nanometer to micrometer range, imparting a high yield strength well in excess of 1 GPa. This heterogeneity results from this alloy’s low stacking fault energy, which facilitates corner twins in recrystallization and stores deformation twins and stacking faults during tensile straining. After yielding, the elastoplastic transition through load transfer and strain partitioning among grains of different sizes leads to an upturn of the strain hardening rate, and, upon further tensile straining at room temperature, corner twins evolve into nanograins. This dynamically reinforced HGS leads to a sustainable strain hardening rate, a record-wide hysteresis loop in load–unload–reload stress–strain curve and hence high back stresses, and, consequently, a uniform tensile strain of 22%. As such, this HGS achieves, in a single-phase FCC alloy, a strength–ductility combination that would normally require heterogeneous microstructures such as in dual-phase steels.

36 MATERIALS SCIENCE↗
DOE OSTI2019

A review of long fibre thermoplastic (LFT) composites [A review of long fibre-reinforced thermoplastic of long fibre thermoplastic (LFT) composites]

Long fibre-reinforced thermoplastic or long fibre thermoplastic (LFT) composites possess superior specific modulus and strength, excellent impact resistance, and other advantages such as ease of processability, recyclability, and excellent corrosion resistance. These advantages make LFT composites one of the most advanced lightweight engineering materials and enable their increasing use in various applications. This review paper summarizes the research and development work that has been conducted on LFT composites since their initial development. Different aspects of LFTs, such as process development, fibre orientation distribution (FOD), fibre length distribution (FLD), and their effects on the mechanical properties of LFT composites are described. The characterization of the FOD and FLD in the LFT composites using advanced imaging technology such as high-resolution 3D micro-CT scanning technique is summarized. Research and development of LFT hybridisation and LFT additives are also discussed. Lastly, conclusions are made and the future outlook of LFT composites is given.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Distributed deep reinforcement learning for simulation control

Abstract Several applications in the scientific simulation of physical systems can be formulated as control/optimization problems. The computational models for such systems generally contain hyperparameters, which control solution fidelity and computational expense. The tuning of these parameters is non-trivial and the general approach is to manually ‘spot-check’ for good combinations. This is because optimal hyperparameter configuration search becomes intractable when the parameter space is large and when they may vary dynamically. To address this issue, we present a framework based on deep reinforcement learning (RL) to train a deep neural network agent that controls a model solve by varying parameters dynamically. First, we validate our RL framework for the problem of controlling chaos in chaotic systems by dynamically changing the parameters of the system. Subsequently, we illustrate the capabilities of our framework for accelerating the convergence of a steady-state computational fluid dynamics solver by automatically adjusting the relaxation factors of the discretized Navier–Stokes equations during run-time. The results indicate that the run-time control of the relaxation factors by the learned policy leads to a significant reduction in the number of iterations for convergence compared to the random selection of the relaxation factors. Our results point to potential benefits for learning adaptive hyperparameter learning strategies across different geometries and boundary conditions with implications for reduced computational campaign expenses 4 4 Data and codes available at https://github.com/Romit-Maulik/PAR-RL . .

42 ENGINEERING↗
DOE OSTI2021

Molecular View on Mechanical Reinforcement in Polymer Nanocomposites

The microscopic origin of mechanical enhancement in polymer nanocomposite (PNC) melts is investigated through the combination of rheology and small-angle neutron scattering. It is shown that in the absence of an extensive particle network, the molecular deformation of polymer chains dominates the stress response on intermediate time scales. Quantitative analyses of small-angle neutron scattering spectra, however, reveal no enhanced structural anisotropy in the PNCs, compared with the pristine polymers under the same deformation conditions. These results can demonstrate that the mechanical reinforcement of PNCs is not due to molecular overstraining, but instead a redistribution of strain field in the polymer matrix, akin to the classical picture of hydrodynamic effect of nanoparticles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Adaptive Reinforcement Learning (ARL) Control of a Multi-port Resonant Converter in UAV Systems

This study presents an adaptive reinforcement learning (ARL) control framework for a multi-port resonant converter used in hybrid unmanned aerial vehicle (UAV) power systems. The converter integrates high-frequency half-bridge input ports connected to a rectified engine–generator set and a battery energy storage system, along with a semi-bridgeless active rectifier supplying the propulsion load. A deep RL agent is trained to dynamically regulate inter-port phase-shift commands in real time based on flight conditions and load power demand. The ARL controller autonomously identifies phase-shift combinations that maximize conversion efficiency while maintaining stable and coordinated power flow, even under rapidly varying operating scenarios. This data-driven approach eliminates the need for explicit system modeling or extensive manual tuning and enables coordinated control among multiple power ports without inter-port communication. Experimental results validate that the ARL based strategy achieves reliable power sharing and consistently high-efficiency operation across diverse UAV operating conditions.

Asa, Erdem [ORNL] (ORCID:0000000190884812)↗
DOE OSTI2025

Reinforcement Learning Control for Buildings Co-Optimizing Energy, Comfort, and Indoor Air Quality: An Annual Assessment

Efficient control of Heating, Ventilation, and Air Conditioning (HVAC) systems is crucial for optimizing energy use and maintaining indoor comfort in buildings. Traditional control methods, such as PID control, cannot handle energy use trade-offs among multiple components in the building energy system at a supervisory level. Reinforcement learning (RL) presents a promising solution, offering adaptive and data-driven control strategies that optimize performance over time. However, RL also faces several challenges, including the conflicts encountered in co-optimizing energy savings, occupant comfort, and indoor air quality, and the requirement for extensive interactions with the environment in training. We proposed a flexible simulation platform that integrates a hybrid model for RL training and designed an RL agent to control the entire central HVAC system, focusing on co-optimizing energy consumption, thermal comfort, and indoor air quality ($\text{CO}_{2}$ and PM2.5 concentrations). Finally, we evaluated the RL agent's performance over an annual cycle. Our findings indicate that the RL agent can effectively manage the HVAC system with 14.7 % energy savings annually and balance multiple objectives, which demonstrates significant potential for improving HVAC system control and sustainability in buildings.

Guo, Fangzhou↗
DOE OSTI2021

Barrier Function-based Reinforcement Learning for Emergency Control of Power Systems

Under voltage load shedding has been considered as a standard and effective measure to recover the voltage stability of the electric power grid under emergency and severe conditions. However, this scheme usually trips a massive amount of load which can be unnecessary and harmful to customers. Recently, deep reinforcement learning (RL) has been regarded and adopted as a promising approach that can significantly reduce the amount of load shedding. However, like most existing machine learning (ML)-based control techniques, RL control usually cannot guarantee the safety of the systems under control. In this paper, we introduce a novel safe RL method for load shedding of power systems, that can enhance the safe voltage recovery of the electric power grid after experiencing faults. Unlike standard RL method, the safe RL method integrates a Barrier function into the reward function. Consequently, the optimal control policy can render the power system to avoid the safety bounds. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark is performed to demonstrate the effectiveness of the proposed safe RL emergency control, as well as its adaptive capability to faults not seen in the training.

Vu, Thanh Long↗
DOE OSTI2020

Optimizing Individualized Treatment Planning for Parkinson’s Disease Using Deep Reinforcement Learning

More than one million people currently live with Parkinson's Disease (PD) in the U.S. alone. Medications, such as levodopa, can help manage PD symptoms. However, medication treatment planning is generally based on patient history and limited interaction between physicians and patients during office visits. This limits the extent of benefit that may be derived from the treatment as disease/patient characteristics are generally non-stationary. Wearable sensors that provide continuous monitoring of various symptoms, such as bradykinesia and dyskinesia, can enhance symptom management. However, using such data to overhaul the current static medication treatment planning approach and prescribe personalized medication timing and dosage that accounts for patient/care-giver/physician feedback/preferences remains an open question. We develop a model to prescribe timing and dosage of medications, given the motor fluctuation data collected using wearable sensors in real-time. We solve the resulting model using deep reinforcement learning (DRL). The prescribed policy determines the optimal treatment plan that minimizes patient's symptoms. Our results show that the model-prescribed policy outperforms the static a priori treatment plan in improving patients' symptoms, providing a proof-of-concept that DRL can augment medical decision making for treatment planning of chronic disease patients.

Watts, Jeremy↗
DOE OSTI2024

Deep Multi-Agent Reinforcement Learning for Real-World Signalized Traffic Corridor Control

Signalized traffic control problem has been addressed recently with deep Reinforcement Learning (RL) approaches involving diverse state, action, and reward structures. While significant progress has been noted in the literature, open challenges still remain in the areas of adaptive signal phase timing, coordination in a multi-intersection corridor setting, and consideration of real-world traffic conditions. In the context of deep RL-based problem framing, extensions are needed that enable adaptive signal phase timings in an intersection agent's action space, computationally efficient information sharing among neighboring signalized intersection agents along a corridor, and experimentation in realistic simulation environments. In this paper, we develop a deep Advantage Actor Critic (A2C) multi-agent RL (MARL) approach capturing the research extensions above and apply it within a real-world calibrated Aimsun Next traffic corridor simulation model based on traffic data from the City of Coral Gables, Florida. For a multi-intersection corridor control setting, our numerical simulation experiments with a decentralized A2C MARL algorithm applied at different time periods led to a total average corridor travel delay reduction (expressed in seconds/mile averaged over vehicles) from 4.9% to 19.9% compared to state-of-the-art actuated control.

Shuvo, Salman S. [BATTELLE (PACIFIC NW LAB)]↗
DOE OSTI2024

Optimal Management of Grid-Interactive Efficient Buildings via Safe Reinforcement Learning

Reinforcement learning (RL)-based methods have achieved significant success in managing grid-interactive efficient buildings (GEBs). However, RL does not carry intrinsic guarantees of constraint satisfaction, which may lead to severe safety consequences. Besides, in GEB control applications, most existing safe RL approaches rely only on the regularisation parameters in neural networks or penalty of rewards, which often encounter challenges with parameter tuning and lead to catastrophic constraint violations. To provide enforced safety guarantees in controlling GEBs, this paper designs a physics-inspired safe RL method whose decision-making is enhanced through safe interaction with the environment. Different energy resources in GEBs are optimally managed to minimize energy costs and maximize customer comfort. The proposed approach can achieve strict constraint guarantees based on prior knowledge of a set of developed hard steady-state rules. Simulations on the optimal management of GEBs, including heating, ventilation, and air conditioning (HVAC), solar photovoltaics, and energy storage systems, demonstrate the effectiveness of the proposed approach.

Huo, Xiang↗
DOE OSTI2022

An Efficient Distributed Reinforcement Learning for Enhanced Multi-Microgrid Management

Economic dispatch in multi-microgrid (MMG) systems requires coordinating distributed energy resources (DERs) of different microgrids, which leads to a significant increase in the number of states for energy management. In these cases, traditional reinforcement learning (RL) approaches become computationally expensive or output a solution that causes extra-operating costs for the system. This paper proposes an RL approach that employs local learning agents to interact with microgrid environments in a distributed manner and aggregates the outcomes to train the global agent to learn the policy for the MMG system. This distributed exploration and aggregation process provides an effective solution and guides the global agent to learn the dispatch policy efficiently. Case studies are performed on a system with three microgrids with different types of DERs. Results obtained using the proposed RL and comparisons with conventional methods substantiate the effectiveness of the proposed approach in terms of operation costs, computation time, and peak-to-average ratio.

Das, Avijit↗
DOE OSTI2020

DeepGrid: Robust Deep Reinforcement Learning-based Contingency Management

Increasing uncertainty raised by the integration of renewable energy resources requires an enormous number of simulations to be carried out for the security assessment of the power grid. However, it is challenging to assess the steady-state and dynamic security indices for different system contingency events by doing an exhaustive analysis in real-time due to the computational and communication constraints. One promising solution is using data-driven techniques along with the system models to train an intelligent contingency management framework to better handle the contingencies in real-time. Nevertheless, implementing a data-driven technique to obtain the best remedial actions necessitates to account for the effect of the measurement noise on the performance of the contingency management. To tackle these challenges, we leverage a robust deep reinforcement learning (DRL) algorithm called Double Deep Q-Network (DDQN) to design a recommender system capable of prescribing optimal control actions with the help of the real-time digital simulator (RTDS). The use of RTDS system in combination with the advanced DRL algorithm allows to explore a wide variety of system contingencies in order to derive better remedial actions. The performance of the proposed algorithm is evaluated in IEEE 9-bus system under different loading conditions, and different network configurations in presence of noisy measurements.

Ghasemkhani, Amir↗