Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Optimizing on-ramp merging for connected and automated vehicles: A hierarchical approach using deep reinforcement learning and optimal control

On-ramp merging for Connected and Automated Vehicles (CAVs) presents significant challenges in dynamic traffic environments. Traditional methods and recent learning-based approaches often fail to simultaneously address decision-making complexity and execution precision under fluctuating conditions. This study introduces a novel hierarchical framework that combines: (1) a high-level Deep Reinforcement Learning (DRL) module that coordinates merging sequences through Virtual Traffic Signals (VTS) with Yield/Green phases and (2) a low-level optimal controller generating collision-free speed trajectories via pseudospectral convex optimization. A convolutional autoencoder compresses high-dimensional traffic states to enhance responsiveness. Extensive simulations demonstrate a 12.5% improvement in mainline throughput a 28% reduction in emergency braking events, and 31.66% lower fuel consumption compared to baseline methods. Furthermore, the framework’s effectiveness in coordinating CAV merges highlights its potential for real-world deployment. Future work will extend validation to multi-lane scenarios with mixed traffic and large-scale multiple merging points.

Connected and automated vehicles↗
DOE OSTI2023

Reinforcement learning building control approach harnessing imitation learning

Reinforcement learning (RL) has shown significant success in sequential decision making in fields like autonomous vehicles, robotics, marketing and gaming industries. This success has attracted the attention to the RL control approach for building energy systems which are becoming complicated due to the need to optimize for multiple, potentially conflicting, goals like occupant comfort, energy use and grid interactivity. However, for real world applications, RL has several drawbacks like requiring large training data and time, and unstable control behavior during the early exploration process making it infeasible for an application directly to building control tasks. To address these issues, an imitation learning approach is utilized herein where the RL agents starts with a policy transferred from accepted rule based policies and heuristic policies. This approach is successful in reducing the training time, preventing the unstable early exploration behavior and improving upon an accepted rule-based policy - all of these make RL a more practical control approach for real world applications in the domain of building controls.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Energy performance evaluation of the ASHRAE Guideline 36 control and reinforcement learning–based control using field measurements

This study evaluates the energy performance of ASHRAE Guideline 36–compliant control (ASHRAE 36 control) and reinforcement learning (RL)–based control through experimental field tests and a simulation study. Three field tests were conducted at Oak Ridge National Laboratory’s commercial building test facility in Oak Ridge, Tennessee: a baseline with a baseline conventional control, a test with ASHRAE 36 control, and a test with RL-based control. The selected ASHRAE 36 controls were trim and respond control, as well as variable air volume (VAV) box control. We compared the measured supply air temperature of the rooftop unit, VAV box supply air temperature, and VAV box supply airflow rate across the three test cases. The field data indicated that ASHRAE 36 controls operated as specified by ASHRAE Guideline 36. Based on these data, ASHRAE 36 control achieved a 45 % reduction in hourly averaged HVAC energy consumption compared with the baseline, and RL-based control achieved a 66 % reduction. These potential annual energy savings were confirmed using a calibrated whole-building energy model. Compared with the baseline, ASHRAE 36 control reduced HVAC energy consumption by 42 %, and RL-based control achieved a 54 % reduction. Furthermore, RL-based control reduced total HVAC energy consumption by 21 % more than ASHRAE 36 control.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

A reinforcement learning approach to long-horizon operations, health, and maintenance supervisory control of advanced energy systems

In this work, we develop a Reinforcement Learning (RL) approach to the supervisory control problem for advanced energy systems, such as novel nuclear reactors and other demand-driven, mission-critical, and component-health-sensitive energy plants. The inclusive problem landscape considered captures the stochastic confluence of plant performance, component health evolution, power demand from the grid, diverse maintenance actions, and operator-defined goals and constraints, all considered over meaningfully long-enough reasoning horizons. Key aspects of the proposed approach are a receding horizon control-inspired technique dictating time- or event-triggered supervisory policy (re-)constructions, as well as additional capability-enabling contributions such as timescale compression, to handle long reasoning horizons and uncertainty in parts of the problem, and practical yet demonstrably-effective handling of hybrid action spaces with continuous and discrete decision variables. The resulting algorithm consists of a simulation-based RL agent constructing stochastic supervisory control policies over nontrivial action spaces and for long horizons, applying the learned policy to the system for a much shorter interval, and perpetually repeating, to construct the next long-horizon policy. That next policy will only be applied, again, for a short interval, yet originally far-in-time events move progressively closer, their associated uncertainty decreases, and new events and aspects enter the reasoning horizon. The proposed methodology bridges fundamental receding horizon concepts with the unequivocally stronger and more scalable reasoning of contemporary RL. Numerical examples using Soft Actor–Critic Deep RL illustrate the operation and efficacy of the proposed technique for a power plant tasked with health-aware load following missions in a dynamic electricity market landscape.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Multi-task deep reinforcement learning for intelligent multi-zone residential HVAC control

In this short communication, a data-driven deep reinforcement learning (deep RL) method is applied to minimize HVAC users’ energy consumption costs while maintaining users’ comfort. The applied deep RL method's efficiency is enhanced by conducting multi-task learning that can achieve an economic control strategy for a multi-zone residential HVAC system in both cooling and heating scenarios. The applied multi-task deep RL method is compared with a rule-based benchmark case and a single-task deep deterministic policy gradient algorithm to verify its effective and generalized application in optimizing HVAC operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Deep reinforcement learning (DRL) algorithms have successfully solved many challenging problems in various power system control scenarios. However, their decision-making process is usually regarded as black-boxes. Furthermore, how DRL models interact with human intelligence remains an open problem. Thus, this paper proposes a policy extraction framework to extract a complex DRL model into an explainable policy. This framework includes three parts: 1) DRL training and data generation. We train an agent for a specific control task and generate data, which contains the control policy of the agent. 2) Policy extraction. We propose an information gain rate based weighted oblique decision tree (IGR-WODT) for DRL policy extraction. 3) Policy evaluation. We define three metrics to evaluate the performance of the proposed approach. A case study for the under-voltage load shedding problem shows that the IGR-WODT presents a performance enhancement compared with DRL, weighted oblique decision tree, and univariate decision tree. The proposed policy extraction method could provide an intuitive explanation of the neural network decision-making process to the dispatchers when making final decisions on power grid operation. Also, the resulted rule-based controller could replace the deep neural network-based controller in many field edge devices with limited computing resources, providing comparable performance.

deep reinforcement learning↗
DOE OSTI2022

A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for alternating current optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvatureinformed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.

alternating current optimal power flow↗
DOE OSTI2023

Off-policy deep reinforcement learning with automatic entropy adjustment for adaptive online grid emergency control

Electric overloading conditions and contingencies put modern power systems at risk of voltage collapse and blackouts. Load shedding is crucial to maintain voltage stability for grid emergency control. However, the rule- or model-based schemes rely on accurate dynamic system models and face considerable challenges in adapting to various operating conditions and uncertain event occurrences. Here, to address these issues, this paper proposes a novel deep reinforcement learning (DRL)-based voltage stability control algorithm with automatic entropy adjustment (AEA) for grid emergency control. Various dynamic network components for complex system operations are modeled to construct the DRL environment. An off-policy soft actor-critic architecture is developed to maximize the expected reward and policy entropy simultaneously. The AEA mechanism is proposed to facilitate the policy maximum entropy procedure, and the proposed method can automatically provide effective discrete and continuous actions against various fault scenarios. Our approach accomplishes high sampling efficiency, scalability, and auto-adaptivity of the control policies under high uncertainties. Comparative studies with the existing DRL-based control methods in IEEE benchmarks indicate salient performance improvement of the proposed method for dynamic system emergency control.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Crushing behaviors of unidirectional carbon fiber reinforced plastic composites under dynamic bending and axial crushing loading

The dynamic crushing characteristics of unidirectional carbon fiber reinforced plastic composites under two loading types, dynamic three-point bending and axial crushing, are investigated by experiment and finite element simulation. Hat section samples with two different layup orientations are tested at various impact velocities to investigate its effects. The experiment results show that delamination plays a critical role in dynamic bending deformation. Different layup orientations lead to similar crushing bending behaviors, but remarkable variations in delamination pattern evolutions. In contrast, layup orientation exerts more significant effects on failure modes and energy absorption in axial crushing. A finite element model with multiple layers of thick shell element and cohesive element is established, and simulations are performed for both dynamic three-point bending and axial crushing. The model can capture the crushing behaviors in the two loading types reasonably well. The damage evolution and energy absorption through various mechanisms at different loading conditions are discussed.

42 ENGINEERING↗
DOE OSTI2022

Mechanical joint performances of friction self-piercing riveted carbon fiber reinforced polymer and AZ31B Mg alloy

Carbon fiber reinforced polymer (CFRP) and AZ31B Mg alloy were joined by the friction self-piercing riveting (F-SPR) with different steel rivet shank sizes. With the increase of rivet shank size, lap shear fracture load and mechanical interlock distance increased. Ultrafine grains were formed at the joint in AZ31B as a result of dynamic recrystallization, which contributed to the higher hardness. Fatigue life of the CFRP-AZ31B joint was studied at various peak loads of 0.5, 1, 2, and 3 kN and compared with the resistance spot welded AZ31B-AZ31B from the open literature. The fatigue performance was better at higher peak load (>2 kN) and comparable to that of resistance spot welding of AZ31B to AZ31B at lower peak loads (<1 kN). From fractography, the crack initiation for lower peak load (<1 kN) case was observed at the fretting positions on the top and bottom surfaces of AZ31B sheet. When peak load was increased, fretting between the rivet and the top of AZ31B became more dominant to initiate a crack during fatigue testing.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Effect of printing parameters on 3D-printed carbon fiber-reinforced polymer composites under magnetic field control

Magnetic field controlled (MFC) 3D printing is a promising method for realizing the intelligent control of the structural performance of carbon fiber-reinforced plastic (CFRP) composites by adjusting the fiber orientation during the printing process. However, few studies have focused on the influence of the printing parameters on the magnetic control of the fiber direction. Therefore, in this study, the mechanical properties and internal fiber distributions of MFC 3D-printed samples of CFRP composites with 10 wt% fiber content, which were printed at different speeds and layer thicknesses, were investigated. Subsequently, the effects of the fiber content on the mechanical properties and fiber distribution of the MFC 3D-printed samples were studied. Furthermore, the origin of fiber fracture was evaluated through observation and analysis of the fracture section using scanning electron microscopy. The fiber orientation could be well adjusted under the action of a magnetic field and adjustment of printing parameters, thereby improving the tensile strength, young's modulus significantly with slightly decrease of fracture elongation. The recommended parameter combinations (printing speed and layer thickness) were 40 mm/s and 0.2 mm for 10 wt% fiber content CFRP composites.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Topology-driven compressive behavior of Inconel 718 lattice structures with Z-strut reinforcement fabricated by laser powder bed fusion

This study investigates the compressive deformation behavior and mechanical performance of Inconel 718 lattice structures fabricated by laser powder bed fusion (LPBF). Four unit-cell topologies—BCC, FCC, BCCZ, and FCCZ—were designed with a unit-cell size of 3 mm and fabricated under identical process conditions to isolate the effect of topology. Measured relative densities ranged from 14.65% to 17.72%. Compressive testing showed that Z-strut-reinforced topologies (BCCZ: 54.6 MPa; FCCZ: 80.2 MPa) exhibited higher strength than their unreinforced counterparts, which may be associated with mixed-mode deformation behavior enabled by the vertically aligned Z-struts. Finite element simulations and Digital Image Correlation (DIC) analysis support the observation of a transition from node-dominated deformation in BCC/FCC to mixed-mode deformation in BCCZ/FCCZ. These findings suggest that unit-cell topology is a key design variable for tailoring deformation mechanisms in LPBF lattice structures.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A safe reinforcement learning algorithm for supervisory control of power plants

Traditional control theory-based methods require tailored engineering for each system and constant fine-tuning. In power plant control, one often needs to obtain a precise representation of the system dynamics and carefully design the control scheme accordingly. Model-free Reinforcement learning (RL) has emerged as a promising solution for control tasks due to its ability to learn from trial-and-error interactions with the environment. It eliminates the need for explicitly modeling the environment’s dynamics, which is potentially inaccurate. However, the direct imposition of state constraints in power plant control raises challenges for standard RL methods. To address this, we propose a chance-constrained RL algorithm based on Proximal Policy Optimization for supervisory control. Our method employs Lagrangian relaxation to convert the constrained optimization problem into an unconstrained objective, where trainable Lagrange multipliers enforce the state constraints. In conclusion, our approach achieves the smallest distance of violation and violation rate in a load-follow maneuver for an advanced Nuclear Power Plant design.

constrained optimization↗
DOE OSTI2021

Charpy impact tests of tungsten fiber–reinforced composite from -150 °C to 1000 °C

Tungsten features a unique combination of properties that makes it a favorable plasma-facing material in fusion reactors. The main drawbacks of tungsten are its brittleness at room temperature and the potential embrittlement during operation. Tungsten fiber–reinforced composites have the potential to overcome these limitations. Charpy impact tests ranging from -150 °C to 1000 °C have been performed. To identify the ductile-to-brittle transition temperature and understand the behavior at high temperature a detailed microscopic investigation was used to identify the transition temperature of fiber and matrix, which was between -100 °C to -50 °C and 1000 °C, respectively.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Deep reinforcement learning for optimal control of induction welding process

Optimizing induction welding (IW) process parameters for the application of joining thermoplastic composites is challenging as it requires achieving complex spatiotemporal thermal characteristics along the weld-line to obtain desired weld quality. We formulate an optimal control problem which captures these requirements and seeks to optimize the IW coil speed using a fast-acting dynamic IW process model. We develop a novel Deep Reinforcement Learning (DRL) framework to solve this computationally challenging control problem and demonstrate via simulation study that the learned DRL feedback control policy results in better spatiotemporal thermal characteristics as compared to the current state-of-the-art.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Development and assessment of hierarchical multi-reward reinforcement learning based potential for silicene with state-of-the-art models

We develop a new interatomic force field for Silicene, a 2D material with a buckled hexagonal lattice structure with high polymorphism. We introduce new parameterizations of a Tersoff model using a hierarchical multi-reward reinforcement learning (RL) methodology coupled with a continuous Monte Carlo Tree Search optimization. Our model significantly outperforms existing methods by enhancing the accuracy of predictions for the structural and thermodynamic properties of seven silicene polymorphs-including structure, energy, equation of state, elasticity, and phonon dispersion-when compared to established models. We further make a comprehensive comparison of the various models in predicting the mechanical and thermal properties of silicene. We trace the origin of the improved performance to the description of the angular dependence in the bond-order term, suggesting that modifying the angular terms in short-range models is essential to capture the structural diversity in low dimensional systems.

2D materials↗
DOE OSTI2025

Optimizing processing conditions for additively reinforced thermoforming (ART) in convergent manufacturing

This study utilized additively reinforced thermoforming (ART) to enhance the thermomechanical properties of polyethylene terephthalate glycol (PETG) sheet. ART materials were produced by overprinting PETG/carbon fiber filament (PETG/CF) on neat PETG sheets at varying conditions. The mechanical properties of the PETG sheet, PETG/CF, and ART materials were assessed, showing that ART exhibited superior tensile strength and modulus of elasticity. The tensile strength and modulus in the x-direction for ART at 265°C were 57.32 ± 2.9 MPa and 3.41 ± 0.4 GPa, respectively, compared to 49.1 ± 0.5 MPa and 1.92 ± 0.09 GPa for neat PETG. Microstructural analysis revealed strong interfacial adhesion between layers, while thermogravimetric analysis (TGA), differential scanning calorimetry (DSC), and heat deflection temperature analysis provided insights into the ART material's thermoforming behavior, aiding design optimization for enhanced stiffness, reduced necking, and improved customization. In conclusion, this information can be used to design for the thermoforming operation.

Additive reinforcement↗
DOE OSTI2022

Multiscale dynamic experiments on fiber-reinforced composites with damage assessment using high-speed synchrotron X-ray phase-contrast imaging

We report fiber-reinforced composites (FRCs) have been widely used as protection materials to replace heavier metals. However, predictive capabilities for the impact-induced failure of FRCs have not been well-developed. In this work, we introduced an integrated technique to visualize the dynamic failure of FRCs, with the modified Kolsky bar loading and the high-speed synchrotron X-ray phase-contrast imaging (PCI). We summarize our experimental studies across the FRCs’ structural levels by using this integrated technique, including single-fiber tensile experiments, fiber/matrix debonding experiments, and single-edge notched bending experiments on composite laminates. It is revealed that the high-speed synchrotron X-ray PCI technique possesses sufficient spatial resolution to identify the failure behavior of a single fiber in the 10-μm scale level, as well as temporal resolution to identify the time sequence of internal dynamic damage evolution. The technique can penetrate through the opaque composite sample and provide real-time visualization of small-scale damaging mechanisms such as the embedded fiber’s sliding in the matrix and the fiber’s transverse debonding with the matrix. At an expanding scale, the technique can identify the crack initiation and track the cracking behavior inside a single ply or between different plies of a composite laminate. High-speed visualization inside the material by this integrated technique provides critical deformation and failure information to develop physical understanding of the damage inside composites under impact.

47 OTHER INSTRUMENTATION↗