Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Visibility-enhanced model-free deep reinforcement learning algorithm for voltage control in realistic distribution systems using smart inverters

Increasing integration of distributed solar photovoltaic (PV) into distribution networks could result in adverse effects on grid operation. Traditional model-based control algorithms require accurate model information that is difficult to acquire and thus are challenging to implement in practice. Here, this paper proposes a surrogate model-enabled grid visibility scheme to empower deep reinforcement learning (DRL) approach for distribution network voltage regulation using PV inverters with minimal system knowledge. In contrast to existing DRL methods, this paper presents and corroborates the adverse impact of missing load information on DRL performance and, based on this finding, proposes a surrogate model methodology to impute load information utilizing observable data. Additionally, a multi-fidelity neural network is utilized to construct the DRL training environment, chosen for its efficient data utilization and enhanced robustness to data uncertainty. The feasibility and effectiveness of the proposed algorithm are assessed by considering DRL testing across varying degrees of observable load information and diverse training environments on a realistic power system.

14 SOLAR ENERGY↗
DOE OSTI2022

Reinforcement Learning for feedback-enabled cyber resilience

The rapid growth in the number of devices and their connectivity has enlarged the attack surface and made cyber systems more vulnerable. As attackers become increasingly sophisticated and resourceful, mere reliance on traditional cyber protection, such as intrusion detection, firewalls, and encryption, is insufficient to secure the cyber systems. Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and uncertainties in real-time and strategically responds to them to maintain the critical functions of the cyber systems in the event of successful attacks. Feedback architectures play a pivotal role in enabling the online sensing, reasoning, and actuation process of the CRM. Reinforcement Learning (RL) is an important gathering of algorithms that epitomize the feedback architectures for cyber resilience. It allows the CRM to provide dynamic and sequential responses to attacks with limited or without prior knowledge of the environment and the attacker. In this work, we review the literature on RL for cyber resilience and discuss the cyber-resilient defenses against three major types of vulnerabilities, i.e., posture-related, information-related, and human-related vulnerabilities. Here we introduce moving target defense, defensive cyber deception, and assistive human security technologies as three application domains of CRMs to elaborate on their designs. The RL algorithms also have vulnerabilities themselves. We explain the major vulnerabilities of RL and present develop several attack models where the attacker target the information exchanged between the environment and the agent: the rewards, the state observations, and the action commands. We show that the attacker can trick the RL agent into learning a nefarious policy with minimum attacking effort. The paper introduces several defense methods to secure the RL-enabled systems from these attacks. However, there is still a lack of works that focuses on the defensive mechanisms for RL-enabled systems. Last but not least, we discuss the future challenges of RL for cyber security and resilience and emerging applications of RL-based CRMs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Ten questions concerning reinforcement learning for building energy management

As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. Here, in this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

Degradation processes of reinforced concretes by combined sulfate–phosphate attack

A novel form of alteration due to the interaction between hydrated cement phases and sulfate and phosphate-based pollutants is described, through the characterization of concrete samples from an industrial reinforced concrete building. Decalcification of the cement matrices was observed, with secondary sulfate and phosphate-based mineral formation, according to a marked mineralogical and textural zoning. Five alteration layers may be detected: the two outermost layers are characterized by the presence of gypsum–brushite solid solution phases associated with anhydrous calcium sulfates and phosphates, respectively, while a progressive increase in apatite and ammonium magnesium phosphates is observable in the three innermost layers, associated with specific apatite precursors (brushite, octacalcium phosphate and amorphous calcium phosphate, respectively). The heterogeneous microstructural development of secondary phases is related to the chemical, pH and thermal gradients in the attacked cementitious systems, caused by different sources of pollutants and the exposure to the sun's radiation.

36 MATERIALS SCIENCE↗
DOE OSTI2015

High rate response of ultra-high-performance fiber-reinforced concretes under direct tension

The tensile response of ultra-high-performance fiber-reinforced concretes (UHPFRCs) at high strain rates (5–24 s{sup −} {sup 1}) was investigated. Three types of steel fibers, including twisted, long and short smooth steel fibers, were added by 1.5% volume content in an ultra high performance concrete (UHPC) with a compressive strength of 180 MPa. Two different cross sections, 25 × 25 and 25 × 50 mm{sup 2}, of tensile specimens were used to investigate the effect of the cross section area on the measured tensile response of UHPFRCs. Although all the three fibers generated strain hardening behavior even at high strain rates, long smooth fibers produced the highest tensile resistance at high rates whereas twisted fiber did at static rate. The breakages of twisted fibers were observed from the specimens tested at high strain rates unlike smooth steel fibers. The tensile behavior of UHPFRCs at high strain rates was clearly influenced by the specimen size, especially in post-cracking strength.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Corrosion resistance of steel fibre reinforced concrete - A literature review

Steel fibre reinforced concrete (SFRC) is increasingly being used in the construction of civil infrastructure. However, there are inconsistencies among international standards and guidelines regarding the consideration of carbon-steel fibres for the structural verification of SFRC exposed to corrosive environments. This paper presents a review of the published research regarding carbonation- and chloride-induced corrosion of SFRC, and proposes a deterioration theory for cracked SFRC exposed to chlorides and carbonation, based on the damage at the fibre-matrix interface. The review confirms an overall agreement among academics and regulators regarding the durability of uncracked SFRC exposed to chlorides and carbonation. Contrariwise, the durability of cracked SFRC is under discussion at the technical and scientific level, as there is a large dispersion on the experimental results and some of the mechanisms governing the corrosion of carbon-steel fibres in cracks and its effects on the fracture behaviour of SFRC are not fully understood.

36 MATERIALS SCIENCE↗
DOE OSTI2018

A discrete-continuum coupled finite element modelling approach for fibre reinforced concrete

Highlights: • An easy-to-implement but effective meso-scale modelling approach is developed for FRC. • It is capable of simulating and interpreting all main failure mechanisms in FRC. • It is critically validated by experiments with single or multiple random fibres under various loading conditions. • It can be used to optimise key design factors of FRC at meso/material scale by parametric studies. - Abstract: Fibre reinforced concrete (FRC) exhibits complicated failure modes such as fibre breakage, mortar cracking, crushing and spalling, fibre-mortar interfacial debonding, depending on many material properties, geometric dimensions, boundary and loading conditions. Most existing numerical models are unable to reproduce these failure modes that may occur simultaneously or sequentially in a specimen, mainly due to difficulties in generating finite element meshes with a large number of randomly-oriented fibres. Herein we develop a discrete-continuum coupled finite element modelling approach for FRC materials capable of effectively simulating all the major failure modes. The continuum damaged plasticity model is used to simulate damage and fracture behaviour of the mortar, while debonding of fibre-mortar interfaces is modelled by nonlinear cohesive interfacial elements. Unique techniques are devised to generate conforming meshes between fibres and the surrounding mortar so that the randomly-oriented fibres are easily modelled. The modelling approach is validated by simulating single fibre pullout tests with different inclination angles, notched and non-notched direct tensile tests and three-point bending beam tests with randomly-distributed multiple fibres.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A novel implicit hybrid machine learning model and its application for reinforcement learning

A novel methodology to develop implicit hybrid models is presented. PyTorch is used to integrate physics-based equations with machine learning models. Automatic differentiation of the hybrid model is leveraged to solve the implicit equations. Iterative solving enables gradient based updates to the machine learning model. The novel methodology is compared to an explicit hybrid approach on a continuously stirred tank reactor (CSTR). The novel method results in a lower modelling error. Both hybrid models effectively train with noisy data. To test the implicit hybrid model, it is employed as a reinforcement learning (RL) training model. The RL algorithm trained on the hybrid model outperforms real time optimization of the CSTR and performs nearly as well as RL trained directly on the CSTR and a traditional gradient based approach. Training RL directly on the CSTR requires over 60,000 system interactions compared to 6000 historical data points for hybrid model development.

42 ENGINEERING↗
DOE OSTI2022

Reinforcement learning for online adaptation of model predictive controllers: Application to a selective catalytic reduction unit

Here we present a novel application of reinforcement learning (RL) for online dynamic tuning of model predictive controllers (MPC). Applying a state-action-reward-state-action (SARSA) algorithm for temporal difference learning with a control-specific reward function improves the error tracking performance of a standard MPC formulation. The proposed RL approach is also readily adaptable to other MPCs, or entirely different control approaches. Practical details for the implementation of the RL-MPC algorithm are also presented. The proposed algorithm is applied to a case study of controlling nitrogen oxide (NO x ) emissions in an industrial selective catalytic reduction (SCR) unit, a control problem characterized by significant nonlinearity and time delay. Along with an RL-MPC formulation for NOx control, another MPC is proposed to mitigate ammonia slip and decrease ammonia consumption in the SCR. Results showing the efficacy of the RL-MPC for NO x control through learning and implementation on the nonlinear SCR dynamic model are presented.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Development of algorithms for augmenting and replacing conventional process control using reinforcement learning

Here, this work seeks to allow for the online operation and training of model-free reinforcement learning (RL) agents but limit the risk to system equipment and personnel. The parallel implementation of RL alongside more conventional process control (CPC) allows for the RL algorithm to learn from CPC. The past performance of both methods are assessed on a continuous basis allowing for a transition from CPC to RL and, if needed, transitioning back to CPC from RL. This allows for the RL algorithm to slowly and safely assume control of the process without significant degradation in control performance. It is shown that the RL can derive a near optimal policy even when coupled with a suboptimal CPC. It is also demonstrated that the coupled RL-CPC algorithm learns at a faster rate than traditional RL methods of exploration while the algorithm’s performance does not deteriorate below CPC, even when exposed to an unknown operating condition.

30 DIRECT ENERGY CONVERSION↗
DOE OSTI2020

Recent advancements of plant-based natural fiber–reinforced composites and their applications

Demands for reducing energy consumption and environmental impacts are the major driving factors for the development of natural fiber–reinforced composites (NFRCs) in many sectors. Compared with synthesized fiber, natural fiber provides several advantages in terms of biodegradability, light weight, low price, life-cycle superiority, and satisfactory mechanical properties. However, the inherent features of plant-based natural fibers have presented challenges to the development and application of NFRCs, such as variable fiber quality, limited mechanical properties, water absorption, low thermal stability, incompatibility with hydrophobic matrices, and propensity to agglomeration. Substantial research has recently been conducted to address these challenges for improved performance of NFRCs and their applications. This article reviews the recent advancements of plant-based NFRCs, focusing on strategies and breakthroughs in enhancing the NFRCs’ performance, including fiber modification, fiber hybridization, lignocellulosic fillers incorporation, conventional processing techniques, additive manufacturing (3D printing), and new fiber source exploration. In conclusion, the sustainability of plant-based NFRCs using life-cycle assessment and the burgeoning applications of NFRCs with emphasis on the automotive industry are also discussed.

3D reinforcement↗
DOE OSTI2019

Failure criteria of unidirectional carbon fiber reinforced polymer composites informed by a computational micromechanics model

Failure prediction for carbon fiber reinforced polymer (CFRP) composites has been a longstanding challenge. In this paper, we address this challenge by first applying a well-established computational micromechanics model based on representative volume element to predict the failure envelopes of unidirectional (UD) CFRP composites. Then, these failure envelopes are compared with the classical failure criteria. We have evaluated the performances of these failure criteria and identified the aspects for further improvement in their accuracies for the UD CFRP composites studied herein. Based on the failure mechanisms from computational analyses and the comparisons between predicted failure envelopes and classical failure criteria, a new set of homogenized failure criteria is proposed. Additionally, the newly proposed failure criteria show significant improvement according to our computational and experimental results. Furthermore, we have compared the proposed failure criteria with existing experimental data and computational results available in the literature for different types of composites. Good agreements are generally observed.

36 MATERIALS SCIENCE↗
DOE OSTI2020

An integrated computational materials engineering framework to analyze the failure behaviors of carbon fiber reinforced polymer composites for lightweight vehicle applications

A bottom-up multi-scale modeling approach is used to develop an Integrated Computational Materials Engineering (ICME) framework for carbon fiber reinforced polymer (CFRP) composites, which has the potential to reduce development to deployment lead time for structural applications in lightweight vehicles. In this work, we develop and integrate computational models comprising of four size scales to fully describe and characterize three types of CFRP composites. In detail, the properties of the interphase region are determined by an analytical gradient model and molecular dynamics analysis at the nano-scale, which is then incorporated into micro-scale unidirectional (UD) representative volume element (RVE) models to characterize the failure strengths and envelopes of UD CFRP composites. Then, the results are leveraged to propose an elasto-plastic-damage constitutive law for UD composites to study the fiber tows of woven composites as well as the chips of sheet molding compound (SMC) composites. Subsequently, the failure mechanisms and failure strengths of woven and SMC composites are predicted by the meso-scale RVE models. Finally, building upon the models and results from lower scales, we show that a homogenized macro-scale model can capture the mechanical performance of a hat-section-shaped part under four-point bending. Along with the model integration, we will also demonstrate that the computational results are in good agreement with experiments conducted at different scales. The present work illustrates the potential and significance of integrated multi-scale computational modeling tools that can virtually evaluate the performance of CFRP composites and provide design guidance for CFRP composites used in structural applications.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Multi-scale computational analysis of unidirectional carbon fiber reinforced polymer composites under various loading conditions

In this study, a multi-scale computational analysis based on representative volume element (RVE) modeling and molecular dynamics (MD) simulations is developed to investigate the microscopic failure mechanisms of unidirectional (UD) carbon fiber reinforced polymer (CFRP) composites. The average properties of the 200-nm thickness interphase region between fiber and matrix are characterized through MD simulations and an analytical gradient model. The results demonstrate that the interphase region has higher Young’s modulus and strength, compared to the bulk matrix. This stiffened interphase region influences the composite response significantly. Specifically, the traditional two-phase model with zero-thickness interface fails to capture the stress-strain behavior compared to the experimental data. However, by adding the interphase region to a modified RVE model, the accuracy of simulation results will be improved significantly. Furthermore, a coupled experimental-computational micromechanics approach is adopted to calibrate and validate the cohesive parameters of the interface. By including the cohesive interface, our modified RVE model accurately captures the failure strength of the composites. Finally, different failure mechanisms for specimens are investigated using our multi-scale computational framework. In conclusion, the results show that the failure modes of UD CFRP composites are very complex and multiple failure mechanisms co-exist depending on the loading conditions, agreeing well with our experimental analyses.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Experimental and computational analysis of failure mechanisms in unidirectional carbon fiber reinforced polymer laminates under longitudinal compression loading

Here, his study investigates the failure mechanisms of notched unidirectional (UD) carbon fiber reinforced polymer (CFRP) laminates subjected to longitudinal compressive loading. A sequence of failure initiation and propagation is observed based on optical microscopy images of specimens during failure. The micrographs reveal that the main failure mechanism in UD laminates is fiber kinking failure. The influence of manufacturing defects and voids on fiber kinking mechanisms is also analyzed. The results show that the voids in the resin-rich area lead to kink-band splitting, while in some cases the voids cause the kink-band to deflect in a new direction depending on the location of the voids. The next focus of this study is to propose a computational micromechanics model considering local fiber waviness to study the fiber kinking failure mechanism. The computational results on kink-band formation and stress-strain response show good consistency with our experimental analysis. Lastly, the failure envelopes of σ 11 -σ 22 and σ 11 -τ 12 subjected to combined loading conditions are obtained from the computational model, which provide essential inputs for future theoretical failure criteria.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Fatigue behavior analysis and multi-scale modelling of chopped carbon fiber chip-reinforced composites under tension-tension loading condition

Here, the fatigue damage behavior of chopped carbon fiber chip-reinforced composite has been experimentally and numerically investigated. A large scatter is exhibited in the S-N diagram due to the random distribution of carbon fiber chips, and therefore a new analysis procedure is proposed to link the local microstructure to the fatigue behavior of the bulk material. Interrupted fatigue tests are also performed so that microstructure characterization can be conducted on the tested samples to analyze the crack initiation and propagation. Inspired from the experimental findings, a multi-scale progressive damage fatigue model is proposed to predict the fatigue behavior. The model incorporates a new stochastic chip-packing algorithm for microstructure reconstruction along with continuum damage models into the representative volume element (RVE) model in ABAQUS/Explicit. Finally, the simulation results based on the proposed model are in good agreement with the experimental data in terms of cracking modes and predicted life.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Notch insensitivity in fatigue failure of chopped carbon fiber chip-reinforced composites using experimental and computational analysis

This study focuses on the notch effect on the fatigue failure of chopped carbon fiber chip-reinforced sheet molding compound (SMC) composites by using both experimental and numerical methods. The S-N diagrams of notched specimens are first obtained experimentally, which exhibit large scatter and are shown to depend on the spatial distribution of local fiber orientations. After considering the effect of this spatial distribution by adopting a new fatigue damage parameter to normalize the results, we achieve a much better correlation in the normalized S-N diagram for all the notched specimens with different hole diameters. Next, compared with the fatigue results of smooth specimens without notches, we find that SMC composites under tension-tension fatigue loading exhibit notch-insensitivity, which renders this type of composites a better candidate in joints or similar applications. Meanwhile, the fatigue failure mechanisms of notched SMC composites are explored through a microstructure characterization procedure. On this basis, a computational micromechanics model considering the notch effect and an improved multi-scale progressive damage fatigue law, calibrated by coupon-level fatigue tests of smooth SMC composites under different loading conditions, are employed to predict the fatigue behavior of notched SMC composites. We show that detailed failure processes and fatigue life of notched SMC composites can be efficiently and adequately predicted by this computational framework.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Experimental and computational analysis of bending fatigue failure in chopped carbon fiber chip reinforced composites

With a better balance among good mechanical performance, high freedom of design, and low material and manufacturing cost, chopped carbon fiber chip reinforced sheet molding compound (SMC) composites show great potential in different engineering applications. Here in this paper, bending fatigue behaviors of SMC composites considering the heterogeneous fiber orientation distributions have been thoroughly investigated utilizing both experimental and computational methods. First, four-point bending fatigue tests are performed with designed SMC composites, and the local modulus is adopted as a metric to represent the local fiber orientation of two opposing sides. Interestingly, SMC composites with and without large discrepancy in local modulus of opposing sides show different fatigue behaviors. Interrupted tests are conducted to explore the bending fatigue failure mechanism, and the damage processes of valid specimens are also closely examined. We find that the fatigue failure of SMC composites under four-point bending is governed by crack propagation instead of crack initiation. Because of this, the heterogeneous local fiber orientations of both sides of the specimen influence fatigue life. The microstructure of the lower side shows a direct influence while that of the upper side also exhibiting influence which becomes more prominent for high cycle fatigue cases. Furthermore, a hybrid micro–macro computational model is proposed to efficiently study the cyclic bending behavior of SMC composites. The region of interest is reconstructed with a modified random sequential absorption algorithm to conserve all the microstructural details including the heterogeneous fiber orientation, while the rest of the regions are modeled as homogenized macro-scale continua. Combined with a framework to capture the progressive fatigue damage under cyclic bending, the bending fatigue behaviors of SMC composites are accurately captured by the hybrid computational model comparing with our experimental analysis.

36 MATERIALS SCIENCE↗