Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

The influence of dynamic rheological properties on carbon fiber-reinforced polyetherimide for large-scale extrusion-based additive manufacturing

Printing high-performance thermoplastics on large scale extrusion-based additive manufacturing platforms requires stability over a range of processing conditions. Yet, research on the melt dynamics and processing conditions of these thermoplastics in big area additive manufacturing (BAAM) are limited. This report characterizes the dynamic rheological behavior of polyetherimide (PEI), a high-performance thermoplastic, as well as carbon fiber (CF)-reinforced PEI composites as a BAAM feedstock material. The viscoelastic properties, such as the storage and loss moduli and complex viscosity, are investigated in relation to the BAAM extrusion process. The results show that CF-PEI composites behave like a viscous liquid during BAAM extrusion. The addition of CF to PEI enhances the shear thinning effect and significantly increases the complex viscosity (2.5× increase for 20% CF, and 3× for 30% CF). The increased viscosity increases the torque on the extruder, which may be alleviated by increasing the material processing temperature.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Reinforcement learning for block decomposition of planar CAD models

Abstract The problem of hexahedral mesh generation of general CAD models has vexed researchers for over 3 decades and analysts often spend more than 50% of the design-analysis cycle time decomposing complex models into simpler blocks meshable by existing techniques. The decomposed blocks are required for generating good quality meshes (tilings of quadrilaterals or hexahedra) suitable for numerical simulations of physical systems governed by conservation laws. We present a novel AI-assisted method for decomposing (segmenting) planar CAD (computer-aided design) models into well shaped rectangular blocks. Even though the simple examples presented here can also be meshed using many conventional methods, we believe this work is proof-of-principle of a AI-based decomposition method that can eventually be generalized to complex 2D and 3D CAD models. Our method uses reinforcement learning to train an agent to perform a series of optimal cuts on the CAD model that result in a good quality block decomposition. We show that the agent quickly learns an effective strategy for picking the location and direction of the cuts and maximizing its rewards. This paper is the first successful demonstration of an agent autonomously learning how to perform this block decomposition task effectively, thereby holding the promise of a viable method to automate this challenging process for more complex cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

A peridynamic failure analysis of fiber-reinforced composite laminates using finite element discontinuous Galerkin approximations

Here, this paper presents a discontinuous Galerkin weak form for bond-based peridynamic models to predict the damage of fiber-reinforced composite laminates. To represent the anisotropy of a laminate in a peridynamic model, a lamina is simplified as a transversely isotropic medium under a plane stress condition. The laminated structure is modeled by stacking the surface mesh layers along the thickness direction according to the laminate sequence. To avoid a mesh dependence on either the fiber orientation or the discretization, the spherical harmonic expansion theory is employed to construct a function for the micro-elastic modulus in terms of the bond-fiber angle. The laminate material is decomposed into an isotropic matrix material part and a transversely isotropic material part. The bond stiffness can be evaluated using the engineering material constants, based on the equivalence between the elastic energy density in the peridynamic theory and the elastic energy density in the classic continuum mechanics theory. Benchmark tests are conducted to verify the proposed model. Numerical results illustrate that the convergence of simulations with different horizon sizes and meshes can be achieved. In terms of damage analysis, the proposed model can capture the dynamic process of the complex coupling of the inner-layer and delamination damage modes.

42 ENGINEERING↗
DOE OSTI2023

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

Microstructure and Mechanical Properties of CNT-Reinforced AZ31 Matrix Composites Prepared Using Hot-Press Sintering

AZ31 matrix composites reinforced by carbon nanotubes (CNTs) were fabricated using hot-press sintering. The microstructure and mechanical properties of the composites (denoted as Mg-3wt.%Al-1wt.%Zn-xwt.%CNTs, x = 0-1.5) were investigated. The results showed that the elastic modulus, yield strength, tensile strength and elongation of the composites were significantly improved when compared with the matrix alloy AZ31. Of the tested composites Mg-3wt.%Al-1wt.%Zn-1.0wt.%CNTs exhibited the most favorable mechanical properties: compared with Mg-3wt.%Al-1wt.%Zn, the elastic modulus improved by 25.85%, yield strength increased by 23.48%, and tensile strength and elongation showed 19.35 and 48.23% improvement, respectively. The fracture surface of the Mg-3wt.%Al-1wt.%Zn-CNTs composites displayed features of brittle fracture, which suggests that CNTs do not markedly enhance the plasticity of AZ31.

36 MATERIALS SCIENCE↗
DOE OSTI2017

Study on the Reliability of Carbon Nanotube-Reinforced Sn-58Bi Lead-Free Solder Joints

The wettability, microstructure and mechanical properties of multi-walled carbon nanotube (CNT)-reinforced Sn-58Bi composite solder joints were investigated. The results indicate that the wettability of the Sn-58Bi solder is improved and the growth of interfacial intermetallic (IMC) compounds for solder joints is restrained by CNTs. The thickness of the IMC layers is decreased with increasing CNTs concentration. The microstructure of the Sn-58Bi composite solders is refined, the maximum tensile strength and shear strength are obtained by adding the optimum CNTs content about 0.01 wt.%. However, excessive content of CNTs addition can deteriorate the properties. The creep rupture life of Sn-58Bi solder joint and Sn-58Bi-0.01CNTs solder joint is decreased with increasing load, temperature and current density. The average creep rupture life of Sn-58Bi-0.01CNTs composite solder joint is 60% higher than that of the plain solder joint.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2018

Elastic and Plastic Behavior of an Ultrafine-Grained Mg Reinforced with BN Nanoparticles

Pure microcrystalline magnesium (µMg) was reinforced with hexagonal boron nitride (hBN) nanoparticles and was fabricated by powder metallurgy process followed by hot extrusion. For comparison pure magnesium powder was consolidated by hot extrusion too. Both materials exhibited a significant fiber texture. Mg-hBN nanocomposites (nc) and pure Mg specimens were deformed between room temperature and 300 °C under tension and compression mode. The yield strength and ultimate tensile and compression strength as well as characteristic stresses were evaluated and reported. The tensile and compressive strengths of Mg-hBN nc are quiet superior in values compared to monolithic counterpart as well as Mg alloys. The compressive yield strength of µMg was recorded as 90 MPa, whereas the Mg-hBN nancomposite shows 125 MPa at 200 °C. The tensile yield strength of µMg was computed as 67 MPa which is quite lower as compared to Mg-hBN nanocomposite’s value which was recorded as 157 MPa at 200 °C. Under tensile stress the true stress–strain curves are flat in nature, whereas the stress–strain curves observed in compression at temperatures up to 100 °C exhibited small local maxima at the onset of deformation followed by a significant work hardening.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Fabrication of Ta-Reinforced Cu-Based Bulk Metallic Glass Composites by High-Pressure Torsion

Ex situ Cu{sub 54}Zr{sub 22}Ti{sub 18}Ni{sub 6} matrix bulk metallic glass composites (BMGCs) reinforced with Ta particles (10–30 vol.%) were fabricated by high-pressure torsion (HPT) under applied pressure of 6 GPa for 1–3 turns at temperatures of 25 or 200 °C. The densification, structure, thermal, and mechanical properties of BMGC samples were investigated by Archimedes densitometry, optical microscopy, x-ray diffraction, differential scanning calorimetry, small punch test, and microhardness measurements. Near full-density BMGC disks (relative densities higher than 0.97) are fabricated using HPT, irrespective of the processing strain and temperature. The microhardness of BMGCs is improved by increasing the number of turns, or the processing temperature due to the improvement in the deformation degree of both the amorphous and Ta particles as well as the reduction in the interlayer spacing of Ta particles. Nevertheless, there is an optimal amount of Ta which gives rise to the peak hardness in the BMGC samples, depending on the amount of imposed strain during HPT. The results of small punch test indicate that the addition of Ta to the monolithic BMG and increasing the HPT temperature significantly enhance the fracture load and deflection.

36 MATERIALS SCIENCE↗
DOE OSTI2018

3D Gel-Printing of TiC-Reinforced 316L Stainless Steel: Influence of the Printing Parameters

In this paper, the composite composed of TiC (35 wt.%) reinforced 316L stainless steel matrix was successfully prepared using a 3D gel-printing (3DGP) process. The “step effect” of 3D printing can be reduced using reasonable parameter settings. The properties of TiC/316L slurry for 3DGP and the influence of the different printing parameters on the surface quality of the samples, such as the extrusion overlaps and printing speed, were researched. The microstructure and fracture morphology of the sintered sample were revealed using scanning electron microscopy. The surface roughness, density, and mechanical properties were also measured. When the internal diameter of the nozzle was 0.50 mm, the result showed that the proper extrusion overlap parameter was 0.08 mm and the maximum printing speed was 20 mm/s. Furthermore, the minimum surface roughness value was 10 μm. After sintering, the mechanical performances were similar to those obtained using a conventional powder metallurgy process.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Metallization of Carbon Fiber Reinforced Polymers for Lightning Strike Protection

Carbon fiber reinforced polymers (CFRPs) are increasingly used in the latest generations of aircraft, but their low electrical conductivity is a concern for lightning strike protection. During the past few years, development of lightning strike protection solutions for CFRP has attracted increasing interest, and cold spray is one coating approach to achieving this. In this work, pure tin coatings and tin–copper composite coatings were cold sprayed on CFRP as possible lightning strike protection materials. The coatings were subjected to various mechanical and electrical characterizations. The effect of annealing on electrical conductivity was also examined. Furthermore, continuous current injection tests, which duplicated component C of a lightning waveform, were performed on the coatings. The results showed that the cold-sprayed coatings can provide effective protection to the CFRP underneath when subjected to currents up to 200 A with 1-s duration.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Optimization of the FRIB beam dump: a hybrid genetic algorithm and reinforcement learning approach

The operational envelope of high-power-density systems, such as particle accelerators and advanced nuclear energy systems, is critically constrained by the need to manage extreme thermal loads. To address this, we present a novel hybrid optimization framework combining a genetic algorithm (GA) with a soft actor-critic (SAC) deep reinforcement learning agent. This framework was applied to a practical high-heat-flux problem: redesigning the beam dump at the Facility for Rare Isotope Beams (FRIB) for a power upgrade from 20 kW to 50 kW. The resulting design, validated by three-dimensional conjugate heat transfer simulations, suppresses hazardous hot spots and yields a markedly more uniform temperature distribution. This provides a robust operating margin, increasing the average power-handling capability by 72% relative to the current design, demonstrating the framework’s potential to solve complex thermal management challenges in both accelerator technology and advanced nuclear systems.

Accelerator↗
DOE OSTI2020

Chapter Nine - Automated Optimal Control in Energy Systems: The Reinforcement Learning Approach

With the development of smart grid technologies an increasing number of new devices and participants have joined modern energy systems and are inevitably making them more complicated and interdependent than ever. Optimally controlling such a complex energy system and maintaining its operation in a high-efficient, secure, and resilient manner are challenging tasks to the system operators. Fortunately, the revolution in deep learning and artificial intelligence (AI), both from hardware and algorithms perspectives, has provided new ideas and solutions to many previously intractable problems. As a result, this advance in computer science also sparked great research interests in utilizing AI in solving engineering problems related to the modern energy systems. Among many AI techniques, deep reinforcement learning (DRL) has demonstrated great potential for solving sequential optimization problems, which are very common in the engineering domains. Its ability to handle nonlinearity and stochasticity in controlled systems has out-competed many traditional optimal control algorithms. Therefore in this chapter, we focus on the state-of-the-art of DRL concepts and related algorithms, compare their pros and cons with traditional optimal control approaches and discuss the typical workflow for leveraging RL in solving complex problems in modern energy systems.

artificial intelligence↗
DOE OSTI2023

Deep reinforcement learning for class imbalance fault diagnosis of equipment in nuclear power plants

In equipment fault diagnosis in nuclear power plants, there may be far more samples in one class (e.g., a health state) than in another class (e.g., a fault state). The distribution of data in each class is highly skewed. Most machine learning algorithms are suitable for balanced training datasets. When faced with imbalanced samples, these algorithms tend to provide good identification for the majority classes and bias for the minority classes. However, the misclassification of minority classes can lead to high costs. To address the above problem, this paper develops a deep reinforcement learning-based diagnosis method that models fault diagnosis as a sequential decision-making process. At each time step, the agent receives the state of the environment represented by the training samples and then takes a diagnosis action guided by a policy. If the action is correct/incorrect, the agent receives a positive/negative reward. The reward for minority classes is higher than that for majority classes. The agent’s goal is to obtain as many cumulative rewards as possible in the process, i.e., to identify the sample as correctly as possible. Six demonstration scenarios are constructed, depending on the selected fault datasets and the designed model structures. Experiments show that the proposed method achieves a higher weighted-averaged F1 score than the classical supervised learning method in most cases of class imbalance. Finally, the proposed method has potential applications in the field of class imbalance fault diagnosis of equipment in nuclear power plants.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Optimal carbon storage reservoir management through deep reinforcement learning

Model-based optimization plays a central role in energy system design and management. The complexity and high-dimensionality of many process-level models, especially those used for geosystem energy exploration and utilization, often lead to formidable computational costs when the dimension of decision space is also large. This work adopts elements of recently advanced deep learning techniques to solve a sequential decision-making problem in applied geosystem management. Specifically, a deep reinforcement learning framework was formed for optimal multiperiod planning, in which a deep Q-learning network (DQN) agent was trained to maximize rewards by learning from high-dimensional inputs and from exploitation of its past experiences. To expedite computation, deep multitask learning was used to approximate high-dimensional, multistate transition functions. Both DQN and deep multitask learning are pattern based. As a demonstration, the framework was applied to optimal carbon sequestration reservoir planning using two different types of management strategies: monitoring only and brine extraction. Both strategies are designed to mitigate potential risks due to pressure buildup. Results show that the DQN agent can identify the optimal policies to maximize the reward for given risk and cost constraints. Finally, experiments also show that knowledge the agent gained from interacting with one environment is largely preserved when deploying the same agent in other similar environments.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Residential heating, ventilation, and air conditioning (HVAC) has been considered as an important demand response resource. However, the optimization of residential HVAC control is no trivial task due to the complexity of the thermal dynamic models of buildings and uncertainty associated with both occupant-driven heat loads and weather forecasts. In this paper, we apply a novel model-free deep reinforcement learning (RL) method, known as the deep deterministic policy gradient (DDPG), to generate an optimal control strategy for a multi-zone residential HVAC system with the goal of minimizing energy consumption cost while maintaining the users’ comfort. Here, the applied deep RL-based method learns through continuous interaction with a simulated building environment and without referring to any prior model knowledge. Simulation results show that compared with the state-of-art deep Q network (DQN), the DDPG-based HVAC control strategy can reduce the energy consumption cost by 15% and reduce the comfort violation by 79%; and when compared with a rule-based HVAC control strategy, the comfort violation can be reduced by 98%. In addition, experiments with different building models and retail price models demonstrate that the well-trained DDPG-based HVAC control strategy has high generalization and adaptability to unseen environments, which indicates its practicability for real-world implementation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency

Behind-the-meter distributed energy resources (DERs), including building solar photovoltaic (PV) technology and electric battery storage, are increasingly being considered as solutions to support carbon reduction goals and increase grid reliability and resiliency. However, dynamic control of these resources in concert with traditional building loads, to effect efficiency and demand flexibility, is not yet commonplace in commercial control products. Traditional rule-based control algorithms do not offer integrated closed-loop control to optimize across systems, and most often, PV and battery systems are operated for energy arbitrage and demand charge management, and not for the provision of grid services. More advanced control approaches, such as MPC control have not been widely adopted in industry because they require significant expertise to develop and deploy. Recent advances in deep reinforcement learning (DRL) offer a promising option to optimize the operation of DER systems and building loads with reduced setup effort. However, there are limited studies that evaluate the efficacy of these methods to control multiple building subsystems simultaneously. Additionally, most of the research has been conducted in simulated environments as opposed to real buildings. This paper proposes a DRL approach that uses a deep deterministic policy gradient algorithm for integrated control of HVAC and electric battery storage systems in the presence of on-site PV generation. The DRL algorithm, trained on synthetic data, was deployed in a physical test building and evaluated against a baseline that uses the current best-in-class rule-based control strategies. Performance in delivering energy efficiency, load shift, and load shed was tested using price-based signals. The results showed that the DRL-based controller can produce cost savings of up to 39.6% as compared to the baseline controller, while maintaining similar thermal comfort in the building. The project team has also integrated the simulation components developed during this work as an OpenAIGym environment and made it publicly available so that prospective DRL researchers can leverage this environment to evaluate alternate DRL algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Deep reinforcement learning based optimization for a tightly coupled nuclear renewable integrated energy system

New ways to integrate energy systems to maximize efficiency are being sought to meet carbon emissions goals. Nuclear-renewable integrated energy system (NR-IES) concepts are a leading solution that couples a nuclear power plant with renewable energy, hydrogen generation plants, and energy storage systems, such that thermal and electrical power are dispatchable to fulfill grid-flexibility requirements while also producing hydrogen and maximizing revenue. Here, this paper introduces a deep reinforcement learning (DRL)-based framework to address the complex decision-making tasks for NR-IES. The objective is to maximize revenue by generating and selling hydrogen and electricity simultaneously according to their time-varying prices while keeping the energy flow in the subsystems in balance. A Python-based simulator for a NR-IES concept has been developed to integrate with OpenAI Gym and Ray/RLlib to enable an efficient and flexible computational framework for DRL research and development. Three state-of-the-art DRL algorithms have been investigated, including two-delayed deep deterministic policy gradient (TD3), soft-actor critic (SAC), proximal policy optimization (PPO), to illustrate DRL’s superiority for controlling NR-IES by comparing it with a conventional control approach, particle swarm optimization (PSO). In this effort, PPO has shown more-stable performance and also better generalization capability than SAC and TD3. Comparisons with PSO have demonstrated that, on average, PPO can achieve 13.9% more mean episode returns from the training process and 29.4% more mean episode returns from the testing process when different hydrogen-production targets are applied.

08 HYDROGEN↗
DOE OSTI2023

Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings

In recent years, advanced control strategies based on Deep Reinforcement Learning (DRL) proved to be effective in optimizing the management of integrated energy systems in buildings, reducing energy costs and improving indoor comfort conditions when compared to traditional reactive controllers. However, the scalability and implementation of DRL controllers are still limited since they require a considerable amount of time before converging to a near-optimal solution. This issue is currently addressed in literature through the offline pre-training of the DRL agent. However this solution results in two main critical issues: (1) the need to develop a building surrogate model to perform the training task, and (2) the need to perform a fine-tuning process over several training episodes to obtain a near-optimal control policy. In this context, this paper introduces an Online Transfer Learning (OTL) strategy that exploits two knowledge-sharing techniques, weight-initialization and imitation learning, to transfer a DRL control policy from a source office building to various target buildings in a simulation environment coupling EnergyPlus and Python. A DRL controller based on discrete Soft Actor–Critic (SAC) is trained on the source building to manage the operation of a cooling system consisting of a chiller and a thermal storage. Several target buildings are defined to benchmark the performance of the OTL strategy with that of a Rule-Based Controller (RBC) and two DRL-based control strategies, deployed in offline and online fashion. The strategy adopted for OTL emulates the real world implementation with a simulation process by implementing the transferred DRL agent for a single episode in the target buildings. Target buildings have the same geometrical features and are served by the same energy system as the source building, but differ in terms of weather conditions, electricity price schedules, occupancy patterns, and building envelope efficiency levels. The results show that the OTL strategy can reduce the cumulated sum of temperature violations on average by 50% and 80% respectively when compared to RBC and online DRL while enhancing the energy system operation with electricity cost savings ranging between 20% and 40%. Furthermore, the OTL agent performs slightly worse than the offline DRL controller but it does not require any modeling effort and can be implemented directly on target buildings emulating a real-world implementation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗