Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

An analysis of physics limited dispatch of nuclear renewable integrated energy systems using deep reinforcement learning and dynamic modeling

Previous approaches to dispatching nuclear integrated energy systems (NIES) have focused on the profitability and flexibility of these systems to operate on energy grids with highly variable pricing. However, due to the complexity involved in modeling and designing these systems, there has been less emphasis on ensuring that these dispatch strategies are physically achievable. It is imperative to develop methods that allow the system to remain within the desired NIES operating conditions and perform this based on realistic limited forecasted information. This research employs next generation artificial intelligence, namely deep reinforcement learning (DRL), and a dynamic system model written in Modelica to find a safe and profitable dispatch strategy for a solar nuclear hybrid design. The DRL agent is shown to find a novel dispatch strategy that manages both power ramping and power levels while respecting operational limits. This DRL-based dispatch is compared to other dispatching strategies including an optimal design solution from mixed integer linear programming (MILP). It is found that incorporating the physics of such a tightly coupled NIES limits the profitability of the MILP-based dispatch strategy. As a result, the MILP solution overestimates the design’s generated revenue. In contrast, DRL significantly reduces the number of breaches of safe operational conditions during energy arbitrage while maintaining profitability. Furthermore, this work paves the way for a more detailed assessment of NIES profitability and could be used to aid operator decisions on future NIES projects.

14 - SOLAR ENERGY↗
DOE OSTI2024

Mild chemical recycling of carbon fiber-reinforced epoxy composites in aqueous buffers and development of hydrothermally recyclable vitrimer composites from recyclates

Carbon fiber-reinforced polymer (CFRP) composites have gained widespread adoption across diverse industries. However, their inherent stability, stemming from the crosslinked structure of the matrix resin, poses a significant challenge in managing the growing volume of CFRP waste. Consequently, there is a pressing need for an eco-friendly upcycling method that effectively recovers and reuses both the valuable carbon fibers and the polymer matrix from CFRP waste. This study presents a novel approach for upcycling CFRP waste under environmentally friendly conditions. Firstly, CFRP waste with an amine-cured epoxy matrix was completely decomposed in aqueous buffer solutions under mild conditions (≤ 220 °C, pH = 4.8). Further, decomposed matrix polymer (DMP) was employed to transform conventional epoxy resins into recyclable vitrimers. Lastly, new composites that can be hydrothermally recycled were fabricated using both DMP and recovered carbon fibers. As a result, this work proposes a novel strategy for achieving a circular economy within the CFRP industry.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Interfacial bond characterization of epoxy adhesives to aluminum alloy and carbon fiber-reinforced polyamide by vibrational spectroscopy

Vibrational spectroscopic technique has been utilized to investigate interfacial bonding chemistry of two epoxy adhesive products, XP0012 and XP5005F, on plasma-treated AA6061 and carbon fiber-reinforced polyamide 66 (CFRP-PA66) surfaces. The change in vibrational peak ratios was measured by attenuated total reflectance-Fourier transform infrared (ATR-FTIR) spectroscopy to deduce bonding mechanisms. Both adhesives showed strong crosslinking polymerization of hydroxyl- and amine-initiated epoxy ring opening on AA6061 surface, but on CFRP surface XP0012 formed a simple amide linkage by the reaction of surface hydroxyl groups and nitrile groups of curing agent, and XP5005F formed a crosslinked network by hydroxyl-initiated epoxy ring opening polymerization. The different interfacial bonding formation of two adhesives on CFRP-PA66 surface is attributed to additive effect. Addition of additives to epoxy adhesives (XP5005F) changed the interfacial bonding mechanism on CFRP-PA66 surface, rather forming hydroxyl-initiated epoxy opening crosslinking polymerization than a simple amide bond formation (XP0012). The interfacial bonding chemistry was also proved by addition of bisphenol A (BA) to a simplified model diglycidyl ether of bisphenol A/dicyandiamide (DGEBA/DICY) epoxy system. When BA was added to the model DGEBA/DICY system, epoxy ring gradually decreased on CFRP-PA66 surface, while without BA, DGEBA/DICY showed only decrease in a nitrile peak intensity in ATR-FTIR. In conclusion, the foregoing different types of interfacial chemical bonds at the adhesive/CFRP-PA66 interfaces can affect the lap shear behavior of the joint.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Acetolysis for Epoxy-Amine Carbon Fibre-Reinforced Polymer Recycling

Carbon fibre-reinforced polymers (CFRPs) are used in many applications in the global energy transition, including for lightweighting aircraft and vehicles and in wind turbine blades, shipping containers and gas storage vessels1,2,3,4. Given the high cost and energy-intensive manufacture of CFRPs5,6,7, recycling strategies are needed that recover intact carbon fibres and the epoxy-amine resin components. Here we show that acetic acid efficiently depolymerizes both aliphatic and aromatic epoxy-amine thermosets used in CFRPs to recoverable monomers, yielding pristine carbon fibres. Deconstruction of materials from multiple sectors demonstrates the broad applicability of this approach, providing clean fibres from 2 h reactions. The optimal conditions were scaled to 80.0 g of post-consumer CFRPs, and demonstrative composites were fabricated from the recycled carbon fibres, which were recycled two more times, maintaining their strength throughout. Process modelling and techno-economic analysis, with feedstock cost informed by wind turbine blade waste generation8, indicates this method is cost effective, with a minimum selling price of US$1.50 per kg for recycled carbon fibres whereas life cycle assessment shows process greenhouse gas emissions around 99% lower than virgin carbon fibre production. Overall, this approach could enable recycling of industrial CFRPs as it provides clean, mechanically viable recycled carbon fibres and recoverable resin monomers from the thermoset.

09 BIOMASS FUELS↗
DOE OSTI2023

Reinforcement learning-based design of shape-changing metamaterials

During the last decade, artificially architected materials have been designed to obtain properties unreachable by naturally occurring materials, whose properties are determined by their atomic structure and chemical composition. In this work, we implement a new reinforcement learning (RL) method able to rationally design unique metamaterial structures at the nano-, micro-, and macroscale, which change shape during operational conditions. As an example, we apply this method to design nanostructured silicon anodes for Li-ion batteries (LIBs). The RL model is designed to apply different actions and predict change during operational conditions. The multi-component reward function comprises an increase in the total storage capacity of the resulting battery electrode and structural parameters, such as the minimum distance between the individual components of the nanostructure. Upon experimental validation using a polymer-based 3D printing technique, we expect that the newly discovered structures improve the current Si-based LIB anodes state-of-the-art by almost three times and almost ten times the current commercial LIB based on a graphitic anode. Furthermore, this RL-based optimization method opens up vast design space for other responsive metamaterials with tailored properties and pre-programmed structural transformation.

25 ENERGY STORAGE↗
DOE OSTI2014

Evaluating the integrity of the reinforced concrete structure repaired by epoxy injection using simulated transfer function of impact-echo response

Cracks and honeycombs are often found inside reinforced concrete (RC) structure caused by excessive external force, or improper casting of concrete. The repairing method usually involves epoxy injection. The impact-echo method, which is a sensitive for detecting of the interior voids, may not be applicable to assess the integrity of the repaired member as both air and epoxy are less in acoustic impedances. In this study, the repaired RC structure was evaluated by the simulated transfer function of the IE displacement waveform where the R-wave displacement waveform is used as a base of a simulated force-time function. The effect of different thickness of the epoxy layer to the amplitude corresponding to the interface is studied by testing on specimen containing repaired naturally delaminated cracks with crack widths about 1 mm, 3 mm and 5 mm. The impact-echo responses were compared with the drilling cores at the test positions. The results showed the cracks were not fully filled with epoxy when the peak amplitude corresponding to the interface dropped less than 20%. The peak corresponding to the thicker epoxy layer tends to be larger in amplitude. A field study was also performed on a column damaged by earthquake before and after repairing.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2014

Scanning tone burst eddy-current thermography (S-TBET) for NDT of carbon fiber reinforced plastic (CFRP) components

Tone Burst Eddy Current technique uses eddy current to apply transient heating inside a component and uses a conventional IR camera for visualization of the response to the transient heating. This technique has been earliest demonstrated for metallic components made of AL, Steel, Stainless Steel, etc., and for detection of cracks, corrosion and adhesive dis-bonds. Although, not nearly as conducting as metals, the Carbon Fibre Reinforced Plastic (CFRP) material absorbs measurable electromagnetic radiation in the frequency range above 10 kHz. When the surface temperature is observed on the surface that is being heated (defined as the surface just beneath and slightly to one side of the heating coil), the surface temperature increases with increasing frequency because the internal heating increases with frequency. A 2-D anisotropic transient Eddy current heating and thermal conduction model has been developed that provides a reasonable description of the processes described above. The inherent anisotropy of CFRP laminates is included in this model by calculating the heating due to three superimposed, tightly coupled isotropic layers having a specified ply-layup. The experimental apparatus consists of an induction heating coil and an IR camera with low NETD and high frame rates. The coil is moved over the sample using a stepper motor controlled manipulator. The IR data recording is synchronized with the motion control to provide a movie of the surface temperature over time. Several components were evaluated for detection of impact damage, location of stiffeners, etc. on CFRP components.

42 ENGINEERING↗
DOE OSTI2014

Oxygen-assisted multipass cutting of carbon fiber reinforced plastics with ultra-short laser pulses

Deep multipass cutting of bidirectional and unidirectional carbon fiber reinforced plastics (CFRP) with picosecond laser pulses was investigated in different static atmospheres as well as with the assistance of an oxygen or nitrogen gas flow. The ablation rate was determined as a function of the kerf depth and the resulting heat affected zone was measured. An assisting oxygen gas flow is found to significantly increase the cutting productivity, but only in deep kerfs where the diminished evaporative ablation due to the reduced laser fluence reaching the bottom of the kerf does not dominate the contribution of reactive etching anymore. Oxygen-supported cutting was shown to also solve the problem that occurs when cutting the CFRP parallel to the fiber orientation where a strong deformation and widening of the kerf, which temporarily slows down the process speed, is revealed to be typical for processing in standard air atmospheres.

36 MATERIALS SCIENCE↗
DOE OSTI2014

Thermal expansion of multiwall carbon nanotube reinforced nanocrystalline silver matrix composite

Multiwall carbon nanotube reinforced silver matrix composite was fabricated by novel molecular level mixing method, which involves nucleation of Ag ions inside carbon nanotube dispersion at the molecular level. As a result the carbon nanotubes get embedded within the powder rather than on the surfaces. Micro structural characterization by X- ray diffraction and scanning electron microscopy reveals that the nanotubes are homogeneously dispersed and anchored within the matrix. The thermal expansion of the composite with the multiwall nanotube content (0, 1.5 vol%) were investigated and it is found that coefficient of thermal expansion decreases with the addition of multiwall nanotube content and reduce to about 63% to that of pure Ag.

36 MATERIALS SCIENCE↗
DOE OSTI2015

The mechanical properties measurement of multiwall carbon nanotube reinforced nanocrystalline aluminum matrix composite

Nanocrystalline aluminum matrix composite containing carbon nanotubes were fabricated using physical mixing method followed by cold pressing. The microstructure of the composite has been investigated using X-ray diffraction, scanning electron microscopy and energy dispersive spectroscopy techniques. These studies revealed that the carbon nanotubes were homogeneously dispersed throughout the metal matrix. The consolidated samples were pressureless sintered in inert atmosphere to further actuate a strong interface between carbon nanotubes and aluminum matrix. The nanoindentation tests carried out on considered samples showed that with the addition of 0.5 wt% carbon nanotubes, the hardness and elastic modulus of the aluminum matrix increased by 21.2 % and 2 % repectively. The scratch tests revealed a decrease in the friction coefficient of the carbon nanotubes reinforced composite due to the presence of lubricating interfacial layer. The prepared composites were promising entities to be used in the field of sporting goods, construction materials and automobile industries.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

Numerical analysis on seismic behavior of reinforced concrete beam to concrete filled steel tubular column connections with ring-beam

This paper presents numerical study of the seismic behavior of reinforced concrete beam to concrete filled steel tube column connections with ring-beam. The material stress-strain relations, element type and boundary condition are selected, which are consistent with actual situation. Then the seismic behavior of this type of joint are researched by ABAQUS, and finite element analyses are conducted under cyclic loading. Its parameters are discussed including thickness of steel tubular column wall, sectional dimension of the ring-beam and strength of the core concrete. The results show that the ultimate capacity of the connections is improved with sectional dimension of the ring-beam increased. In the meanwhile, the influence on skeleton curve of the joints is slight of which included thickness of steel tubular column wall and strength of the core concrete.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Reinforcement learning based hybrid bond-order coarse-grained interatomic potentials for exploring mesoscale aggregation in liquid–liquid mixtures

Exploring mesoscopic physical phenomena has always been a challenge for brute-force all-atom molecular dynamics simulations. Although recent advances in computing hardware have improved the accessible length scales, reaching mesoscopic timescales is still a significant bottleneck. Coarse-graining of all-atom models allows robust investigation of mesoscale physics with a reduced spatial and temporal resolution but preserves desired structural features of molecules, unlike continuum-based methods. Here, we present a hybrid bond-order coarse-grained forcefield (HyCG) for modeling mesoscale aggregation phenomena in liquid–liquid mixtures. The intuitive hybrid functional form of the potential offers interpretability to our model, unlike many machine learning based interatomic potentials. We parameterize the potential with the continuous action Monte Carlo Tree Search (cMCTS) algorithm, a reinforcement learning (RL) based global optimizing scheme, using training data from all-atom simulations. The resulting RL-HyCG correctly describes mesoscale critical fluctuations in binary liquid–liquid extraction systems. cMCTS, the RL algorithm, accurately captures the mean behavior of various geometrical properties of the molecule of interest, which were excluded from the training set. The developed potential model along with the RL-based training workflow could be applied to explore a variety of other mesoscale physical phenomena that are typically inaccessible to all-atom molecular dynamics simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Entanglement engineering of optomechanical systems by reinforcement learning

Entanglement is fundamental to quantum information science and technology, yet controlling and manipulating entanglement—so-called entanglement engineering—for arbitrary quantum systems remains a formidable challenge. There are two difficulties: the fragility of quantum entanglement and its experimental characterization. We develop a model-free deep reinforcement-learning (RL) approach to entanglement engineering, in which feedback control together with weak continuous measurement and partial state observation is exploited to generate and maintain desired entanglement. We employ quantum optomechanical systems with linear or nonlinear photon–phonon interactions to demonstrate the workings of our machine-learning-based entanglement engineering protocol. In particular, the RL agent sequentially interacts with one or multiple parallel quantum optomechanical environments, collects trajectories, and updates the policy to maximize the accumulated reward to create and stabilize quantum entanglement over an arbitrary amount of time. The machine-learning-based model-free control principle is applicable to the entanglement engineering of experimental quantum systems in general.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Reinforcement learning for bluff body active flow control in experiments and simulations

Significance Reinforcement learning (RL) has been applied effectively in games and robotic manipulation. We demonstrate the effectiveness of RL in experimental fluid mechanics by applying it to reduce the drag of circular cylinders in turbulent flow, a canonical fluid–structure interaction problem. Although physics agnostic, RL managed to reduce the drag by 30 % or reach another specified optimum point very quickly. Following this discovery, we used high-fidelity simulations to probe the underlying physical mechanisms so that the discovered control techniques can be generalized to other similar flow problems. More broadly, RL-guided active control can lead to efficient exploration of additional flow-control strategies in experimental fluid mechanics, potentially paving the way for accelerating scientific discovery and different designs in flow-related engineering problems.

42 ENGINEERING↗
DOE OSTI2022

On-policy learning-based deep reinforcement learning assessment for building control efficiency and stability

Artificial intelligence technologies have emerged as a game changer not only in specific applications such as image recognition and machine translation but also in many scientific domains. In particular, as deep reinforcement learning (DRL) has shown great success in complex control problems, DRL-based control has been considered as a potential solution to efficiently control and manage building systems. However, broad assessment of DRL-based building control is still required to characterize their pros and cons in comparison with conventional building control methods (e.g., rule-based feedback controls). In this paper, we assessed DRL-based controls with on-policy learning-based algorithms and continuous control actions for cooling control of large office buildings in the summer season to minimize whole-building energy use and occupant discomfort. We compared DRL-based control methods with two baseline control methods: (1) a pre-determined schedule with supply temperature and static pressure setpoints, and (2) advanced reset method that adjusts setpoints based on heuristic rules, i.e., ASHRAE Guideline 36. We also tested the DRL algorithms to evaluate their performances in multiple climate locations. We found that DRL-based control methods outperformed the baseline control methods in terms of energy savings while maintaining a thermal comfort. DRL reduced energy use between ~4%–22% on average compared to the baseline methods, depending on climate location. We also evaluated DRL-based control in terms of control stability and showed that DRL-based methods should address the span of hardware lifetimes in practical operations.

control stability↗
DOE OSTI2026

Demonstration of reconstruction-free static magnetic control of DIII-D plasma with deep reinforcement learning

This paper presents the development and experimental validation of a reinforcement learning (RL)-based magnetic controller on the DIII-D tokamak. The controller directly maps raw magnetic diagnostic signals to actuator commands, replacing the traditional isoflux control algorithm based on equilibrium reconstruction. Four RL controllers are trained using the Soft Actor–Critic algorithm with an asymmetric Actor–Critic architecture in the NSFsim simulator. All controllers are deployed in the DIII-D Plasma Control System and operated with a 4 kHz feedback loop. Two randomization strategies are evaluated during training: evolving kinetic profiles and fixed kinetic profiles within each episode. The latter approach is found to better capture experimental deviations in the current density profile and to provide overall improved control performance. Robust operation is demonstrated across heating power scans in both L- and H-mode plasmas, as well as during transient events such as L–H transitions and pellet injections. Control errors in plasma shape and radial position remained within 1.5–2.0 cm and 1 cm, respectively. A notable discrepancy was observed in the vertical X-point position, with errors of up to approximately 4 cm, attributed to the current density distribution mismatches between simulations and experiments.

DIII-D↗
DOE OSTI2021

Reinforcement learning in discrete action space applied to inverse defect design

Abstract Reinforcement learning (RL) algorithms that include Monte Carlo Tree Search (MCTS) have found tremendous success in computer games such as Go, Shiga and Chess. Such learning algorithms have demonstrated super-human capabilities in navigating through an exhaustive discrete action search space. Motivated by their success in computer games, we demonstrate that RL can be applied to inverse materials design problems. We deploy RL for a representative case of the optimal atomic scale inverse design of extended defects via rearrangement of chalcogen (e.g. S) vacancies in 2D transition metal dichalcogenides (e.g. MoS 2 ). These defect rearrangements and their dynamics are important from the perspective of tunable phase transition in 2D materials i.e. 2H (semi-conducting) to 1T (metallic) in MoS 2 . We demonstrate the ability of MCTS interfaced with a reactive molecular dynamics simulator to efficiently sample the defect phase space and perform inverse design—starting from randomly distributed S vacancies, the optimal defect rearrangement of defects corresponds a line defect of S vacancies. We compare MCTS performance with evolutionary optimization i.e. genetic algorithms and show that MCTS converges to a better optimal solution (lower objective) and in fewer evaluations compared to GA. We also comprehensively evaluate and discuss the effect of MCTS hyperparameters on the convergence to solution. Overall, our study demonstrates the effectives of using RL approaches that operate in discrete action space for inverse defect design problems.

42 ENGINEERING↗
DOE OSTI2021

Gaming the beamlines—employing reinforcement learning to maximize scientific outcomes at large-scale user facilities

Abstract Beamline experiments at central facilities are increasingly demanding of remote, high-throughput, and adaptive operation conditions. To accommodate such needs, new approaches must be developed that enable on-the-fly decision making for data intensive challenges. Reinforcement learning (RL) is a domain of AI that holds the potential to enable autonomous operations in a feedback loop between beamline experiments and trained agents. Here, we outline the advanced data acquisition and control software of the Bluesky suite, and demonstrate its functionality with a canonical RL problem: cartpole. We then extend these methods to efficient use of beamline resources by using RL to develop an optimal measurement strategy for samples with different scattering characteristics. The RL agents converge on the empirically optimal policy when under-constrained with time. When resource limited, the agents outperform a naive or sequential measurement strategy, often by a factor of 100%. We interface these methods directly with the data storage and provenance technologies at the National Synchrotron Light Source II, thus demonstrating the potential for RL to increase the scientific output of beamlines, and layout the framework for how to achieve this impact.

36 MATERIALS SCIENCE↗