Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,729 records · Page 96

Swing Contract-Based Valuation for Distributed Energy Resources in Transactive Energy Systems: A Reinforcement Learning Approach

With the proliferation of distributed energy resources (DERs) and power grids with high fractions of renewable energy, market constructs are evolving to allow DERs to participate in multiple possible markets, at different levels of grid hierarchy. The effective participation of DERs in market environments is aided by swing contract-based pricing mechanisms, whereby DERs have a two-part compensation structure – one for their reservation/commitment and another for performancedriven ex-post payment for their actual mobilization during dispatch. In this paper, we propose a reinforcement learningbased (Q-learning) approach that allows a rational DER agent to select the market it wants to participate in within a composite market environment where individual markets are coordinated by possibly different actors. The proposed Q-learning framework aids DERs in their self-valuation by implicitly maximizing their own payoff through market participation, assuming a swing contract-based compensation structure. We complement our work through simulation-based investigations where factors affecting the DER decision making process, such as parametric uncertainties in market (and grid) environments, are studied.

Naqvi, Syed Ahsan Raza↗

LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers

Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.

Naug, Avisek [Hewlett Packard Enterprise]↗

Learning to Trigger: Reinforcement Learning at the Large Hadron Collider

High-throughput scientific facilities such as the Large Hadron Collider depend on real-time event filtering (\textit{triggering}) under tight constraints on bandwidth, latency, and storage. In practice, trigger menus are largely static and hand-tuned and can become suboptimal as detector conditions, pileup, and background composition drift over time. We cast online threshold tuning as a sequential decision-making problem: a reinforcement learning agent ingests streaming summaries of recent rates and signal-sensitive features and updates trigger thresholds to maximize signal efficiency while tracking a target background rate within a tolerance band. We adapt Group-Filtered Policy Optimization (GFPO) to streaming control and introduce two variants (GFPO-F, GFPO-FR) that enforce background rate feasibility during training. On a benchmark that emulates realistic collider operation, we study two representative triggers: a total transverse energy ($H_{T}$) trigger sensitive to pileup variation, and an anomaly-detection (AD) trigger based on reconstruction loss for rare or non-standard signatures. On Monte Carlo streams, our agent increases the fraction of in-tolerance time intervals by 48% ($H_T$) and 28% (AD), with a cumulative gain of up to 2% in signal efficiency on those in-tolerance intervals. Transferring from simulation to \emph{real} collision data (CMS Run 283408), the same agent, without fine-tuning, achieves a 56% ($H_T$) and 28% (AD) in-tolerance improvement over baselines, with further signal-efficiency gain on both triggers. To our knowledge, this is the \emph{first} demonstration of RL-based trigger control on real Large Hadron Collider collision data. Code is available at https://github.com/Zixind/GFPO_LHC (see repo for details).

Ding, Zixin [Chicago U.]↗

A Method for Finding Unknown Signals Using Reinforcement FFT Differencing

This note addresses a simple yet powerful method of discovering the spectral character of an unknown but intermittent signal buried in a background made up of a distribution of other signals. Knowledge of when the unknown signal is present and when it is not, along with samples of the combined signal when the unknown signal is present and when it is not are all that is necessary for this method. The method is based on reinforcing Fast Fourier Transform (FFT) power spectra when the signal of interest occurs and subtracting spectra when it does not. Several examples are presented. This method could be used to discover spectral components of unknown chemical species within spectral analysis instruments such as Mass Spectroscopy, Fourier Transform Infrared Spectroscopy (FTIR) and Gas Chromatography. In addition, this method can be used to isolate device loading signatures on power transmission lines.

99 GENERAL AND MISCELLANEOUS↗

Derivation of a crack opening deflection relationship for fibre reinforced concrete panels using a stochastic model: Application for predicting the flexural behaviour of round panels using stress crack opening diagrams

This study is aimed at proposing a simple analytical model to investigate the post-cracking behaviour of FRC panels, using an arbitrary tension softening, stress crack opening diagram, as the input. A new relationship that links the crack opening to the panel deflection is proposed. Due to the stochastic nature of material properties, the random fibre distribution, and other uncertainties that are involved in concrete mix, this relationship is developed from the analysis of beams having the same thickness using the Monte Carlo simulation (MCS) technique. The softening diagrams obtained from direct tensile tests are used as the input for the calculation, in a deterministic way, of the mean load displacement response of round panels. A good agreement is found between the model predictions and the experimental results.

36 MATERIALS SCIENCE↗

Numerical Modelling of Glass Fibre Reinforced Laminates Subjected to a Low Velocity Impact

This paper presents a series of numerical predictions of the perforation behaviour of glass fibre laminates subjected to quasi-static and low-velocity impact loading. Both shear and tensile failure criteria were used in the finite element models to simulate the post-failure processes via an automatic element removal procedure. The appropriate material properties, obtained through a series of uniaxial tension and bending tests on the composites, were used in the numerical models. Four, eight and sixteen ply glass fibre laminates panels were perforated at quasi-static rates and under low-velocity impact loading. Reasonably good correlation was obtained between the numerical simulations and the experimental results, both in terms of the failure modes and the load-deflection relationships before and during the penetration phase. The predicted impact energies of the GFRP panels were compared with the experimental data and reasonable agreement was observed.

36 MATERIALS SCIENCE↗

Polymer grafted aramid nanofiber reinforces immiscible waste polypropylene/poly(ethylene terephthalate)

Polypropylene (PP) and poly(ethylene terephthalate) (PET) are plastics commonly used for packaging because of their excellent barrier and mechanical properties. The properties of these plastics are often diminished after mechanical recycling, inevitably causing down-cycling. Furthermore, this problem is exacerbated when different kinds of polymers mix. Aramid nanofibers have the potential to improve the mechanical properties of polymers due to their excellent mechanical properties but their poor dispersion in polymers is a challenge. Grafting polymers onto nanofibers can help address this challenge. In this work, different loading levels (1%, 2%, and 5%) of polymer grafted aramid nanofibers (ANFs) are blended with waste PP/PET (90/10), simulating a PP waste stream containing traces of PET contaminants. Scanning electronic microscopy, rheology, and differential scanning calorimetry results show the affinity of PP functionalized aramid nanofibers (PP_ANF) towards the PP matrix. At 1 wt% of the nanofiber, the size of the PET droplets in the PP matrix of the PP_ANF blend range from 0.2 to 2.0 μm while that of unmodified ANF and PET_ANF blends are in the range of 0.1–6.2 and 0.5–7.4 μm, respectively. In summary, polymer grafted ANFs have the tendency of improving properties of its like polymers due to similarity in the grafting polymer and the polymer matrix.

36 MATERIALS SCIENCE↗

Reinforcement learning for real-time process control in high-temperature superconductor manufacturing

With high efficiency and low energy loss, high-temperature superconductors (HTS) have demonstrated their profound applications in various fields, such as medical imaging, transportation, accelerators, microwave devices, and power systems. The high-field applications of HTS tapes have raised the demand for producing cost-effective tapes with long lengths in superconductor manufacturing. However, achieving the uniform and enhanced performance of a long HTS tape is challenging due to the unstable growth conditions in the manufacturing process. Although it is confirmed that the process parameters during the advanced metal organic chemical vapor deposition (A-MOCVD) process influence the uniformity of the produced HTS tapes, the high-dimensional process parameter signals and their complicated interactions make it difficult to develop an effective control policy. In this paper, we propose a local measure for the uniformity of HTS tapes to provide instant feedback for our control policy. Then, we model the manufacturing of HTS tapes as a Markov decision process (MDP) with continuous state and action spaces to assess the instant reward in real time in our feedback control model. As our MDP involves continuous and high-dimensional state and action spaces, a neural fitted Q-iteration (NFQ) algorithm is adopted to solve the MDP with artificial neural network (ANN) function approximation. The collinearity of process parameters can restrict our capability of adjusting the process parameters, which is addressed by the principal component analysis (PCA) in our method. The control policy adjusts the PCA of process parameters using the NFQ algorithm. In conclusion, based on our case studies on real A-MOCVD dataset, the obtained control policy increases the average uniformity of tapes by 5.6% and performs especially well on sample HTS tapes with a low uniformity.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Biodegradable PBAT-Based Nanocomposites Reinforced with Functionalized Cellulose Nanocrystals from Pseudobombax munguba: Rheological, Thermal, Mechanical and Biodegradability Properties

Cellulose nanocrystals (CNC) were isolated from Munguba (Pseudobombax munguba) fibers and then functionalized with octadecyl isocyanate. Nanocomposites based on poly(butylene adipate-co-terephthalate) (PBAT) were prepared with different concentrations of cellulose nanocrystals (3, 5 and 7 wt%). We show that the addition of functionalized CNC leads to PBAT-based nanocomposites with enhanced thermal, rheological and mechanical performances, maintaining the biodegradability of the matrix. The better properties of the nanocomposites were related to the optimal amount and the uniform dispersion of CNC in PBAT. The study here presented expands the application of Munguba fibers, exploring their use to prepare PBAT-based biodegradable nanocomposites with improved properties. These nanocomposites have potential for replacement the conventional polymers in future applications with the advantage of exhibiting biodegradability.

36 MATERIALS SCIENCE↗

Effect of Prolonged Ultraviolet Radiation Exposure on the Blast Response of Fiber Reinforced Composite Plates

An experimental study with corresponding numerical simulations was conducted to evaluate the blast response of composite plates after prolonged exposure to ultraviolet radiation. Two composite materials were used in this study, namely carbon fiber/epoxy and glass fiber/vinyl ester. The composite materials consisted of four unidirectional fiber plies oriented in a [± 45°]s layup. The plates were placed in a QUV Accelerated Weathering Tester for 500 h of exposure on each face, totaling 1000 h of ultraviolet radiation exposure, equivalent to 2.5 MJ/m{sup 2} of energy. The QUV Accelerated Weathering Tester was equipped with eight UVA-340 lamps to provide real service life exposure conditions. Material characterization experiments were performed on virgin specimens, as well as specimens exposed to ultraviolet radiation for 1000 h to determine its effect on the mechanical properties of the composites. Blast experiments were conducted on the composite plates to investigate the dynamic response before and after exposure to ultraviolet radiation. The plates were clamped on all edges and subjected to an air blast using a shock tube apparatus. Three-dimensional digital image correlation was coupled with high-speed photography to obtain full-field displacements of the specimens during blast loading. Furthermore, three piezoelectric pressure transducers were mounted on the shock tube apparatus and utilized to measure the pressure history of the shock wave loading. The quasi-static material characterization showed that the tensile modulus E{sub 1} had minimal change after exposure to ultraviolet radiation for 1000 h. However, matrix-dominated properties such as the in-plane shear modulus G{sub 12} increased after exposure to ultraviolet light for both the carbon and glass fiber composites, which resulted in lower out-of-plane displacements during blast loading. For the carbon fiber plates, the forced vibration frequency induced by the transverse blast load was higher in plates exposed to ultraviolet radiation when compared to virgin specimens. However, for the glass fiber plates exposed to ultraviolet radiation, the forced vibration frequency decreased when compared to virgin specimens. Finally, the finite element simulations were in good agreement with the experimental results. The simulations revealed that the glass fiber plates exposed to ultraviolet radiation had higher failure strains at the boundary than virgin specimens, despite having higher stiffness. A parametric study in which higher loading was simulated also showed that the glass fiber plates experienced more damage after 1000 h of ultraviolet radiation exposure.

36 MATERIALS SCIENCE↗

A time-dependent chloride diffusion model for predicting initial corrosion time of reinforced concrete with slag addition

The effect of granulated blast furnace slag (GBFS) addition on the threshold chloride concentration (TCC) for rebar corrosion was investigated. A modified diffusion model, coupled with a time-dependent effective diffusion coefficient and surface chloride concentration, was proposed to predict the chloride profile. The corrosion initiation time was estimated based on the model predictions and the measured TCC. The results indicate that adding GBFS decreases the TCC by lowering the pH value of the pore solution. The evolution of corrosion potential and current density is found to obey a 3-parameter Weibull distribution. MIP tests show that adding GBFS contributes to refinement of pore structure by decreasing the fraction of large capillary pores. The time-dependent model exhibits good predictive strength and helps understand how GBFS addition delays the corrosion initiation by retarding the chloride diffusion, though a lower TCC is obtained.

36 MATERIALS SCIENCE↗

Biologically inspired reinforcement using polydopamine of polymer bound composites

Interfacial strength plays a critical role in the mechanical properties of a composite system. In this work, a proven mock high explosive (HE), 5-iodo-2’-deoxyuridine (IDOX) was coated by the adhesive promoting polymer polydopamine (PDA), then formulated with an Estane binder system to form an improved HE mock composite. An extensive comparison study of mechanical properties was performed to determine the effects, if any, of PDA on the composite system. To aid in interpretation of the mechanical test data, neutron reflectometry experiments were performed to determine the thickness and structure of the as deposited PDA films. The mechanical performance of PDA-treated IDOX was then compared to untreated specimens by compression testing in the Brazilian disk geometry. The results strongly suggested that the structure of PDA consists of two layers: one low in density caused by polymer agglomerates attaching to the substrate during polymerization, and the second high in density caused by controlled linear deposition of polydopamine. Finally, the mechanical testing showed that PDA greatly increased the stiffness and yield strength of IDOX based mocks without causing any disruption to the underlying crystal system.

42 ENGINEERING↗

Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters

In this report, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Residual microstructure and damage geometry associated with high speed impact crater in Al{sub 2}O{sub 3} and TiB{sub 2} particles reinforced 2024 Al composite

The resistance of Al{sub 2}O{sub 3} + TiB{sub 2}/2024Al composite to hypervelocity impact was tested by a two-stage light gas gun. The impact damage behaviors of the Al{sub 2}O{sub 3} + TiB{sub 2}/2024Al composite by different-sized Al projectiles with a velocity of 2.49 km/s and the residual microstructures associated with the crater impacted by a 1.2 mm aluminum projectile were investigated by transmission electron microscopy and high-resolution transmission electron microscopy. Both the diameters of craters at front face and spalling areas at back face increased with the aluminum projectile diameter. The diameter of perforation on the 2 mm thick Al{sub 2}O{sub 3} + TiB{sub 2}/Al composite target was zero when impacted by 1.2 mm aluminum projectile and it increased to 2.4 mm when the projectile diameter was 1.5 mm, indicating that the critical perforation diameter of the aluminum projectile was between 1.2 mm and 1.5 mm when the 2 mm thick Al{sub 2}O{sub 3} + TiB{sub 2}/Al composite target was impacted by 2.49 km/s aluminum projectiles. The diameter of perforation increases with the diameter of Al projectile. In addition, under each impact condition, the diameters of craters at front face were smaller than that of spalling areas at back face. Microstructure observations by transmission electron microscopy demonstrated four characteristics: stacking faults around TiB{sub 2} particle and dislocations within the TiB{sub 2} particle; twins in the Al{sub 2}O{sub 3} particle; recrystal grains in 2024 Al matrix; and mixture of amorphous microstructure and nanograins in the matrix. - Highlights: Black-Right-Pointing-Pointer Stacking faults were produced around the edge of TiB{sub 2} particle after impact. Black-Right-Pointing-Pointer Twins with the twin plane of (2{sup Macron }112) were observed in Al{sub 2}O{sub 3} particle after impact. Black-Right-Pointing-Pointer Recrystal grains with size of 100 nm were formed in aluminum matrix after impact. Black-Right-Pointing-Pointer Mixture of amorphous microstructure and nanograins was also found in matrix.

36 MATERIALS SCIENCE↗