Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Stereolithography 3D Printing of Lignin-Reinforced Composites with Enhanced Mechanical Properties

Due to the availability, biodegradability, and biological effects, lignin has emerged as an interesting alternative to petroleum-based compounds for developing sustainable chemicals, materials, and composites. In this study, lignin at various concentrations was incorporated into methacrylate resin via solution blending to fabricate lignin-reinforced composites using stereolithography apparatus three-dimensional printing. Softwood kraft lignin in the amounts of 0.2, 0.4, 0.5, 0.8, and 1.0 wt % in the methacrylate resin was used as a printing ink, and the gel contents and relative contents of the residual resin in the printed samples with various lignin concentrations were measured. The effects of the lignin on the ultimate mechanical properties of the non-postcured and postcured printed composites were determined. The tensile testing results revealed that the incorporation of lignin in the composite increased the tensile strength by 46–64% and Young’s modulus by 13–37% for the postcured printed composites compared with that of the control sample (no lignin added). Employing a 0.4 wt % softwood kraft lignin, the tensile strength of the postcured printed composite reached the highest value of 49.0 MPa, which was a 60% increase in comparison to that of the control sample with 30.7 MPa. Scanning electron microscopy images of the fracture samples illustrated that the lignin-incorporated composites exhibited a rougher fracture surface that can presumably dissipate the stress, which could be a contributing factor for the mechanical enhancement.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Chemically Fueled Reinforcement of Polymer Hydrogels

Carbodiimide-fueled anhydride bond formation has been used to enhance the mechanical properties of permanently crosslinked polymer networks, giving materials that exhibit transitions from soft gels to covalently reinforced gels, eventually returning to the original soft gels. Temporary changes in mechanical properties result from a transient network of anhydride crosslinks, which eventually dissipate by hydrolysis. Over an order of magnitude increase in the storage modulus is possible through carbodiimide fueling. Here, the time-dependent mechanical properties can be modulated by the concentration of carbodiimide, temperature, and primary chain architecture. Because the materials remain rheological solids, new material functions such as temporally controlled adhesion and rewritable spatial patterns of mechanical properties have been realized.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Autonomous reinforcement learning agent for chemical vapor deposition synthesis of quantum materials

Abstract Predictive materials synthesis is the primary bottleneck in realizing functional and quantum materials. Strategies for synthesis of promising materials are currently identified by time-consuming trial and error and there are no known predictive schemes to design synthesis parameters for materials. We use offline reinforcement learning (RL) to predict optimal synthesis schedules, i.e., a time-sequence of reaction conditions like temperatures and concentrations, for the synthesis of semiconducting monolayer MoS 2 using chemical vapor deposition. The RL agent, trained on 10,000 computational synthesis simulations, learned threshold temperatures and chemical potentials for onset of chemical reactions and predicted previously unknown synthesis schedules that produce well-sulfidized crystalline, phase-pure MoS 2 . The model can be extended to multi-task objectives such as predicting profiles for synthesis of complex structures including multi-phase heterostructures and can predict long-time behavior of reacting systems, far beyond the domain of molecular dynamics simulations, making these predictions directly relevant to experimental synthesis.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Autonomous reinforcement learning agent for stretchable kirigami design of 2D materials

Abstract Mechanical behavior of 2D materials such as MoS 2 can be tuned by the ancient art of kirigami. Experiments and atomistic simulations show that 2D materials can be stretched more than 50% by strategic insertion of cuts. However, designing kirigami structures with desired mechanical properties is highly sensitive to the pattern and location of kirigami cuts. We use reinforcement learning (RL) to generate a wide range of highly stretchable MoS 2 kirigami structures. The RL agent is trained by a small fraction (1.45%) of molecular dynamics simulation data, randomly sampled from a search space of over 4 million candidates for MoS 2 kirigami structures with 6 cuts. After training, the RL agent not only proposes 6-cut kirigami structures that have stretchability above 45%, but also gains mechanistic insight to propose highly stretchable (above 40%) kirigami structures consisting of 8 and 10 cuts from a search space of billion candidates as zero-shot predictions.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Multi-reward reinforcement learning based development of inter-atomic potential models for silica

Abstract Silica is an abundant and technologically attractive material. Due to the structural complexities of silica polymorphs coupled with subtle differences in Si–O bonding characteristics, the development of accurate models to predict the structure, energetics and properties of silica polymorphs remain challenging. Current models for silica range from computationally efficient Buckingham formalisms (BKS, CHIK, Soules) to reactive (ReaxFF) and more recent machine-learned potentials that are flexible but computationally costly. Here, we introduce an improved formalism and parameterization of BKS model via a multireward reinforcement learning (RL) using an experimental training dataset. Our model concurrently captures the structure, energetics, density, equation of state, and elastic constants of quartz (equilibrium) as well as 20 other metastable silica polymorphs. We also assess its ability in capturing amorphous properties and highlight the limitations of the BKS-type functional forms in simultaneously capturing crystal and amorphous properties. We demonstrate ways to improve model flexibility and introduce a flexible formalism, machine-learned ML-BKS, that outperforms existing empirical models and is on-par with the recently developed 50 to 100 times more expensive Gaussian approximation potential (GAP) in capturing the experimental structure and properties of silica polymorphs and amorphous silica.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Electricity use in big area additive manufacturing of fiber-reinforced polymer composites

In recent years, additive manufacturing (AM), especially large-format additive manufacturing (LFAM), has gained momentum in the manufacturing industry. While LFAM offers benefits over conventional manufacturing processes, such as minimizing material waste and providing vast geometric freedom, assessing its sustainability remains challenging due to limited data, particularly on energy consumption. Most existing data pertain to small-scale or desktop AM and are not directly applicable to LFAM. In this study, we conducted real-time measurements of electricity usage for a type of LFAM known as big area additive manufacturing (BAAM), which typically uses fiber-reinforced polymer pellets as feedstock. We collected electricity usage data from fifteen printing jobs over two months in an industrial production setting. These data fill the existing gap and can be reused to enhance the community’s understanding of LFAM electricity usage, support further research, and promote sustainable development in advanced manufacturing technologies.

ecology↗
DOE OSTI2022

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

Abstract A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement learning (RL) is a powerful yet relatively uncharted method in spatiotemporally extended systems. In this study, we put forth a modular dynamic closure modeling and discovery framework to stabilize the Galerkin projection based reduced order models that may arise in many nonlinear spatiotemporal dynamical systems with quadratic nonlinearity. However, a key element in creating a robust RL agent is to introduce a feasible reward function, which can be constituted of any difference metrics between the RL model and high fidelity simulation data. First, we introduce a multi-modal RL to discover mode-dependant closure policies that utilize the high fidelity data in rewarding our RL agent. We then formulate a variational multiscale RL (VMRL) approach to discover closure models without requiring access to the high fidelity data in designing the reward function. Specifically, our chief innovation is to leverage variational multiscale formalism to quantify the difference between modal interactions in Galerkin systems. Our results in simulating the viscous Burgers equation indicate that the proposed VMRL method leads to robust and accurate closure parameterizations, and it may potentially be used to discover scale-aware closure models for complex dynamical systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Hierarchical deep reinforcement learning reveals a modular mechanism of cell movement

Time-lapse images of cells and tissues contain rich information about dynamic cell behaviours, which reflect the underlying processes of proliferation, differentiation and morphogenesis. However, we lack computational tools for effective inference. Here we exploit deep reinforcement learning (DRL) to infer cell–cell interactions and collective cell behaviours in tissue morphogenesis from three-dimensional (3D) time-lapse images. We use hierarchical DRL (HDRL), known for multiscale learning and data efficiency, to examine cell migrations based on images with a ubiquitous nuclear label and simple rules formulated from empirical statistics of the images. When applied to Caenorhabditis elegans embryogenesis, HDRL reveals a multiphase, modular organization of cell movement. Imaging with additional cellular markers confirms the modular organization as a novel migration mechanism, which we term sequential rosettes. Furthermore, HDRL forms a transferable model that successfully differentiates sequential rosettes-based migration from others. Our study demonstrates a powerful approach to infer the underlying biology from time-lapse imaging without prior knowledge.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

A mathematical analysis of carbon fixing materials that grow, reinforce, and self-heal from atmospheric carbon dioxide

Carbon fixing materials are a new class of self-healing, self-reinforcing materials we have introduced that utilize ambient CO 2 to chemically add to an ever extending carbon backbone. This class of materials can utilize biological or non-biological photocatalysts and support a wide range of potential backbone chemistries. However, there is no analysis to date that describes their fundamental limits in terms of chemical kinetics and mass transfer. In this computational study, we employ a reaction engineering, and materials science analysis to answer basic questions about the maximum growth rate, photocatalytic requirements and limits of applicable materials. Our proposed mathematical framework envelops three main functions required for carbon fixing materials: (1) adsorption of CO 2 from air, (2) photocatalytic reduction of CO 2 into selected monomers, and (3) polymerization of CO 2 -derived monomers. First, by performing a Damköhler number analysis, we derive criteria for the cross over from kinetic control to mass transfer limited growth, setting upper limits on performance of a potential photocatalyst. Next, we analyze photocatalytic reduction of CO 2 to single carbon products, using known catalytic pathways and kinetic data. We identify formaldehyde as a C1 intermediate having unique potential for incorporation into the material backbone of carbon fixing materials. As an example, we find that a diamond-shaped reaction network graph for CO 2 reduction, passing through CO and HCOOH intermediates, accurately describes kinetic data for cobalt-promoted TiO 2 nanoparticles at room temperature and 1 atm CO 2 . Finally, as an applied case study, we consider and analyze the example of a carbon fixing poly(oxymethylene) system with embedded catalyst promoting the photocatalytic reduction of CO 2 to formaldehyde. The latter then trimerizes to a trioxane monomer which subsequently polymerizes to polyoxymethylene. This reaction engineering analysis introduces benchmarks for carbon fixing materials with respect to achievable rates of photocatalysis, adsorption, and polymerization. These results should prove valuable for the design, evaluation, and benchmarking of this emergent and new class of environmentally sustainable materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Harnessing deep reinforcement learning to construct time-dependent optimal fields for quantum control dynamics

Here, we present an efficient deep reinforcement learning (DRL) approach to automatically construct time-dependent optimal control fields that enable desired transitions in dynamical chemical systems. Our DRL approach gives impressive performance in constructing optimal control fields, even for cases that are difficult to converge with existing gradient-based approaches. We provide a detailed description of the algorithms and hyperparameters as well as performance metrics for our DRL-based approach. Our results demonstrate that DRL can be employed as an effective artificial intelligence approach to efficiently and autonomously design control fields in quantum dynamical chemical systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Decentralised Reinforcement Learning for Dynamic Cyberattack Response in Microgrid Networks

Microgrids rely on communication networks for reliable operation, which makes them inherently vulnerable to cyberattacks. Such attacks can destabilise system dynamics and drive states away from their nominal operating trajectories. Although several physics-informed and machine learning-based strategies have been developed to counter these threats, the rapidly evolving cyber landscape enables adversaries to bypass static defences or rules-based mitigation approaches. This paper proposes a dynamic, online-trained and fully decentralised reinforcement learning (RL)-based cyberattack response framework to protect microgrids from evolving cyberattacks. The proposed framework deploys multiple deep Q-networks (DQNs), each associated with a distributed energy resource (DER), to enable localised and adaptive attack mitigation. In this framework, each DQN processes local voltage and frequency measurements—combined with intrusion detection system (IDS) alerts—as observations and rewards to guide decision-making. Extensive simulation studies demonstrate the robustness of the proposed framework under diverse attack scenarios and varying IDS-induced detection delays. Comparative analysis highlights its superiority over existing static or preexisting rules-based mitigation approaches. Finally, we present an analysis that shows the framework's scalability to real-life microgrids with more interacting agents.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Damping inter-area oscillation using reinforcement learning controlled TCSC

Appropriate damping control of the inter-area oscillation would ensure the grid stability and maintain the tie-line power transfer capability. In this paper, we proposed a novel reinforcement learning (RL) based power oscillation damping (POD) controller that uses Thyristor Controlled Series Compensators (TCSC) to damp inter-area oscillations. By leveraging the unbiased gradient direction estimation of the natural evolution strategy (NES), we successfully regulated the power flows on the tie-lines and damped inter-area oscillations through dynamically modulating the inserted reactance of the TCSC. The proposed RL-based POD controller has been tested on both two-area four-machine system and North American Minni-WECC system. Extensive studies have demonstrated the excellent performance of the proposed RL-based TCSC POD controller in damping inter-area oscillations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Improving DFIG performance under fault scenarios through evolutionary reinforcement learning based control

The double-fed induction generator (DFIG) usually experiences high rotor current and DC capacitor link voltage during system fault events. In this paper, a novel data-driven approach is proposed to enhance the DFIG performance under fault scenarios. An advanced reinforcement learning (RL) algorithm called guided surrogate-gradient-based evolution strategy (GSES) is used to control the DFIG power and capacitor DClink voltage by adjusting the optional reference signals. This controller is able to prevent the rotor of DFIG from overcurrent risk and maintain the grid-connected operation. The proposed GSES-based control algorithm was evaluated through simulations on a 3.6-MW DFIG in PSCAD/EMTDC software. Results have validated the effectiveness of the proposed GSESbased control algorithm in improving the DFIG performance under fault scenarios.

42 ENGINEERING↗
DOE OSTI2019

Reinforcement learning method for plug‐in electric vehicle bidding

This study proposes a novel multi‐agent method for electric vehicle (EV) owners who will take part in the electricity market. Each EV is considered as an agent, and all the EVs have vehicle‐to‐grid capability. These agents aim to minimise the charging cost and to increase the privacy of EV owners due to omitting the aggregator role in the system. Each agent has two independent decision cores for buying and selling energy. These cores are developed based on a reinforcement learning (RL) algorithm, i.e. Q‐learning algorithm, due to its high efficiency and appropriate performance in multi‐agent methods. Based on the proposed method, agents can buy and sell energy with the cost minimisation goal, while they should always have enough energy for the trip, considering the uncertain behaviours of EV owners. Numeric simulations on an illustrative example with one agent and a testing system with 500 agents demonstrate the effectiveness of the proposed method.

Najafi, Soroush↗
DOE OSTI2020

Decentralised demand response market model based on reinforcement learning

A new decentralised demand response (DR) model relying on bi‐directional communications is developed in this study. In this model, each user is considered as an agent that submits its bids according to the consumption urgency and a set of parameters defined by a reinforcement learning algorithm called Q‐learning. The bids are sent to a local DR market, which is responsible for communicating all bids to the wholesale market and the system operator (SO), reporting to the customers after determining the local DR market clearing price. From local markets’ viewpoint, the goal is to maximise social welfare. Four DR levels are considered to evaluate the effect of different DR portions in the cost of the electricity purchase. The outcomes are compared with the ones achieved from a centralised approach (aggregation‐based model) as well as an uncontrolled method. Numerical studies prove that the proposed decentralised model remarkably drops the electricity cost compare to the uncontrolled method, being nearly as optimal as a centralised approach.

Shafie‐Khah, Miadreza↗