Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,693 records · Page 94

Reinforcement Learning for Generating Toolpaths in Additive Manufacturing

Generating toolpaths plays a key role in additive manufacturing processes. In the case of 3-Dimensional (3D) printing, these toolpaths are the pathways the printhead will follow to fabricate a part in a layer-by-layer fashion. Most toolpath generators use nearest neighbor (NN), branch-and-bound, or linear programming algorithms to produce valid toolpaths. These algorithms often produce sub-optimal results or cannot handle large sets of traveling points. In this paper, the researchers at Oak Ridge National Laboratory’s (ORNL) Manufacturing Demonstration Facility (MDF) propose using a machine learning (ML) approach called reinforcement learning (RL) to produce toolpaths for a print. RL is the process of two agents, the player and the critic, learning how to maximize a score based upon the actions of the player in a defined state space. In the context of 3D printing, the player will learn how to find the optimal toolpath that reduces printhead lifts and global print time.

Patrick, Steven↗

Boron carbide fiber reinforced articles

Methods of producing continuous (or discontinuous) boron carbide fibers. The method comprises reacting a continuous or discontinuous carbon fiber material and a boron oxide gas within a temperature range of from approximately 1400.degree. C. to approximately 2200.degree. C. Articles including such partially or fully converted fibers may be provided, including such reinforcing fibers in a matrix of ceramic (a CMC), in metal (a MMC), or other matrix (e.g., polymer, etc.).

36 MATERIALS SCIENCE↗

Rigid stiffener-reinforced flexible neural probes, and methods of fabrication using wicking channel-distributed adhesives and tissue insertion and extraction

A stiffener-reinforced microelectrode array probe and fabrication method using wicking channel-distributed adhesives which temporarily adheres a flexible device onto a rigid stiffener for insertion and extraction. Assembly is by dispensing a liquid adhesive into a narrow open groove wicking channel formed on the stiffener so that the adhesive is wicked along and tills the channel by capillary action, and adhering the adhesive-filled bonding side of the elongated section of the rigid substrate to a flexible device.

Shah, Kedar G.↗

Sky Survey Scheduling Using Reinforcement Learning

Modern cosmic sky surveys (e.g., CMB S4, DES, LSST) collect a complex diversity of astronomical objects. Each of class objects presents different requirements for observation time and sensitivity. For determining the best sequence of exposures for mapping the sky systematically, conventional scheduling methods do not optimize the use of survey time and resources. Dynamic Sky survey scheduling is an np-hard problem that has been therefore treated primarily with heuristic methods. We present an alternative scheduling method based on reinforcement learning (RL) that aims to optimize the use of telescope resources for scheduling sky surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Reinforced electrolyte membrane

An electrolyte membrane having a proton conducting polymer reinforced with a nanofiber mat made from a nanofiber comprising a fiber material selected from polymers and polymer blends; wherein the fiber material has a fiber material proton conductivity; wherein the proton conducting polymer has a proton conducting polymer conductivity; and wherein the fiber material proton conductivity is less than the proton conducting polymer conductivity, and methods of making. In some embodiments, the nanofiber further comprises a proton conducting polymer.

Yandrasits, Michael Andrew↗

Modeling Cell Migration with Convolutional Neural Network and Deep Reinforcement Learning

Cell migration modeling is a longstanding biological challenge, which is regulated by a highly complex set of regulatory mechanisms at multiple scales in a developmental system. This study presents a generic framework for regulatory mechanisms discovery during cell migration. This framework uses convolutional neural networks and reinforcement learning to better study navigation rules and mechanisms during cell migration. This framework adopts a flexible model-free approach that directly takes raw images as the sensory input. It can better handle simulation scenarios that involve cell division during embryogenesis. Computational experiments also prove that this model achieves better performance than a previous model with a fully connected neural network.

Wang, Zi↗

MICRO-CELLULOSE FIBER REINFORCED BIOCOMPOSITES FOR ADDITIVE MANUFACTURING

Polymer additive manufacturing (AM) technology is rapidly growing and it is transitioning from being primarily a prototyping method to an advanced manufacturing technique. Melt extrusion/fused deposition modeling, a method in which molten thermoplastic-based feedstock material is deposited in a raster pattern layer by layer, is the most commonly used technique in polymer AM. The technique enables direct digital manufacturing of desired geometry with controlled anisotropic distribution of reinforcing phase. While thermoplastic composites are being used in many areas of industry including AM, due to increasing environmental and long-term sustainability concerns, there is an increasing interest in bio-based renewable alternatives. Nano-to-mm scale plant-based fibers are utilized to improve properties of thermoplastics and produce bio-composites. The effect of commercially available micro-scale cellulose fibers (MFC) on the mechanical and rheological behavior of polylactic acid to produce fully bio-based composites are being investigated and selected formulations will be 3D-printed to investigate the effect of printing process on MFC alignment. Significant increases in both tensile strength and elastic modulus of compression-molded MFC-PLA composites were observed at 40 % (by wt) fiber levels.

Tekinalp, Halil↗

Distributed Reinforcement Learning with ADMM-RL

NREL researcher Peter Graf discusses distributed reinforcement learning with ADMM-RL as part of the Innovative Optimization and Control Methods for Highly Distributed Autonomous Systems workshop. NREL hosted the workshop to exchange ideas on state-of-the-art control and optimization strategies as well as get feedback on work being done at NREL.

97 MATHEMATICS AND COMPUTING↗

RL-HEMS: Reinforcement Learning Based Home Energy Management System for HVAC Energy Optimization

Heating, ventilation, and air conditioning (HVAC) is one of the major energy consumers in the residential sector. It is important to be able to monitor and control the energy consumed to provide utility services such as load shaping while satisfying the comfort and economic constraints of the homeowner. The objective of this work is to create the optimal schedule for HVAC operation to reduce the cost while satisfying the home-owner and equipment’s constraints using a model-free Reinforcement Learning (RL)-based optimization. The specific goal is to find the right balance between reducing energy cost, consumption, and customer comfort level. Our research effort addresses this optimization problem using multiple components: the development of initial learning test-bed and implementation of RL techniques on a real home. This will enable the rapid evaluation of the RL techniques and provide an early baseline to train before implementation on site. The RL algorithm is designed to learn the energy use patterns and generate the optimized schedule for HVAC within an acceptable time-interval to satisfy the homeowner’s comfort and minimize the energy usage. Our preliminary results are promising and we have observed a 17% reduction in the total cost and a 15% reduction in the power utilization using our RL-based HVAC model—RL-HEMS.

Kotevska, Olivera↗

Flexible Reinforcement Learning Framework for Building Control using EnergyPlus-Modelica Energy Models

In recent years, reinforcement learning (RL) methods have been greatly enhanced by leveraging deep learning approaches. RL methods applied to building control have shown potential in many applications due to their ability to complement or replace conventional methods such as model-based or rule-based controls. However, RL-based building control software is likely tailored either to one target building system or to a specific RL method so that significant additional effort would be required to customize the RL-based controller for use in other building systems or with other RL approaches. Also, RL-based building controls usually depend on building energy simulations to train controllers, so emulating building dynamics (i.e., thermal dynamics and control dynamics) and capturing sub-hourly dynamic profiles are crucial to further the development of effective RL-based building control methods. To address these challenges, we present an open source RL-based control software employing a high-fidelity hybrid EnergyPlus-Modelica building energy model which emulates building dynamics at 1-minute resolution. This software consists of decoupled components (environment, building emulator, control agent, and RL algorithm), which allows for quick prototyping and benchmarking of standard RL algorithms in different systems; for example, a single component can be replaced without revising all of the software. To demonstrate this software framework, we conducted a benchmark study using an EnergyPlus-Modelica building energy model for a Chicago office building with an RL-based controller to dynamically control the chilled water temperature setpoint and the air handling unit supply air temperature setpoint on selected floors.

Lee, Joon-Yong↗

Automated Adversary Emulation for Cyber-Physical Systems via Reinforcement Learning

Adversary emulation is an offensive exercise that provides a comprehensive assessment of a system’s resilience against cyber attacks. However, adversary emulation is typically a manual process, making it costly and hard to deploy in cyber-physical systems (CPS) with complex dynamics, vulnerabilities, and operational uncertainties. In this paper, we develop an automated, domain-aware approach to adversary emulation for CPS. We formulate a Markov Decision Process (MDP) model to determine an optimal attack sequence over a hybrid attack graph with cyber (discrete) and physical (continuous) components and related physical dynamics. We apply model-based and model-free reinforcement learning (RL) methods to solve the discrete-continuous MDP in a tractable fashion. As a baseline, we also develop a greedy attack algorithm and compare it with the RL procedures. We summarize our findings through a numerical study on sensor deception attacks in buildings to compare the performance and solution quality of the proposed algorithms.

Bhattacharya, Arnab↗

ReLVis: Visual Analytics for Situational Awareness During Reinforcement Learning Experimentation

Reinforcement learning (RL) is a branch of machine learning where an agent learns to maximize reward through trial and error. RL is challenging and data/compute intensive, and RL practitioners can easily become overwhelmed and make poor modeling decisions. Our contribution is a visual analytics tool designed to help data scientists maintain situation awareness during RL experimentation. Our tool allows the user to under- stand which hyper-parameter values lead to better or worse outcomes, what behaviors are associated with high and low reward, and how behaviors evolve and progress over training time. We evaluated our tool by applying state of the art deep RL models to six different RL tasks. We present three tasks as use cases that show how our tool leads to insightful findings and improves situation awareness.

Saldanha, Emily G.↗

Single Episode Policy Transfer in Reinforcement Learning

Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing near-optimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience rollouts for adaptation. To achieve single episode transfer in a family of environments with related dynamics, we propose a general algorithm that optimizes a probe and an inference model to rapidly estimate underlying latent variables of test dynamics, which are then immediately used as input to a universal control policy. This modular approach enables integration of state-of-the-art algorithms for variational inference or RL. Moreover, our approach does not require access to rewards at test time, allowing it to perform in settings where existing adaptive approaches cannot. In diverse experimental domains with a single episode test constraint, we report our method significantly outperforms existing adaptive approaches and shows favorable performance against baselines for robust transfer.

42 ENGINEERING↗

The Effect of Antagonistic Behavior in Reinforcement Learning

The significant achievements of deep reinforcement learning (RL) have motivated researchers to also investigate its shortcomings. Such work has shown that typical methods in deep RL tend to produce brittle policies that overfit to the training environment. In this paper, we introduce the notion of purely antagonistic behavior in value-based agents, where the objective is not to maximize reward but to minimize the victim’s value over time. This notion is motivated by the scenario in which an antagonistic human architect, without access to the environment’s reward function, wants to build an RL agent that can impede another well-trained RL victim agent. First, we formalize a notion of antagonistic behavior in RL. Then, we provide experiments that show how a purely antagonistic agent performs compared to a well-trained victim that learns directly from the game’s rewards. Our results suggest that if one’s goal is to find vulnerabilities in well-trained agents, direct access to the environment’s rewards is not necessary, and antagonistic behavior can be measured independently from environment wins and losses.

Fujimoto, Ted C.↗

Resin transfer molding systems and control logic for manufacturing fiber-reinforced composite parts

Presented are manufacturing control systems for fabricating composite-material structures, methods for making/operating such systems, and resin transfer molding techniques for ameliorating race-tracking effects in fiber-reinforced polymer panels. A method for forming a composite-material construction includes confirming, via a system electronic control unit (ECU), that a fiber-based preform is placed in a mold cavity and that opposing mold segments of the molding apparatus are sealed together. A filler, such as a compressible bladder, a cluster of spring-biased pins, or a spray-chopped fiber bed, is introduced into a void between the fiber-based preform and a tool face of one mold segment to thereby eliminate an unwanted resin race track. The system ECU commands a resin pump to inject resin through a primary gate of the molding apparatus and into the mold cavity to thereby impregnate the fiber-based preform with the resin. One or more vents operate to evacuate air from the mold.

Rodgers, William R.↗

THE DESIGN OF LAYER TIME OPTIMIZATION IN LARGE SCALE ADDITIVE MANUFACTURING WITH FIBER REINFORCED POLYMER COMPOSITES

In this study, we have developed a method to optimize a layer deposition time (a.k.a. layer time) for large-scale additive manufacturing (AM) via physics-based simulations. A long layer time leads to an over-cooled surface on which a new layer is deposited, and therefore, it may result in a weak bonding or debonding between layers, cracking, or warping. A short layer time leads to a high temperature of the structure due to insufficient cooling, and therefore, the structure may not be stiff enough and may collapse during manufacturing. Therefore, it is important to estimate the optimal layer time in additive manufacturing for a high-quality product. The temperature of a top layer right before deposition is recommended to be slightly higher than the glass temperature of the material. A temperature cooling was approximated to an exponential function of time, and the optimized layer time was obtained based on a target temperature while maintaining a minimal printing time. The material used is carbon fiber-reinforced polycarbonate (CF/PC), and the large-scale deposition system used is LSAM TM from Thermwood Corporation. Three different layer time cases were used for experiments, and a series of thermal images were obtained via an infra-red (IR) camera during the entire AM processes. AM process simulations were performed using a finite element method and the temperature profiles from the simulation were in good agreement with those from experiments. The layer time optimization was performed based on the temperature profiles from the simulations. A layer temperature with the optimal layer time was confirmed as the target temperature through simulation.

Jo, Eonyeon↗