Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

PWR loading pattern optimization with reinforcement learning

The core loading pattern optimization problem belongs to the class of combinatorial optimization problem and has been studied since the dawn of commercial nuclear energy industry. It is characterized by multiple objectives and constraints, with a very high number of candidate patterns, which makes it impossible to solve explicitly. Stochastic optimization methodologies including Genetic Algorithms and Simulated Annealing are used by different nuclear utilities and vendors to perform fuel cycle reload design. Nevertheless, hand-designed solutions continue to be the prevalent method in the industry. To improve the state-of-the-art core reload patterns, we aim to create a method as scalable as possible, that agrees with the designer's goal of performance and safety. To help in this task Deep Reinforcement Learning (DRL), in particular Proximal Policy Optimization is leveraged. DRL has recently experienced a strong impetus from its successes applied to games, sometimes even reaching 'super-human' performances. This paper lays out the foundation of this method and proposes to study the behavior of several hyper-parameters that influence the DRL algorithm. The algorithm is highly dependent on multiple factors such as an exploration/exploitation trade-off that manifests through different parameters such as the number of loading patterns seen and the number of samples collected before a policy update, but also the shape of the objective function derived for the core design. Experimental results also demonstrate the effectiveness of the method in finding high-quality solutions from scratch within a reasonable amount of time. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Tetramerization Reinforces the Dimer Interface of MnSOD

Two yeast manganese superoxide dismutases (MnSOD), one from Saccharomyces cerevisiae mitochondria (ScMnSOD) and the other from Candida albicans cytosol (CaMnSODc), have most biochemical and biophysical properties in common, yet ScMnSOD is a tetramer and CaMnSODc is a dimer or ‘‘loose tetramer’’ in solution. Although CaMnSODc was found to crystallize as a tetramer, there is no indication from the solution properties that the functionality of CaMnSODc in vivo depends upon the formation of the tetrameric structure. To elucidate further the functional significance of MnSOD quaternary structure, wild-type and mutant forms of ScMnSOD (K182R, A183P mutant) and CaMnSODc (K184R, L185P mutant) with the substitutions at dimer interfaces were analyzed with respect to their oligomeric states and resistance to pH, heat, and denaturant. Dimeric CaMnSODc was found to be significantly more subject to thermal or denaturant-induced unfolding than tetrameric ScMnSOD. The residue substitutions at dimer interfaces caused dimeric CaMnSODc but not tetrameric ScMnSOD to dissociate into monomers. We conclude that the tetrameric assembly strongly reinforces the dimer interface, which is critical for MnSOD activity.

59 BASIC BIOLOGICAL SCIENCES↗

Reinforcement Learning as a Parsimonious Alternative to Prediction Cascades: A Case Study on Image Segmentation

Deep learning architectures have achieved state-of-the-art (SOTA) performance on computer vision tasks such as object detection and image segmentation. This may be attributed to the use of over-parameterized, monolithic deep learning architectures executed on large datasets. Although such large architectures lead to increased accuracy, this is usually accompanied by a larger increase in computation and memory requirements during inference. While this is a non-issue in traditional machine learning (ML) pipelines, the recent confluence of machine learning and fields like the Internet of Things (IoT) has rendered such large architectures infeasible for execution in low-resource settings. For some datasets, large monolithic pipelines may be overkill for simpler inputs. To address this problem, previous efforts have proposed decision cascades where inputs are passed through models of increasing complexity until the desired performance is achieved. However, we argue that cascaded prediction leads to sub-optimal throughput and increased computational cost due to wasteful intermediate computations. To address this, we propose PaSeR (Parsimonious Segmentation with Reinforcement Learning) a non-cascading, cost-aware learning pipeline as an efficient alternative to cascaded decision architectures. Through experimental evaluation on both real-world and standard datasets, we demonstrate that PaSeR achieves better accuracy while minimizing computational cost relative to cascaded models. Further, we introduce a new metric IoU/GigaFlop to evaluate the balance between cost and performance. On the real-world task of battery material phase segmentation, PaSeR yields 179% improvement over SOTA MatPhase model and a 196% improvement over IDK Cascades under the IoU/GigaFlop metric. We also demonstrate PaSeR’s adaptability to complementary models trained on a noisy MNIST dataset, where it outperforms all baselines on IoU/GigaFlop by an average of 44%.

Srikshan, Bharat↗

Codification of Fiber Reinforced Composite Piping

The goal of the overall project is to successfully adapt spoolable FRP currently used in the oil industry for use in hydrogen pipelines. The use of FRP materials for hydrogen service will rely on the demonstrated compatibility of these materials for pipeline service environments and operating conditions. The ability of the polymer piping to withstand degradation while in service, and development of the tools and data required for life management are imperative for successful implementation of these materials for hydrogen pipeline. The information and data provided in this report provides the technical basis for the codification for fiber reinforced piping (FRP) for hydrogen service. The DOE has invested in the evaluation of FRP for the delivery for gaseous hydrogen to support the development of a hydrogen infrastructure. The codification plan calls for detailed investigation of the following areas: System design and applicable codes and standards; Service degradation of FRP; Flaw tolerance and flaw detection; Integrity management plan; Leak detection and operational controls evaluation; Repair evaluation. The FRP codification process started with commercially available products that had extensive use in the oil and gas industry. These products have been evaluated to assure that sufficient structural integrity is available for a gaseous hydrogen environment.

08 HYDROGEN↗

Fracture Toughness Prediction for MWCNT Reinforced Ceramics

This report describes the development of a micromechanics model to predict fracture toughness of multiwall carbon nanotube (MWCNT) reinforced ceramic composites to guide future experimental work for this project. The modeling work described in this report includes (i) prediction of elastic properties, (ii) development of a mechanistic damage model accounting for matrix cracking to predict the composite nonlinear stress/strain response to tensile loading to failure, and (iii) application of this damage model in a modified boundary layer (MBL) analysis using ABAQUS to predict fracture toughness and crack resistance behavior (R-curves) for ceramic materials containing MWCNTs at various volume fractions.

36 MATERIALS SCIENCE↗

Mineral-Ground Micro-Fibrillated Cellulose Reinforcement for Polymer Compounds

ORNL worked with Imerys to demonstrate reinforcement of additive manufacturing feedstock materials using mineral-ground microfibrillated cellulose (MFC). Properly prepared/dried mineral-ground cellulose microfibrils significantly improved mechanical properties of both ABS and PLA resins. While tensile strength increases up to ~40% were observed, elastic modulus of the both resins doubled with the addition of 30% MFC.

36 MATERIALS SCIENCE↗

Effect of Alkali-Silica Reaction on Shear Strength of Reinforced Concrete Structural Members

Alkali-silica reaction (ASR) was discovered in the early 40s by Stanton (1940) of the California Division of Highways. Since, it has been recognized as a major degradation mechanism for concrete dams and transportation infrastructures. Sometimes described as the ’cancer of concrete’, this internal swelling mechanism causes expansion, cracking and loss of mechanical properties. There are no known economically viable solutions applicable to massive concrete to prevent the reaction once initiated. The e ciency of the mitigation strategies for ASR subjected structures is limited. Several cases of ASR in nuclear generating stations have been disclosed in Japan (Takatura et al. 2005), Canada at Gentilly 2 NPP (Tcherner and Aziz 2009) 1, and more recently, in the United States for which the U.S. Nuclear Regulatory Commission issued Information Notice (IN) 2011-20, ’Concrete Degradation by Alkali Silica Reaction,’ on November 18, 2011, to provide the industry with information related to the ASR identified at Seabrook. Considering that US commercial reactors in operation enter the age when ASR degradation can be visually detected and that numerous non nuclear infrastructures (transportation, energy production) have already experienced ASR in a large majority of the States (e.g., Department of Transportation survey reported by Touma (Touma 2000)), the susceptibility and significance of ASR for nuclear concrete structures must be addressed in the perspective of license renewal and long-term operation beyond 60 years. The aim of this report is to perform an extensive parametric series of 3D nonlinear finite element analyses of three di erent “beam-like” geometries, including two di erent depths, three di erent types of boundary conditions, and four other parameters: namely, the ASR volumetric expansion, the reinforcement ratio, the loss of elastic modulus induced by ASR and the loss of tensile strength caused by ASR.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Assembly and Delivery of Rabbit Capsules for Irradiation of Reinforced Radiation Resistant SiC-SiC Composites in the High Flux Isotope Reactor

The development of reinforced radiation resistant composites presents a critical challenge for in-vessel component application in advanced high temperature reactors. In its pursuit of this, Physical Optics Corporation (POC) is interested in developing a silicon carbide (SiC)-based ceramic matrix composite (CMC) with improved thermal conductivity in a high neutron radiation environment. The US Department of Energy (DOE) is supporting research efforts to develop Advanced Reactor Technology. Ultimately, the results of this project will determine the viability of using SiC matrix composites in hot structures in advanced high temperature reactors. The first objective of this project is to irradiate torsion and bend bar specimens in the High Flux Isotope Reactor (HFIR) and develop an initial data set to evaluate the evolution of composite mechanical strength, dimensional changes, and thermal conductivity. This report briefly describes the irradiation experiment design concepts, summarizes the irradiation test matrix, and reports on the successful delivery of three rabbit capsules to the HFIR. Rabbits of both torsion and bend bar configurations have been assembled, welded, evaluated, and delivered to the HFIR along with a complete quality assurance fabrication package.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Low-Cost Scaffold-Reinforced Perovskite Solar Modules with Integrated Light Management

Perovskite solar cells are a promising new solar technology because they are highly efficient and cheaper to manufacture compared to silicon solar cells. However, perovskites suffer from mechanical and thermal fragility, important factors limiting device lifetimes, which leads to problems in making reliable solar cells. In our recent work, we showed how to make perovskite solar cells that look like the eye of a fly by subdividing a solar cell into thousands of microcells that are surrounded by a honeycomb-shaped reinforcing scaffold. This discovery was a new architecture called a compound solar cell (CSC). The perovskite is effectively shielded from mechanical forces by the scaffold and leads to robust solar cells that are no longer fragile. This was a promising advancement, but the drawback of the scaffold structure is that the scaffold absorbs light from the solar cell and leads to a reduced efficiency. In this project, we address this reduced efficiency by integrating concentrating lens arrays into CSCs to focus any incident light away from the scaffolds. Perovskite is deposited into the microcells between these scaffolds, and upon sunlight exposure, the lenses concentrate light into the microcells and away from the insulating scaffolds, resulting in significantly improved efficiencies. The lenses redirect over 90% of the light that otherwise would have been incident on the scaffolds into the microcells leading to a corresponding 90% recovery of efficiency. Furthermore, the lenses show passive light tracking capabilities, meaning as the angle of incident light changes (simulating how the sunlight moves across the sky during the course of the day), a majority of the light is still focused into the perovskite microcells. The improved efficiency and passive tracking capabilities suggest that lens-integrated CSCs are a potential pathway to high-performance, robust, commercially viable perovskite solar cells.

14 SOLAR ENERGY↗

Reinforcement Learning-based Traffic Control to Optimize Energy Usage and Throughput (CRADA report)

The US roadways are critical to meeting the mobility and economic needs of the nation. The United States uses 28% of its energy in moving goods and people, with approximately 60% of that utilized by cars, light trucks, and motorcycles. Thus, improved transportation efficiency is vital to America’s economic progress. The increasing congestion and energy resource requirements of transportation systems for metropolitan areas require research in methods to improve and optimize control methods. Coordinating and optimizing traffic in urban areas may introduce hundreds of thousands of vehicles and traffic management systems, which can require high performance computing (HPC) resources to model and manage. In this work, we seek to use machine learning, computer vision, and HPC to improve the energy efficiency aspects of traffic control by leveraging GRIDSMART traffic cameras as sensors for adaptive traffic control, with a sensitivity to the fuel consumption characteristics of the traffic in the camera’s visual field. Traffic control use cases using reinforcement learning have been published and achieved good results. Surveys from DOE national laboratories estimate that the fuel cost of idling is 6 billion gallons wasted annually [Argonne National Laboratory, 2019]. GRIDSMART cameras—an existing, fielded commercial product—sense the presence of vehicles at intersections and replace more conventional sensors (such as inductive loops) to issue calls to traffic control. These cameras, which have horizon-to-horizon view, offer the potential for an improved view of the traffic environment which can be used to generate better control algorithms.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Carbon Fiber Reinforced Polyolefin Body Panels

This project addresses an automotive industry desire for a cost-effective, lightweight material for body panels that offers mass reduction compared to steel and aluminum body panels, and a cost savings compared to incumbent polymer body panels. Inexpensive polypropylene (PP) combined with <10 weight % reclaimed/recycled carbon fiber (rCF) have the potential to meet both performance and affordability targets for lightweight, paintable vertical body panels. This project supports DOE and IACMI technical goals of reducing production cost of CF composites >25% in 5 years on a path to >50% in 10 years and demonstrating technologies for >80% recyclability or reuse of fiber reinforced composites in 5 years on a path to >95% in 10 years. Alternative body panel materials must meet all fit and surface finish requirements, and match the appearance of metal body panels adjacent to them. The PP/rCF materials must maintain exterior dimensional requirements with a low co-efficient of thermal expansion, excellent paintability, and mechanical performance. Thermoplastic injection molding offers significant reduction in tooling costs as injection molding dies are typically less than half the cost of stamping dies. This project utilized rCF compounded with special formulations of PPs to produce automotive components that meet requirements for fit, finish, mechanical performance, and paintability while offering significant mass savings over incumbent metal panels. It will also offer a cost savings versus other composite solutions. Fenders were molded at the IACMI – Corktown facility using an existing fender tool. Ford Motor Company was the industry lead for this project and provided project management, testing, painting and adhesion testing, and developed the cost-model/business case. Ford engaged and aggregated cost share with Borealis who provided materials, technical support for process trials, and participated in coordination meetings. Vehicles Technology Area staff in Corktown assisted in project management, process trials, molded fenders, and benchmarked physical characteristics. Over 450 plaques were molded with two different gating systems to look at processing conditions and specimens were excised for mechanical testing. Plaques were also used for paint trials, surface analysis, and dimensional analysis. Subsequently, over 200 fenders were molded at IACMI – Corktown. These fenders were used for paint trials, surface analysis, and dimensional analysis. The PP/rCF systems met all of the requirements except for paint appearance. Additional development of paint primer and top coat is necessary to fulfill automotive OEM paint surface requirements.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Polymer Composites Reinforced by Carbon-Allotrope Fillers for Selective Laser Sintering (SLS) - A Review

The limited commercial availability of applicable polymer powders has hindered the widespread implementation of SLS as a legitimate manufacturing method. This is partly due to the mechanism of SLS, which requires powders to meet rigid specifications if processing is to be successful. Furthermore, powders that do meet requirements are often incapable of producing quality, high-performance parts. Academic and industrial research has focused on composite development to overcome this problem. The incorporation of fillers including calcium carbonate, molybdenum disulfide, and aluminum have all been found to influence the processing behavior of PA12 powder and ultimately, the properties of the final part.3-5 Here, the thermal and mechanical properties of sintered composite parts produced using polymer powders reinforced by various carbon-based fillers is discussed.

36 MATERIALS SCIENCE↗

Using Reinforcement Learning to Optimize Quantum Circuits in the Presence of Noise

As we move towards devices which utilize more qubits, it becomes increasing more important to map quantum circuits in a way that uses resources efficiently as well as maximizes the reliability of the results of that circuit. To this end, we will need to rely on heuristic algorithms, specifically reinforcement learning (RL) as a method of building quantum circuits based on observations of the noise characteristics in its environment.

Guy, Khalil↗

Autonomous reinforcement learning agents for improving predictions and observations of extreme climate events

Primary Focus Area: This proposal addresses focus area 2, “Predictive modeling through the use of AI techniques.” Science Challenge: Extreme climate events associated with severe weather, coastal and inland flooding, droughts, heat waves and wildfires are expected to increase in frequency and severity in the future. Due to the complexity and chaotic behavior of the climate system, accurately predicting and observing extreme climate events requires a tremendous amount of human intervention to run predictive climate simulations and deploy measurement systems. Extreme events often unfold very quickly, leaving little time to iterate on simulations or re-position instruments. Through reinforcement learning, autonomous AI agents can be designed to make real-time decisions to characterize extreme climate events more efficiently through adaptive models and targeted observations.

54 ENVIRONMENTAL SCIENCES↗

Optimizing and Extending the Functionality of EXARL for Scalable Reinforcement Learning [Slides]

The main goal of the Co-Design Summer School 2021 is to provide algorithmic improvements to EXARL framework by improving performance and by adding functionalities. This presentation includes an introduction to reinforcement learning and to EXARL. The researchers expanded the capability of EXARL by including additional agents like (Asynchronized) Advantage Actor Critic (A2C/A3C) and Twin Delayed Deep Deterministic Policy Gradient (TD3). They also explored algorithmic improvements such as v-trace and Prioritized Experience Replay. They found that A2C/A3C performed best with v-trace and outperformed Deep Q-Network (DQN) on both the CartPole game and the ExaBooster scientific environment. Additionally, they found that TD3 performed as good as the existing Deep Deterministic Policy Gradient (DDPG) agent and that adding Prioritized Experience Replay to DDPG accelerated convergence.

97 MATHEMATICS AND COMPUTING↗

Lattice Reinforced Cermet Materials

This project developed ceramic-metal (cermet) composite materials reinforced with embedded metal lattices. The goal of the project was to develop materials that have a higher tensile and compressive strength as well as greater fracture toughness than conventional boron carbide-aluminum cermets. These materials have applications in structural applications as well as advanced vehicle and personnel armors.

36 MATERIALS SCIENCE↗

Integrated TBC/EBC for SiC Fiber Reinforced SiC Matrix Composites for Next Generation Gas Turbines (Final Report)

In this ambitious project, a significant challenge preventing widespread use of ceramic matrix composites in hot section components of gas power generating turbines was addressed. Hot section components made from SiC fiber reinforced SiC matrix composites (SiC/SiC composites), have the potential to achieve a combined cycle energy efficiency of above 65%. To achieve this efficiency, the temperature of the combustion gases on the turbine component surface is expected to reach as high as ~1700ºC (3100ºF). However, SiC/SiC composites suffer from volatilization at high temperature under high velocity steam environment – typical in a turbine hot chamber. This integrated collaborative project, between two research groups at Clemson University and a team of engineers and scientists at GE Gas Power, focused on design, fabrication and evaluation of a new graded environmental barrier coating. Using the excellent compositional tailorability of polymer derived ceramics (PDCs), we developed an integrated EBC/BC that smoothly transitions from bottom SiC/SiC bond interface to hermetic sesquioxide (Y 2 O 3 ) top surface. This smooth compositional transition avoided the sharp CTE mismatch at interfaces to greatly reduce the thermal stress. The developed EBC is compatible with and bonds well with the bond coat that GE uses. The top hermetic oxide surface ensures low oxygen transportation and volatilization rates under the high velocity steam environment. The graded microstructure with increasing sesquioxide concentration also decreases the amount of Si and C elements near the top hermetic sesquioxide layer. Coatings made using a process suitable for manufacturing – atmospheric plasma spraying was demonstrated. The microstructure of coatings made using this process were fully characterized. Finally, the performance of these coatings was investigated under the most demanding conditions for this application. Specifically, high temperature, high velocity steam exposure; thermal shock test; and thermal cycling tests. The graded coatings performed exceptionally well in all these tests. Specific suggestions for future development in this area have also been identified. In addition to the excellent technical progress, this project helped initiate a robust collaboration between the research groups at Clemson and GE Gas Power. It also provided a high-quality learning and mentoring opportunity for the three post-doctoral research associates and two UG students at Clemson University who worked on this project. All proposed tasks, sub-tasks were completed, and all milestones achieved.

03 NATURAL GAS↗