Results for “REINFORCEMENT” — Page 22

NASA NTRSDate not supplied

Adaptive Stress Testing of Collision Avoidance Systems for Small UASs with Deep Reinforcement Learning

The next-generation Airborne Collision Avoidance System for smaller UASs (ACAS sXu) is currently being developed and tested by the Federal Aviation Administration (FAA) to provide detect-and-avoid capability for small unmanned aircraft operating beyond line-of-sight. Due to the complexity and safety-critical nature of the system, safety validation is important not only for the certification of the final system, but also for informing changes during the iterative development process. In this paper, we analyze a prototype of ACAS sXu in simulated aircraft encounters to discover scenarios of small near mid-air collisions (sNMACs), an important safety event in which two aircraft come closer than 50 feet horizontally and 15 feet vertically. Due to the size and complexity of the system as well as rarity of sNMAC events, traditional methods such as Monte Carlo testing often require informed setup and targeting to elicit failures. However, such a dependence on domain knowledge can be incompatible with the independent verification and validation (IV&V) process, the aim of which is to discover unforeseen issues. To address these challenges, we apply an accelerated validation method called adaptive stress testing (AST) to find the most likely sNMAC scenarios without reliance on system introspection. AST uses reinforcement learning to adapt the search towards the most promising areas of the search space as it progresses. We use a state-of-the-art deep reinforcement learning algorithm, proximate policy optimization, to more efficiently search the large and continuous state space. We find that this approach significantly improves the performance of AST compared to a prior approach based on Monte Carlo tree search. We perform experiments using AST to find sNMAC events under various encounter configurations, varying parameters pertaining to dynamics and coordination. Our experiments show AST to be very effective at finding sNMAC scenarios. We summarize our findings, presenting high-level categories of discovered sNMACs and specific examples of encounters in each category.

aircraft collision avoidance↗

NASA NTRSDate not supplied

Collision Avoidance Approach Using Deep Reinforcement Learning

A method to enable autonomous robots moving in a 2D space collision free motivates the purposed approach for collision avoidance for autonomous UAM vehicles. Challenges of autonomous collision free navigation for both problems are similar. Agents in each environment do not know the intent, or goal, of the other. Finding the time efficient paths require some level of anticipation with neighboring agents which is computationally expensive. In the original work, these obstacles were overcome with a novel application of deep reinforcement learning which offloads the online computation to an offline learning algorithm. A value network that encodes the estimated time to the goal given the agent’s state and the observable portion of the other agent’s state is trained on a baseline policy and further refined with reinforcement learning to promote time efficient collision free navigation. Online, the value network efficiently informs the agent’s decision making in the face of uncertainty of the other agent’s next move. In this paper, challenges extending this methodology to the 3D environment of autonomous UAM vehicles with kinematic constraints are discussed and initial results shown.

Collision Avoidance↗

NASA NTRSDate not supplied

Collision Avoidance Approach Using Deep Reinforcement Learning

A method to enable autonomous robots moving in a 2D space collision free motivates the purposed approach for collision avoidance for autonomous UAM vehicles. Challenges of autonomous collision free navigation for both problems are similar. Agents in each environment do not know the intent, or goal, of the other. Finding the time efficient paths require some level of anticipation with neighboring agents which is computationally expensive. In the original work, these obstacles were overcome with a novel application of deep reinforcement learning which offloads the online computation to an offline learning algorithm. A value network that encodes the estimated time to the goal given the agent’s state and the observable portion of the other agent’s state is trained on a baseline policy and further refined with reinforcement learning to promote time efficient collision free navigation. Online, the value network efficiently informs the agent’s decision making in the face of uncertainty of the other agent’s next move. In this paper, challenges extending this methodology to the 3D environment of autonomous UAM vehicles with kinematic constraints are discussed and initial results shown.

Collision Avoidance↗

NASA NTRSDate not supplied

On the Representation of Through-The-Thickness Reinforcements in Finite Element Analysis of Stitched, Blade Stiffened Panels

Modern aircraft employ laminated composites for their tailorable in-plane properties, high specific strengths, and high specific stiffnesses. However, laminated composites exhibit relatively poor interlaminar properties without through-the-thickness reinforcements. Quantifying the necessary amount of through-the-thickness reinforcements is necessary to reduce cost and meet damage tolerance certification requirements. In this study, a discrete superposed cohesive element (DSCE) approach is applied to represent the mixed-mode delamination behavior of stitched stiffened panels subjected to seven-point bending. This approach is compared to a one-dimensional embedded spring element (ESE) method. The DSCE approach uses two superposed bilinear traction-separation laws to obtain a representative load-displacement response determined from interlaminar tensile and shear tests. Additionally, several stitch configurations (unstitched, stitched, and overstitched) are evaluated in terms of their load-displacement response and crack-arrestment capability. Results indicate that the DSCE and ESE approaches show good agreement with respect to the predicted load-displacement response, but the ESE method tends to overpredict the crack growth behavior by approximately 13%. Stitches were not observed to fail during skin-stringer separation. Using an overstitched laminate with stitches near the flange edge provides the greatest crack-arrestment capability. Furthermore, the skin retains 92% of its stiffness after skin-stringer separation occurs.

composites↗

NASA NTRSDate not supplied

On the Representation of Through-The-Thickness Reinforcements in Finite Element Analysis of Stitched, Blade Stiffened Panels

Modern aircraft employ laminated composites for their tailorable in-plane properties, high specific strengths, and high specific stiffnesses. However, laminated composites exhibit relatively poor interlaminar properties without through-the-thickness reinforcements. Quantifying the necessary amount of through-the-thickness reinforcements is necessary to reduce cost and meet damage tolerance certification requirements. In this study, a discrete superposed cohesive element (DSCE) approach is applied to represent the mixed-mode delamination behavior of stitched stiffened panels subjected to seven-point bending. This approach is compared to a one-dimensional embedded spring element (ESE) method. The DSCE approach uses two superposed bilinear traction-separation laws to obtain a representative load-displacement response determined from interlaminar tensile and shear tests. Additionally, several stitch configurations (unstitched, stitched, and overstitched) are evaluated in terms of their load-displacement response and crack-arrestment capability. Results indicate that the DSCE and ESE approaches show good agreement with respect to the predicted load-displacement response, but the ESE method tends to overpredict the crack growth behavior by approximately 13%. Stitches were not observed to fail during skin-stringer separation. Using an overstitched laminate with stitches near the flange edge provides the greatest crack-arrestment capability. Furthermore, the skin retains 92% of its stiffness after skin-stringer separation occurs.

composites↗

NASA NTRS2024

Investigation of Carbon-Reinforced Acrylonitrile Butadiene Styrene 3D-Printed Honeycomb Composites

The expansive utility of polymeric 3D printing technologies and demand for high-performance lightweight structures has prompted the emergence of various carbon-reinforced polymer composite filaments. However, detailed characterization of the processing-microstructure-property relationships of these materials is still required to realize their full potential. In this study, acrylonitrile butadiene styrene (ABS) and carbon reinforced ABS variants, both carbon nanotubes (CNT) and 5 wt.% chopped carbon fiber (CF), are fabricated in a honeycomb geometry and investigated across a range of layer thicknesses and hex sizes. Microscopy of material cross-sections is conducted to evaluate the relationship between print parameters and porosity. Additionally, mechanical properties are evaluated through compression testing, demonstrating the potential of honeycomb ABS, ABS-CNT, and ABS-5wt.% CF polymer composites for novel 3D printed structures.

ABS (Acrylonitrile Butadiene Styrene)↗

NASA NTRS2023

Reinforcement Learning Approach to Flight Control Allocation with Distributed Electric Propulsion

The flight control system of the SUSAN Electrofan concept aircraft achieves attitude control using both conventional flight control surfaces and differential thrust through distributed electric propulsion (DEP) from sixteen wing-mounted electric engines. The introduction of eight pairs of wing fans for attitude control creates a highly actuated system. Such a system requires more sophisticated control to operate, especially in the presence of wingfan failures where the loss of a single wingfan can result in a thrust imbalance. This paper investigates the use of deep reinforcement learning (RL) using proximal policy optimization (PPO) to achieve attitude control through a combination of DEP and control surface deflections. First, the paper examines the aircraft undergoing a coordinated turn. Then, it examines the aircraft experiencing a wingfan failure during cruise conditions. It is shown that deep reinforcement learning can be a potential avenue for nonlinear flight control design.

Distributed Electric Propulsion↗

NASA NTRSDate not supplied

Investigation of Carbon-Reinforced Acrylonitrile Butadiene Styrene 3D-Printed Honeycomb Composites

The expansive utility of polymeric 3D printing technologies and demand for high-performance lightweight structures has prompted the emergence of various carbon-reinforced polymer composite filaments. However, detailed characterization of the processing-microstructure-property relationships of these materials is still required to realize their full potential. In this study, acrylonitrile butadiene styrene (ABS) and carbon reinforced ABS variants, both carbon nanotubes (CNT) and 5 wt.% chopped carbon fiber (CF), are fabricated in a honeycomb geometry and investigated across a range of layer thicknesses and hex sizes. Microscopy of material cross-sections is conducted to evaluate the relationship between print parameters and porosity. Additionally, mechanical properties are evaluated through compression testing, demonstrating the potential of honeycomb ABS, ABS-CNT, and ABS-5wt.% CF polymer composites for novel 3D printed structures.

ABS (Acrylonitrile Butadiene Styrene)↗

NASA NTRSDate not supplied

Energy-Optimized Path Planning for Uas in Varying Winds Via Reinforcement Learning

In this paper we propose a reinforcement learning (RL) algorithm for path planning of Unmanned Aviation Vehicles (UAVs) under varying wind conditions. Solutions to UAV path planning problems are becoming increasingly necessary as autonomous UAVs continue to enter commercial and government spaces. Path-planning is inherently challenging, as UAVs need to account for dynamically changing flying conditions such as weather, obstacle or no-fly zones, degraded vehicle health, and off-nominal battery power consumption. Machine learning methods such as reinforcement learning (RL) have the potential to revolutionize how vehicles navigate in such uncertain environments. In this study, we compute UAV trajectories from a pre-determined starting position to a target cell within a 7X7 grid environment by optimizing parameters for mission assurance and safety limits in addition to the energy consumption and operation time. The UAV navigates the grid by taking actions to move in any of the eight cardinal and inter-cardinal directions, under constant thrust profile. The resultant UAV state is sampled from a probability distribution which accounts for the UAV’s action, local wind velocity, and the presence of obstacles or boundaries. As the unmanned airspace gets more complex due to multiple vehicles and environmental uncertainties, trade-offs between energy consumption, operation time, risk tolerance, and mission assurance need to be made. Our Markov Decision Process (MDP) environment model can capture any combination of these in the optimization objective, making it novel compared to other work in the field.

trajectory planning↗

DOE OSTI2019

Quasi-Stochastic Approximation and Off-Policy Reinforcement Learning: Preprint

The Robbins-Monro stochastic approximation algorithm is a foundation of many algorithmic frameworks for reinforcement learning and often an efficient approach to solving (or approximating the solution to) complex optimal control problems. In many cases, however, practitioners are unable to apply these techniques because of an inherent high variance. This paper aims to provide a general foundation for 'quasi-stochastic approximation,' in which all of the processes under consideration are deterministic, much like quasi-Monte Carlo for variance reduction in simulation. The variance reduction can be substantial, subject to tuning of pertinent parameters in the algorithm. This paper introduces a new coupling argument to establish the optimal rate of convergence provided the gain is sufficiently large. These results are established for linear models and tested also in nonideal settings. A major application of these general results is a new class of reinforcement learning algorithms for deterministic state space models. In this setting, the main contribution is a class of algorithms for approximating the value function for a given policy, using a different policy designed to introduce exploration.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2018

Multiple-objective Reinforcement Learning for Inverse Design and Identification

Inverse chemical design using generative deep learning (DL) models is an emerging tool, which has early success in generating molecular structures with desired properties. We enhance this tool by developing heuristics for curriculum-learning based multiple-objective (20+ objectives) reinforcement learning, and apply it to the context of chemical identi?cation. We develop a generative DL framework that utilizes constraints pertaining to the unknown molecule’s mass and functional group composition, and show that multiple-objective RL-based generative DL models can correctly identify unknown molecules with a 80% success rate, compared to the baseline approach of 0%. Lastly, the heuristics developed are not limited to just chemistry research challenges; we anticipate any problem that utilizes reinforcement learning with multiple-objectives will bene?t from it.

Artificial Intelligence, Data Science, deep learni↗

DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

actuated signal control↗

DOE OSTI2025

Lightweight, Flexible Electromagnetic Shielding Composite Films Reinforced with Recycled Carbon Fibers and Carbon Nanofillers

Lightweight, flexible polymer composites are widely adopted in modern electronic devices and systems for high-efficiency electromagnetic interference (EMI) shielding. Reinforcing polymer composites with conductive fillers offers a promising alternative to conventional metal-based shielding material thanks to their low density, tunable properties, and flexibility. Herein, lightweight, flexible ultra-high molecular weight polyethylene composite films reinforced with recycled carbon fibers (rCFs) and carbon-based nanofillers, including graphene nanoplatelets (GNPs) and carbon nanotubes, are reported for EMI shielding applications. The incorporation of these nanofillers significantly reduces the required content of rCFs and improves processability while maintaining comparable shielding effectiveness. The addition of these nanofillers with rCFs enhances the EMI shielding effectiveness by up to 15 dB. Incorporating 1 wt% GNPs can replace 5 wt% rCFs and achieve comparable EMI shielding performance, while 5 wt% of either nanofillers can substitute for 10 wt% rCFs. Numerical modeling of electromagnetic wave transmission reveals that increasing nanofiller concentrations enhances both reflection and absorption losses, with absorption consistently dominating across all levels. Furthermore, this study not only provides insight into the synergistic contributions of rCFs and carbon nanofillers to shielding effectiveness but also paves the way for the design of sustainable, lightweight EMI shielding composite films for applications in electric vehicles, medical equipment, and portable electronics.

carbon nanofiller↗

DOE OSTI2020

3D Ion-Conducting, Scalable, and Mechanically Reinforced Ceramic Film for High Voltage Solid-State Batteries

Concerning the safety aspects of Li+ ion batteries, an epoxy-reinforced thin ceramic film (ERTCF) is prepared by firing and sintering a slurry-casted composite powder film. The ERTCF is composed of Li + ion conduction channels and is made of high amounts of sintered ceramic Li 1+ x Ti 2-x Al x (PO 4 ) 3 (LATP) and epoxy polymer with enhanced mechanical properties for solid-state batteries. The two- and three-dimensional characterizations are conducted not only for showing continuous Li+ ion channels thorough LATP ceramic channels with over 10-4 S cm-1 of ionic conductivity but also to investigate small amounts of epoxy polymer with enhanced mechanical properties. Solid-state Li + ion cells are fabricated using the ERTCF and they show initial charge–discharge capacities of 139/133 mAh g -1 . Furthermore, the scope of the ERTCF is expanded to high-voltage (>8 V) solid-state Li + ion batteries through a bipolar stacked cell design. Hence, it is expected that the present investigation will significantly contribute in the preparation of the next generation reinforced thin ceramic film electrolytes for high-voltage solid-state batteries.

25 ENERGY STORAGE↗

DOE OSTI2018

A Silica-Aerogel-Reinforced Composite Polymer Electrolyte with High Ionic Conductivity and High Modulus

High-energy all-solid-state lithium (Li) batteries have great potential as next-generation energy-storage devices. Among all choices of electrolytes, polymer-based systems have attracted widespread attention due to their low density, low cost, and excellent processability. However, they are generally mechanically too weak to effectively suppress Li dendrites and have lower ionic conductivity for reasonable kinetics at ambient temperature. Here, an ultrastrong reinforced composite polymer electrolyte (CPE) is successfully designed and fabricated by introducing a stiff mesoporous SiO 2 aerogel as the backbone for a polymer-based electrolyte. The interconnected SiO 2 aerogel not only performs as a strong backbone strengthening the whole composite, but also offers large and continuous surfaces for strong anion adsorption, which produces a highly conductive pathway across the composite. As a consequence, a high modulus of ≈0.43 GPa and high ionic conductivity of ≈0.6 mS cm -1 at 30°C are simultaneously achieved. Furthermore, LiFePO 4 –Li full cells with good cyclability and rate capability at ambient temperature are obtained. Full cells with cathode capacity up to 2.1 mAh cm -2 are also demonstrated. The aerogel-reinforced CPE represents a new design principle for solid-state electrolytes and offers opportunities for future all-solid-state Li batteries.

25 ENERGY STORAGE↗

DOE OSTI2022

Dual–Interface–Reinforced Flexible Perovskite Solar Cells for Enhanced Performance and Mechanical Reliability

Two key interfaces in flexible perovskite solar cells (f-PSCs) are mechanically reinforced simultaneously: one between the electron-transport layer (ETL) and the 3D metal-halide perovskite (MHP) thin film using self-assembled monolayer (SAM), and the other between the 3D-MHP thin film and the hole-transport layer (HTL) using an in situ grown low-dimensional (LD) MHP capping layer. The interfacial mechanical properties are measured and modeled. This rational interface engineering results in the enhancement of not only the mechanical properties of both interfaces but also their optoelectronic properties holistically. As a result, the new class of dual-interface-reinforced f-PSCs has an unprecedented combination of the following three important performance parameters: high power-conversion efficiency (PCE) of 21.03% (with reduced hysteresis), improved operational stability of 1000 h T90 (duration at 90% initial PCE retained), and enhanced mechanical reliability of 10 000 cycles n88 (number of bending cycles at 88% initial PCE retained). Furthermore, the scientific underpinnings of these synergistic enhancements are elucidated.

14 SOLAR ENERGY↗

DOE OSTI2025

Polypropylene Composites Reinforced With Recycled Waste Cellulosic Fiber/Fine Mixture: The Impact of Cellulose Sieving on Performance

This study explores how a sieving step of waste cellulosic fiber and fine (WCFF) mixture affects the performance of WCFF‐loaded polypropylene (PP) composites and whether the separation of fines from fibers offers an added benefit. The WCFF samples were downsized, and four different filler size ranges were sieved using a series of mesh sizes from 4 to 0.85 mm. The WCFF/PP composites were then compounded at 20 wt.% loading of WCFF using a twin‐screw extruder. Incorporating WCFF increased the tensile strength to 41.28 MPa and the modulus to 3207 MPa, accounting for 28% and 38% enhancements, respectively. Interestingly, the greatest improvements were associated with the nonsieved WCFF case, and the sieved WCFF fibers provided only marginal enhancements over virgin PP. The outperformance of nonsieved WCFF was attributed to the synergistic reinforcement of hybrid fibers and fines as well as the maintenance of longer fibers in the system. However, the strain at break and impact strength of PP decreased after introducing WCFF. Moreover, the complex viscosity and storage modulus increased with an increase in the filler size, due to the formation of a more effective percolative network. The PP's crystallinity exhibited a relatively strong dependency on the sieving, where WCFF samples with short‐aspect‐ratio fillers promoted the crystallinity significantly. It was also found that the WCFF degradation onset temperature increased once it was incorporated into PP. This study suggests that waste cellulosic feedstocks can be utilized as a reinforcement without additional sieving to manufacture high‐performance and cost‐effective composites.

36 MATERIALS SCIENCE↗

DOE OSTI2013

Reinforced Electrode Architecture for a Flexible Battery with Paperlike Characteristics

Compliant energy storage has not kept pace with flexible electronics. Herein we demonstrate a technique to reinforce arbitrary battery electrodes by supporting them with mechanically tough, low-cost fibrous membranes, which also serve as the separator. The membranes were laminated to form a full cell, and this stacked membrane reinforcement bears the loads during flexing. This technique was used to make a high energy density, nontoxic Zn-MnO2 battery with printed current collectors. The Zn and MnO2 electrodes were prepared by using a solution-based embedding process. The cell had a nominal potential of 1.5 V and an effective capacity of approximately 3 mA h cm(-2). We investigated the effect of bending and fatigue on the electrochemical performance and mechanical integrity of the battery. The battery was able to maintain its capacity even after 1000 flex cycles to a bend radius of 2.54 cm. The battery showed an improvement in discharge capacity (ca. 10%) if the MnO2 electrode was flexed to tension as a result of the improvement of particle-to-particle contact. In a demonstration, the flexible battery was used to power a light-emitting diode display integrated with a strain sensor and microcontroller.

Gaikwad, AM↗