Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,675 records · Page 93

Using Reinforcement Learning to Design Missed Thrust Resilient Trajectories

From ion thrusters to solar sails, spacecraft continue to adopt new and more efficient forms of propulsion. As these low-thrust propulsion meth- ods have become more prevalent, new challenges have arisen. Depending on the mission, low-thrust propulsion elements may need to thrust con- tinuously for days/months. During these thrusting periods, external fac- tors, such as a micro-meteoroid impact or a software glitch, may cause the spacecraft to prematurely cease its thrust stage. Half of all deep space missions enter a safe mode where they cannot thrust every four months. These missed thrust events can result in the complete loss of a space- craft for time-dependent trajectories like planetary rendezvous. This paper demonstrates how neural networks, trained using reinforcement learning, can autonomously correct for missed thrust events during an interplanetary trajectory.

Laipert, Frank E.↗

Characterizing Char Rate and Extent in Fiber-Reinforced Plastics Using X-ray Computed Tomography

Orbital debris is a growing problem for the space industry and the world in general, and an important component of the problem is what happens when that debris reenters the Earth’s atmosphere. With more spacecraft opting for fiber-reinforced polymer (FRP) components, we need to understand the thermal destruction process of these materials during an atmospheric reentry and how much of the material can survive to impact the ground. The NASA Orbital Debris Program Office (ODPO) has developed a new charring model for FRP components to be integrated into the latest version of the Object Reentry Survival Analysis Tool (ORSAT). To validate this new model, the ODPO performed several test series using the Inductively Coupled Plasma (ICP) Torch facility at UT Austin. To measure the extent of charred material at different conditions, some of the test samples were scanned using x-ray computed tomography (CT) by the Astromaterials Curation Lab at Johnson Space Center. 3D image analysis was then used to calculate the volume and density of the char in each test sample. This paper presents the image analysis methodology, an assessment of the accuracy of the data analysis, and a comparison with calculations using the ORSAT charring model.

Benton Greene↗

Airport Runway Configuration Management with Offline Model-free Reinforcement Learning

Runway configuration management (RCM) deals with the optimal selection of runways to operate on (for arrivals and departures) based on traffic, surface wind speed, wind direction and other environmental variables. RCM is one of the most challenging tasks in air traffic management, as it relies on operational and environmental variables (e.g., weather forecast) that are highly uncertain and complex to model. In this paper, an innovative and automated approach is deployed using offline model-free reinforcement learning to provide decision-support for RCM. The proposed technology processes historical data about variables of interest, decisions made regarding RCM, and their subsequent outcome, to identify a policy that would encourage good decisions and avoid the poor ones. The policy search is guided by an appropriately chosen weighted utility function (e.g., based on minimizing delays and go-arounds). Finally, the performance of the proposed tool is validated using Charlotte Douglas International Airport as the case study, which shows that the proposed method is superior to other conventional rule-based approaches.

Milad Memarzadeh↗

Airport Runway Configuration Management with Offline Model-free Reinforcement Learning

Runway configuration management (RCM) deals with the optimal selection of runways to operate on (for arrivals and departures) based on traffic, surface wind speed, wind direction and other environmental variables. RCM is one of the most challenging tasks in air traffic management, as it relies on operational and environmental variables (e.g., weather forecast) that are highly uncertain and complex to model. In this paper, an innovative and automated approach is deployed using offline model-free reinforcement learning to provide decision-support for RCM. The proposed technology processes historical data about variables of interest, decisions made regarding RCM, and their subsequent outcome, to identify a policy that would encourage good decisions and avoid the poor ones. The policy search is guided by an appropriately chosen weighted utility function (e.g., based on minimizing delays and go-arounds). Finally, the performance of the proposed tool is validated using Charlotte Douglas International Airport as the case study, which shows that the proposed method is superior to other conventional rule-based approaches.

Milad Memarzadeh↗

Exploring the Low-Thrust Transfer Design Space in an Ephemeris Model via Multi-Objective Reinforcement Learning

Multi-Reward Proximal Policy Optimization (MRPPO) is a multi-objective reinforcement learning algorithm used to train multiple policies to uncover solutions within a multi-objective solution space. MRPPO is used in this paper to train policies to construct low-thrust transfers for a SmallSat from the vicinity of 𝐿2 to an 𝐿5 short period orbit in the Sun-Earth-Moon system. First, the policies are trained in this scenario in the circular restricted three-body problem. This information is used to initialize the policies before training in a higher-fidelity ephemeris model; a process known as transfer learning. The recovered segments of the solution space will be compared to fundamental dynamical structures to both examine the results of MRPPO in this complex design scenario and explore the effectiveness of transfer learning.

Mashiku, Alinda K.↗

Joint Communication Resource Allocation and Velocity Selection in Urban Air Mobility via Multi-agent Reinforcement Learning

With traffic congestion problems becoming more severe in urban areas, the National Aeronautics and Space Administration promotes the Urban Air Mobility (UAM) concept, which envisages a safe and efficient air transportation system. However, the increased communication demands in UAM can exacerbate the spectrum scarcity. Therefore, a new communication resource allocation solution is necessary. In this paper, we focus on uplink UAM communications, where multiple aerial vehicles (AV) perform cargo/passenger delivery tasks. With predefined flight paths, AVs make decisions on communication resource allocation and velocity selection to complete their missions under safety constraints. Accordingly, we formulate a joint optimization problem to minimize the weighted sum of the total travel time and communication outage time. We first model the optimization problem as a Markov game and propose a multi-agent reinforcement learning based solution. Simulation results corroborate the effectiveness of the proposed solution.

Ruixuan Han↗

Optimization of Airport Runway Configuration with Forecast-Augmented Offline Reinforcement Learning

Runway configuration Management (RCM) governs the optimal utilization of runways based on variables such as traffic and meteorological conditions, making it a daunting task in air traffic management due to its dependency on volatile operational and environmental factors. This paper improves upon our previous work [1] on using offline model-free reinforcement learning for creating a Runway Configuration Assistance (RCA) decision-support tool. A novel integration of forecast data from LAMP (Localized Aviation Model Output Statistics Program) and TAF (Terminal Area Forecast) is introduced, enhancing the tool’s accuracy and also its adaptability to quick wind changes. The performance is evaluated using two major US airports, Charlotte Douglas International Airport (CLT) and Denver International Airport (DEN). To counter scalability issues presented by the addition of discrete forecast variables, we transitioned to a continuous state space model, ensuring scalability and inclusion of longer forecast data. The results of our experiments reflect significant improvements in the RCA tool’s prediction accuracy.

Sumanth Nethi↗

Modeling Dynamic Crush Behavior of Carbon Fiber Reinforced Polymer Composite Structures Using MAT213

Modeling crushing of carbon fiber reinforced polymer (CFRP) composites is challenging, and current simulation methodologies involve tuning of non-physical parameters. MAT213, a next-generation material model, has advanced functionality to better simulate dynamic impact loading. The objective of the present investigation is to evaluate the potential for using MAT213 to simulate dynamic crushing of CFRPs. Two sets of simulations were performed: one for calibration based on a set of coupon-level experiments and another for prediction of the response of structural elements. Simulations involving dynamic crushing of flat specimens were iteratively run to calibrate model parameters. The calibration demonstrated that MAT213 could produce a simulated force-displacement response within experimental scatter. The simulated failure morphology was also comparable to the experiments. After successful calibration, predictive simulations of dynamic crushing of C-channel shaped specimens were completed using a simulated crash sled test rig and two pairs of impactor mass/velocity conditions. The simulated force-displacement curve in the crash sled simulations for the lower-velocity condition fell within the experimental scatter, but the stable crush force was underpredicted by 27% in the higher-velocity simulations. Better correlation in the lower-velocity test condition likely results from the calibration condition being a similar velocity to the lower-velocity crash sled condition.

Dynamic Crush↗

SoMoGym: A Toolkit for Developing and Evaluating Controllers and Reinforcement Learning Algorithms for Soft Robots

Soft robotsoffer a host of benefits over traditional rigid robots, including inherent compliance that lets them passively adapt to variable environments and operate safely around humans and fragile objects. However, that same compliance makes it hard to use model-based methods in planning tasks requiring high precision or complex actuation sequences. Reinforcement learning (RL) can potentially find effective control policies, but training RL using physical soft robots is often infeasible, and training using simulations has had a high barrier to adoption. To accelerate research in control and RL for soft robotic systems, we introduce SoMoGym ( So ft Mo tion Gym ), a software toolkit that facilitates training and evaluating controllers for continuum robots. SoMoGym provides a set of benchmark tasks in which soft robots interact with various objects and environments. It allows evaluation of performance on these tasks for controllers of interest, and enables the use of RL to generate new controllers. Custom environments and robots can likewise be added easily. We provide and evaluate baseline RL policies for each of the benchmark tasks. These results show that SoMoGym enables the use of RL for continuum robots, a class of robots not covered by existing benchmarks, giving them the capability to autonomously solve tasks that were previously unattainable.

Moritz A. Graule↗

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learningautonomous systems; flight simulat↗

A Reinforcement Learning Hyper-Heuristic in Multi-Objective Optimization with Application to Structural Damage Identification

Multi-objective optimization allows satisfying multiple decision criteria concurrently, and generally yields multiple solutions. It has the potential to be applied to structural damage identification applications which are oftentimes under-determined. How to achieve high-quality solutions in terms of accuracy, diversity, and completeness is a challenging research subject. The solution techniques and parametric selections are believed to be problem specific. In this research, we formulate a reinforcement learning hyper-heuristic scheme to work coherently with the single-point search algorithm MOSA/R (Multi-Objective Simulated Annealing Algorithm based on Re-seed). The four low-level heuristics proposed can meet various optimization requirements adaptively and autonomously using the domination amount, crowding distance, and hypervolume calculations. The new approach exhibits improved and more robust performance than AMOSA, NSGA-II, and MOEA/D when applied to benchmark test cases. It is then applied to an active damage interrogation scheme for structural damage identification where solution diversity/completeness and accuracy are critically important. Results show that this approach can successfully include the true damage scenario in the solution set identified. The outcome of this research can potentially be extended to a variety of applications.

Pei Cao↗

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learning↗

Microscale Constitutive Model Sensitivity on Multiscale Modeling of Fiber Reinforced Composites

Fiber reinforced composites are desirable in applications where low weight and high strength are needed, but are susceptible to microscale variability during manufacturing, making failure predictions difficult. The impact microscale variability has on macroscale mechanical response is difficult to predict due to the computational efficiency needed to simulate many, large, high fidelity, microscale models. In this study, a multiscale approach was taken to model 3-point bend, 4-point bend, and tensile experiments of a unidirectional composite from only having microstructure scans of these samples and constituent properties from literature. These scans were sampled with different sized windows, and statistically equivalent microstructures were generated, then simulated for stiffness, strength, and fracture toughness using an efficient micromechanical. Mesoscale models were created where element sizes equaled microstructure size, and properties were assigned through sampling of microscale simulation results. First, this study showed the effect of using Weibull scaling on constituent matrix strength on macroscale response. Then, a comparison was made between different element sizes and experiments. Finally, model dimensions were fixed, and the effect of randomly distributed local properties alone was examined. Results showed that the scatter of strength and stiffness in the experiments could be predicted well using images of the microscale fiber morphologies and that using stochastic properties produced a 3% coefficient of variation of strength for all experiments.

statistical microstructure↗

Manufacturing Process Development of a Carbon Fiber Reinforced Polymer Composite Shaft for Electric Motors

Electric aircraft applications require electric motors with increased specific power and efficiency. Composite structural components in motors are a potential solution for reducing motor mass, reducing magnetic losses, and limiting undesired conduction paths for fault, electromagnetic interference, or common-mode currents. In this report, manufacturing trials for a high-speed carbon fiber reinforced polymer composite motor shaft are presented. Four prototype shafts were produced using a hybrid biaxial/triaxial fabric that was circumferentially wrapped onto an additively manufactured high-temperature washout mandrel. An additional traditional overbraid approach was also evaluated and shows promise for high-rate, high-performance parts using automated manufacturing. This paper discusses the shaft design, manufacturing methods explored, material selection, the manufacturing trials, and the lessons learned. The results of this manufacturing investigation show feasibility for manufacturing composite shafts for electric motors.

Electric moto shaft↗

Mechanical Properties of Carbon Fiber Reinforced Composites Exposed to Cryogenic Conditions and Space Radiation via Simulation and Testing

As NASA missions extend beyond low Earth orbit, increasing reliance is placed on carbon fiber reinforced polymer (CFRP) composites for spacecraft structures where mass efficiency, durability, and long-term reliability are critical. In service, these materials are subjected to a combination of ultraviolet radiation, vacuum, ionizing radiation, atomic oxygen, and extreme thermal excursions under sustained mechanical loading. Flight systems such as the Boeing Starliner and SpaceX Dragon employ external composite structures that will experience these environments for extended durations. Although prior spaceflight and ground studies have reported limited changes in bulk mechanical properties, the synergistic effects of these environments on composite microstructure, particularly at the fiber matrix interphase, remain insufficiently characterized and represent a potential qualification and reliability risk. This study investigates the effects of short-term cryogenic exposure on a radiation shielding carbon epoxy composite, SC2020, as a ground-based analog for space relevant thermal extremes. The SC2020 material system has previously flown on the International Space Station under the Materials International Space Station Experiment (MISSE) program. Composite specimens were exposed to liquid nitrogen for 6 and 24 hours and evaluated using a multiscale characterization framework that combined ASTM D3039 tensile testing, Atomic Force Microscopy (AFM) based interphase analysis, and helium gas permeability measurements. Tensile testing showed no statistically significant or permanent degradation in global strength or modulus following cryogenic exposure. In contrast, AFM measurements revealed reductions in interphase modulus, weakened adhesion, and increased nanoscale heterogeneity, indicating localized degradation mechanisms not captured by conventional bulk testing. Gas permeability measurements showed a progressive increase in helium diffusion with exposure duration, consistent with micro-void formation or partial interfacial debonding. The results indicate that cryogenic exposure initiates degradation at the fiber matrix interphase while leaving global mechanical properties largely unchanged over short durations. These findings underscore the importance of multiscale diagnostics for identifying early-stage damage mechanisms that may influence long term performance and qualification margins for spaceflight composite structures. The data presented establish a cryogenic baseline for comparison with forthcoming MISSE flight exposure results and support ongoing NASA Established Program to Stimulate Competitive Research (EPSCoR) efforts aimed at improving composite qualification methodologies, risk assessment, and reliability prediction for space environments.

composite reliability↗

Morphological characterization of carbon-nanofiber-reinforced epoxy nanocomposites using ultra-small angle scattering

Studies of the properties of nanocomposites reinforced with vapor-grown carbon nanofibers (VGCFs) can be found throughout the literature. Electrical, mechanical, viscoelastic, and rheological properties are just a few of the characteristics that have been well discussed. Although these properties depend on morphology, morphological characterization is rare. Due to its 2-dimensional nature, microscopy is of limited value when analyzing network morphologies. This work will show how the characterization of the three-dimensional geometry and network formation of VGCFs can be determined using ultra-small angle scattering techniques. Ultra-small angle x-ray and neutron scattering (USAXS and USANS) were used to characterize the morphology of carbon nanofibers suspended in epoxy. Using a simplified tube model, we estimate the dimensions of suspended fibers. The assumption of tubular fibers accounts for the increased surface area observed with USAXS that is not accounted for using a solid rod model. Furthermore, USANS was used to search for a structural signature associated with the electrical percolation threshold. USANS extends to longer dimensional scales than USAXS, which measures a smaller range of momentum transfer. To determine the electrical percolation threshold, AC impedance spectroscopy was employed to verify that an electrically conductive, percolated network forms at VGCNF loadings of 0.8% < CNF wt% < 1.2%. These values correlate with the USANS data, where a morphological transition is seen at {approx}1.2% loading.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Turbomachine blade reinforcement

Embodiments of the present disclosure include a system having a turbomachine blade segment including a blade and a mounting segment coupled to the blade, wherein the mounting segment has a plurality of reinforcement pins laterally extending at least partially through a neck of the mounting segment.

42 ENGINEERING↗

Sensored fiber reinforced polymer grate

Various technologies described herein pertain to a sensored grate that can be utilized for various security fencing applications. The sensored grate includes a grate framework and an embedded optical fiber. The grate framework is formed of a molded polymer such as, for instance, molded fiber reinforced polymer. Further, the grate framework includes a set of elongated elements, where the elongated elements are spaced to define apertures through the grate framework. The optical fiber is embedded in the elongated elements of the grate framework. Moreover, bending or breaking of one or more of the elongated elements can be detected based on a change in a characteristic of input light provided to the optical fiber compared to output light received from the optical fiber.

36 MATERIALS SCIENCE↗