Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “RL”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2014

Announced Strategy Types in Multiagent RL for Conflict-Avoidance in the National Airspace

The use of unmanned aerial systems (UAS) in the national airspace is of growing interest to the research community. Safety and scalability of control algorithms are key to the successful integration of autonomous system into a human-populated airspace. In order to ensure safety while still maintaining efficient paths of travel, these algorithms must also accommodate heterogeneity of path strategies of its neighbors. We show that, using multiagent RL, we can improve the speed with which conflicts are resolved in cases with up to 80 aircraft within a section of the airspace. In addition, we show that the introduction of abstract agent strategy types to partition the state space is helpful in resolving conflicts, particularly in high congestion.

National Airspace↗
NASA NTRS2015

Comparison Between CCCM and CloudSat Radar-Lidar (RL) Cloud and Radiation Products

To enhance cloud properties, LaRC and CIRA developed each combination algorithm for obtained properties from passive, active and imager in A-satellite constellation. When comparing global cloud fraction each other, LaRC-produced CERES-CALIPSO-CloudSat-MODIS (CCCM) products larger low-level cloud fraction over tropic ocean, while CIRA-produced Radar-Lidar (RL) shows larger mid-level cloud fraction for high latitude region. The reason for different low-level cloud fraction is due to different filtering method of lidar-detected cloud layers. Meanwhile difference in mid-level clouds is occurred due to different priority of cloud boundaries from lidar and radar.

Ham, Seung-Hee↗
DOE OSTI2019

Distributed Reinforcement Learning with ADMM-RL

NREL researcher Peter Graf discusses distributed reinforcement learning with ADMM-RL as part of the Innovative Optimization and Control Methods for Highly Distributed Autonomous Systems workshop. NREL hosted the workshop to exchange ideas on state-of-the-art control and optimization strategies as well as get feedback on work being done at NREL.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Raman Lidar (RL) Instrument Handbook

The Raman lidar at the ARM Climate Research Facility (ACRF) Southern Great Plains (SGP) Central Facility (SGPRL) is an active, ground-based laser remote sensing instrument that measures height and time resolved profiles of water vapor mixing ratio and several cloud- and aerosol-related quantities. The system is a non-commercial custom-built instrument developed by Sandia National Laboratories specifically for the ARM Program. It is fully computer automated, and will run unattended for many days following a brief (~5-minute) startup period. The self-contained system (requiring only external electrical power) is housed in a climate-controlled 8’x8’x20’ standard shipping container.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Reference document for LANL stack sampling and ANSI N13.1 (Article) Gielow RL and McNamee MR 1993. Numerical Flue Gas Flow Modeling for Continuous Emissions Monitoring Applications. EPRI CEM Users Group Meeting. Baltimore. RP1961-13

American National Standard N13.1 “sets forth guidelines and performance criteria for sampling the emissions of airborne radioactive substances in the air discharge ducts and stacks of nuclear facilities. Emphasis is on extractive sampling from a location in a stack or duct where the contaminant is well mixed. At such a location, sampling may be conducted at a single point. This standard provides performance-based criteria for the use of air sampling probes, transport lines, sample collectors, sample monitoring instruments, and gas flow measuring methods. This standard also covers sampling program objectives, quality assurance issues, developing air sampling action levels, system optimization, and system performance verification. Workplace, containment, and environmental air monitoring are not addressed. Specific sample analysis methods and the reporting or interpreting of results are also not addressed.” (HPS 2011).

61 RADIATION PROTECTION AND DOSIMETRY↗
NASA NTRS1975

Stress relaxation and mechanical properties of RL-1973 and PD-200-16 silicone resin sponge materials

Stress relaxation tests were conducted by loading specimens in double-lap shear to a preselected strain level and monitoring the decay of stress with time. The stress relaxation response characteristics were measured over a temperature range of 100 to 300 K and four strain levels. It is concluded that only a slight amount of stress relaxation was observed, and the stiffness increased approximately two orders of magnitude over the range of temperatures.

Saylak, D.↗
NASA NTRS1999

RL-10 Based Combined Cycle For A Small Reusable Single-Stage-To-Orbit Launcher

This paper discusses a new application of the combined propulsion known as the KLIN(TM) cycle, consisting of a thermally integrated deeply cooled turbojet (DCTJ) and liquid rocket engine (LRE). If based on the RL10 rocket engine family, the KLIN (TM) cycle makes a small single-stage-to-orbit (SSTO) reusable launcher feasible and economically very attractive. Considered in this paper are the concept and parameters of a small SSTO reusable launch vehicle (RLV) powered by the KLIN (TM) cycle (sSSTO(TM)) launcher. Also discussed are the benefits of the small launcher, the reusability, and the combined cycle application. This paper shows the significant reduction of the gross take off weight (GTOW) and dry weight of the KLIN(TM) cycle-powered launcher compared to an all-rocket launcher.

Balepin, Vladimir↗
NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

reinforcement learning↗
NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

Validation↗
NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

Validation↗
NASA NTRS2026

Deep RL for Fast Long-Horizon Operations Scheduling on NASA's Carruthers Geocorona Observatory Mission

Spacecraft operations scheduling is a highly constrained, long-horizon combinatorial optimization problem that traditionally relies on heuristics, constraint programming, or manual planning. We present a scalable deep reinforcement learning framework developed and deployed for NASA’s Carruthers Geocorona Observatory mission. Our framework introduces a macro-action abstraction known as activity blocks coupled with dynamic action-masking to navigate the intractably large search space and strictly enforce complex power, thermal, and instrument constraints. The resulting architecture generates globally feasible schedules with overwhelming probability, establishes operational trust, and executes a full training cycle in under six hours, circumventing the need for policy robustness by enabling rapid, on-demand retraining. Further, resulting schedules outperform baseline heuristics in scheduled science quality. The deep reinforcement learning framework was deployed as the default operational scheduler for the Carruthers Geocorona Observatory mission from the outset of the mission, demonstrating that deep reinforcement learning can be trusted for real spacecraft operations under complex, evolving constraints.

Geocorona↗
DOE OSTI2023

Reinforcement-Learning-Based Smart Water Heater Control: An Actual Deployment

Utilizing smart control algorithms for electric water heaters (EWHs) is essential for fully harnessing the demand response (DR) potential of EWHs. For this reason, the use of reinforcement learning (RL) algorithms for EWHs has received increasing attention in recent years. However, existing RL approaches are either simulation-based or use pretrained RL agents. To this end, this paper presents the real-world deployment of a set of model-free RL approaches that aim to minimize the electricity cost of a EWH under a time-of-use electricity pricing policy using standard DR commands (e.g., shed, load up). The experiment results showed that the RL agents can help save electricity cost in the range of 11% to 14% compared to the baseline operation. This study demonstrated that RL-based EWH controllers can be deployed in real world without any prior training and can still save electricity cost.

deep learning↗
DOE OSTI2020

Flexible Reinforcement Learning Framework for Building Control using EnergyPlus-Modelica Energy Models

In recent years, reinforcement learning (RL) methods have been greatly enhanced by leveraging deep learning approaches. RL methods applied to building control have shown potential in many applications due to their ability to complement or replace conventional methods such as model-based or rule-based controls. However, RL-based building control software is likely tailored either to one target building system or to a specific RL method so that significant additional effort would be required to customize the RL-based controller for use in other building systems or with other RL approaches. Also, RL-based building controls usually depend on building energy simulations to train controllers, so emulating building dynamics (i.e., thermal dynamics and control dynamics) and capturing sub-hourly dynamic profiles are crucial to further the development of effective RL-based building control methods. To address these challenges, we present an open source RL-based control software employing a high-fidelity hybrid EnergyPlus-Modelica building energy model which emulates building dynamics at 1-minute resolution. This software consists of decoupled components (environment, building emulator, control agent, and RL algorithm), which allows for quick prototyping and benchmarking of standard RL algorithms in different systems; for example, a single component can be replaced without revising all of the software. To demonstrate this software framework, we conducted a benchmark study using an EnergyPlus-Modelica building energy model for a Chicago office building with an RL-based controller to dynamically control the chilled water temperature setpoint and the air handling unit supply air temperature setpoint on selected floors.

Lee, Joon-Yong↗