Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “RL”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1998

Safety Analysis of FMS/CTAS Interactions During Aircraft Arrivals

This grant funded research on human-computer interaction design and analysis techniques, using future ATC environments as a testbed. The basic approach was to model the nominal behavior of both the automated and human procedures and then to apply safety analysis techniques to these models. Our previous modeling language, RSML, had been used to specify the system requirements for TCAS II for the FAA. Using the lessons learned from this experience, we designed a new modeling language that (among other things) incorporates features to assist in designing less error-prone human-computer interactions and interfaces and in detecting potential HCI problems, such as mode confusion. The new language, SpecTRM-RL, uses "intent" abstractions, based on Rasmussen's abstraction hierarchy, and includes both informal (English and graphical) specifications and formal, executable models for specifying various aspects of the system. One of the goals for our language was to highlight the system modes and mode changes to assist in identifying the potential for mode confusion. Three published papers resulted from this research. The first builds on the work of Degani on mode confusion to identify aspects of the system design that could lead to potential hazards. We defined and modeled modes differently than Degani and also defined design criteria for SpecTRM-RL models. Our design criteria include the Degani criteria but extend them to include more potential problems. In a second paper, Leveson and Palmer showed how the criteria for indirect mode transitions could be applied to a mode confusion problem found in several ASRS reports for the MD-88. In addition, we defined a visual task modeling language that can be used by system designers to model human-computer interaction. The visual models can be translated into SpecTRM-RL models, and then the SpecTRM-RL suite of analysis tools can be used to perform formal and informal safety analyses on the task model in isolation or integrated with the rest of the modeled system. We had hoped to be able to apply these modeling languages and analysis tools to a TAP air/ground trajectory negotiation scenario, but the development of the tools took more time than we anticipated.

Nancy G. Leveson↗
NASA NTRS1999

A Survey of Collective Intelligence

This chapter presents the science of "COllective INtelligence" (COIN). A COIN is a large multi-agent systems where: i) the agents each run reinforcement learning (RL) algorithms; ii) there is little to no centralized communication or control; iii) there is a provided world utility function that, rates the possible histories of tile full system. Tile conventional approach to designing large distributed systems to optimize a world utility does not use agents running RL algorithms. Rather that approach begins with explicit modeling of the overall system's dynamics, followed by detailed hand-tuning of the interactions between the components to ensure that they "cooperate" as far as the world utility is concerned. This approach is labor-intensive, often results in highly non-robust systems, and usually results in design techniques that, have limited applicability. In contrast, with COINs we wish to solve the system design problems implicitly, via the 'adaptive' character of the RL algorithms of each of the agents. This COIN approach introduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we do not have phenomena like the tragedy of the commons, or Braess's paradox? Although still very young, the science of COINs has already resulted in successes in artificial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's "El Farol bar problem". It is expected that as it matures not only will COIN science expand greatly the range of tasks addressable by human engineers, but it will also provide much insight into already established scientific fields, such as economics, game theory, or population biology.

Wolpert, David H.↗
NASA NTRS1996

Shape deformation of the organ of Corti associated with length changes of outer hair cell

Cochlear outer hair cells (OHC) are commonly assumed to function as mechanical effectors as well as sensory receptors in the organ of Corti (OC) of the inner ear. OHC in vitro and in organ explants exhibit mechanical responses to electrical, chemical or mechanical stimulation which may represent an aspect of their effector process that is expected in vivo. A detailed description, however, of an OHC effector operation in situ is still missing. Specifically, little is known as to how OHC movements influence the geometry of the OC in situ. Previous work has demonstrated that the motility of isolated OHCs in response to electrical stimulation and to K(+)-gluconate is probably under voltage control and causes depolarisation (shortening) and hyperpolarization (elongation). This work was undertaken to investigate if the movements that were observed in isolated OHC, and which are induced by ionic stimulation, could change the geometry of the OC. A synchronized depolarization of OHC was induced in guinea pig cochleae by exposing the entire OC to artificial endolymph (K+). Subsequent morphometry of mid-modiolar sections from these cochleae revealed that the distance between the basilar membrane (BM) and the reticular lamina (RL) had decreased considerably. Furthermore, in the three upper turns OHC had significantly shortened in all rows. The results suggest that OHC can change their length in the organ of Corti (OC) thus deforming the geometry of the OC. The experiments reveal a tonic force generation within the OC that may change the position of RL and/or BM, contribute to damping, modulate the BM-RL-distance and control the operating points of RL and sensory hair bundles. Thus, the results suggest active self-adjustments of cochlear mechanics by slow OHC length changes. Such mechanical adjustments have recently been postulated to correspond to timing elements of animal communication, speech or music.

Non-NASA Center↗
NASA NTRS1973

Design study of RL10 derivatives. Volume 2: Engine design characteristics, appendices

Calculations, curves, and substantiating data which support the engine design characteristics of the RL-10 engines are presented. A description of the RL-10 ignition system is provided. The performance calculations of the RL-10 derivative engines and the performance results obtained are reported. The computer simulations used to establish the control system requirements and to define the engine transient characteristics are included.

Source record↗
NASA NTRS1999

Designing Specification Languages for Process Control Systems: Lessons Learned and Steps to the Future

Previously, we defined a blackbox formal system modeling language called RSML (Requirements State Machine Language). The language was developed over several years while specifying the system requirements for a collision avoidance system for commercial passenger aircraft. During the language development, we received continual feedback and evaluation by FAA employees and industry representatives, which helped us to produce a specification language that is easily learned and used by application experts. Since the completion of the PSML project, we have continued our research on specification languages. This research is part of a larger effort to investigate the more general problem of providing tools to assist in developing embedded systems. Our latest experimental toolset is called SpecTRM (Specification Tools and Requirements Methodology), and the formal specification language is SpecTRM-RL (SpecTRM Requirements Language). This paper describes what we have learned from our use of RSML and how those lessons were applied to the design of SpecTRM-RL. We discuss our goals for SpecTRM-RL and the design features that support each of these goals.

Leveson, Nancy G.↗
NASA NTRS2000

Reinforcement Learning in Distributed Domains: Beyond Team Games

Distributed search algorithms are crucial in dealing with large optimization problems, particularly when a centralized approach is not only impractical but infeasible. Many machine learning concepts have been applied to search algorithms in order to improve their effectiveness. In this article we present an algorithm that blends Reinforcement Learning (RL) and hill climbing directly, by using the RL signal to guide the exploration step of a hill climbing algorithm. We apply this algorithm to the domain of a constellations of communication satellites where the goal is to minimize the loss of importance weighted data. We introduce the concept of 'ghost' traffic, where correctly setting this traffic induces the satellites to act to optimize the world utility. Our results indicated that the bi-utility search introduced in this paper outperforms both traditional hill climbing algorithms and distributed RL approaches such as team games.

Wolpert, David H.↗
NASA NTRS2004

Adaptive, Distributed Control of Constrained Multi-Agent Systems

Product Distribution (PO) theory was recently developed as a broad framework for analyzing and optimizing distributed systems. Here we demonstrate its use for adaptive distributed control of Multi-Agent Systems (MASS), i.e., for distributed stochastic optimization using MAS s. First we review one motivation of PD theory, as the information-theoretic extension of conventional full-rationality game theory to the case of bounded rational agents. In this extension the equilibrium of the game is the optimizer of a Lagrangian of the (Probability dist&&on on the joint state of the agents. When the game in question is a team game with constraints, that equilibrium optimizes the expected value of the team game utility, subject to those constraints. One common way to find that equilibrium is to have each agent run a Reinforcement Learning (E) algorithm. PD theory reveals this to be a particular type of search algorithm for minimizing the Lagrangian. Typically that algorithm i s quite inefficient. A more principled alternative is to use a variant of Newton's method to minimize the Lagrangian. Here we compare this alternative to RL-based search in three sets of computer experiments. These are the N Queen s problem and bin-packing problem from the optimization literature, and the Bar problem from the distributed RL literature. Our results confirm that the PD-theory-based approach outperforms the RL-based scheme in all three domains.

Bieniawski, Stefan↗
NASA NTRS1990

Light-stimulated cell expansion in bean (Phaseolus vulgaris L.) leaves. II. Quantity and quality of light required

The quantity and quality of light required for light-stimulated cell expansion in leaves of Phaseolus vulgaris L. have been determined. Seedlings were grown in dim red light (RL; 4 micromoles photons m-2 s-1) until cell division in the primary leaves was completed, then excised discs were incubated in 10 mM sucrose plus 10 mM KCl in a variety of light treatments. The growth response of discs exposed to continuous white light (WL) for 16 h was saturated at 100 micromoles m-2 s-1, and did not show reciprocity. Extensive, but not continuous, illumination was needed for maximal growth. The wavelength dependence of disc expansion was determined from fluence-response curves obtained from 380 to 730 nm provided by the Okazaki Large Spectrograph. Blue (BL; 460 nm) and red light (RL; 660 nm) were most effective in promoting leaf cell growth, both in photosynthetically active and inhibited leaf discs. Far-red light (FR; 730 nm) reduced the effectiveness of RL, but not BL, indicating that phytochrome and a separate blue-light receptor mediate expansion of leaf cells.

NASA Discipline Plant Biology↗
NASA NTRS2010

Evolution of Interplanetary Shocks and their CME Drivers

Shock-driving coronal mass ejections (CMEs) constitute the most energetic phenomena in the heliosphere. The shocks can be identified in a number of ways based on remote-sensing and in situ observations. Type II radio bursts are the earliest indicators of shocks that accelerate electrons to energies up to -10 keV. Solar energetic particle (SEP) events are always accompanied by long wavelength type II bursts indicating that the same shock accelerates ions and electrons. A recent investigation involving a large number of interplanetary (IP) shocks revealed that about 35% of them do not produce type II bursts (radio quiet, RQ) or SEPs. Comparison of the RQ shocks with the radio loud (RL) ones revealed some interesting results such as: (1) the lack of evidence for blast waves,(2) energetic particle enhancement in the shock front in -20% of RQ shocks, and (3) determination of the difference between the RQ and RL shocks in terms of the different kinematic properties of the associated CMEs. On the other hand the shock properties measured at I AU are not too different for the RQ and RL cases. This can be attributed to the interaction with the IP medium, which seems to erase the difference. Implications of this evolution for the geoeffectiveness is also discussed.

Gopalswamy, Natchimuthuk↗
NASA NTRS2010

Influences of the Driver and Ambient Medium Characteristics on the Formation of Shocks in the Solar Atmosphere

Traveling interplanetary (IP) shocks were discovered in the early 1960s, but their solar origin has been controversial. Early research focused on solar flares as the source of the shocks, but when coronal mass ejections (CMEs) were discovered, it became clear that fast CMEs clearly can drive the shocks. Type II radio bursts are excellent signatures of shocks near the Sun. The close correspondence between type II radio bursts and solar energetic particles (SEPs) makes it clear that the same shock accelerates ions and electrons. A recent investigation involving a large number of IP shocks revealed that about 35% of IP shocks do not produce type II bursts or SEPs. Comparing these radio quiet (RQ) shocks with the radio loud (RL) ones revealed some interesting results: (1) there is no evidence for blast waves, in that all IP shocks can be attributed to CMEs, (2) a small fraction (20%) of RQ shocks is associated with ion enhancements at the shocks when they move past the observing spacecraft, (3) the primary difference between the RQ and RL shocks can be traced to the different kinematic properties of the associated CMEs and the variation of the characteristic speeds of the ambient medium, and (4) the shock properties measured at 1 AU are not too different for the RQ and RL cases due to the interaction of the shock driver with the IP medium that seems to erase the difference.

Nat, Gopalswamy↗
NASA NTRS2010

Shock-Driving CMEs Near the Sun, in the Interplanetary Medium, and Near Earth

The excellent correspondence between type II radio bursts and solar energetic particles (SEPs) made it clear that the same shock accelerates ions and electrons. A recent investigation involving a large number of I shocks revealed that about 35% of IP shocks do not produce type II bursts (radio quiet) or SEPs. Comparing the RQ shocks with the radio loud (RL) ones revealed some interesting results, which will be summarized in this poster. (1) There is no evidence for blast waves. (2) Even a small fraction (20%) of RQ shocks is associated with ion enhancements at the shock when the shock passes the spacecraft. (3) The primary difference between the RQ and RL shocks can be traced to the different kinematic properties of the associated CMEs, although the shock properties measured at 1 AU are not too different for the RQ and RL cases. This can be attributed to the interaction with the IP medium, which seems to erase the difference. More details can be found in Astrophysical Journal 710, 1111, 2010 (http://adsabs.harvard.edu/absZ2009arXivO9l2.4719G).

Gopalswamy, N.↗
NASA NTRS2011

On Interplanetary Shocks Driven by Coronal Mass Ejections

Traveling interplanetary (IP) shocks were first detected in the early 1960s, but their solar origin has been controversial. Early research focused on solar flares as the source of the shocks, but when CMEs were discovered, it became clear that fast CMEs are the shock drivers. Type radio II bursts are excellent signatures of shocks near the Sun (Type II radio bursts were known long before the detection of shocks and CMEs). The excellent correspondence between type II bursts and solar energetic particle (SEP) events made it clear that the same shock accelerates ions and electrons. Shocks near the Sun are also seen occasionally in white-light coronagraphic images. In the solar wind, shocks are observed as discontinuities in plasma parameters such as density and speed. Energetic storm particle events and sudden commencement of geomagnetic storm are also indicators of shocks arriving at Earth. After an overview on these shock signatures, I will summarize the results of a recent investigation of a large number of IP shocks. The study revealed that about 35% of IP shocks do not produce type II bursts (radio quiet - RQ) or SEPs. Comparing the RQ shocks with the radio loud (RL) ones revealed some interesting results: (1) There is no evidence for blast wave shocks. (2) A small fraction (20%) of RQ shocks is associated with ion enhancements at the shock when the shock passes the spacecraft. (3) The primary difference between the RQ and RL shocks can be traced to the different kinematic properties of the associated CMEs. On the other hand the shock properties measured at 1 AU are not too different for the RQ and RL cases. This can be attributed to the interaction with the IP medium, which seems to erase the difference between the shocks.

Gopalswarmy, Nat↗
NASA NTRS2012

Coronal Mass Ejection-driven Shocks and the Associated Sudden Commencements-sudden Impulses

Interplanetary (IP) shocks are mainly responsible for the sudden compression of the magnetosphere, causing storm sudden commencement (SC) and sudden impulses (SIs) which are detected by ground-based magnetometers. On the basis of the list of 222 IP shocks compiled by Gopalswamy et al., we have investigated the dependence of SC/SIs amplitudes on the speed of the coronal mass ejections (CMEs) that drive the shocks near the Sun as well as in the interplanetary medium. We find that about 91% of the IP shocks were associated with SC/SIs. The average speed of the SC/SI-associated CMEs is 1015 km/s, which is almost a factor of 2 higher than the general CME speed. When the shocks were grouped according to their ability to produce type II radio burst in the interplanetary medium, we find that the radio-loud (RL) shocks produce a much larger SC/SI amplitude (average approx. 32 nT) compared to the radio-quiet (RQ) shocks (average approx. 19 nT). Clearly, RL shocks are more effective in producing SC/SIs than the RQ shocks. We also divided the IP shocks according to the type of IP counterpart of interplanetary CMEs (ICMEs): magnetic clouds (MCs) and nonmagnetic clouds. We find that the MC-associated shock speeds are better correlated with SC/SI amplitudes than those associated with non-MC ejecta. The SC/SI amplitudes are also higher for MCs than ejecta. Our results show that RL and RQ type of shocks are important parameters in producing the SC/SI amplitude.

Coronal mass ejections↗
NASA NTRS2021

Scheduling the NASA Deep Space Network with Deep Reinforcement Learning

With three complexes spread evenly across the Earth, NASA’s Deep Space Network (DSN) is the primary means of communications as well as a significant scientific instrument for dozens of active missions around the world. A rapidly rising number of spacecraft and increasingly complex scientific instruments with higher bandwidth requirements have resulted in demand that exceeds the network’s capacity across its 12 antennae. The existing DSN scheduling process operates on a rolling weekly basis and is time-consuming; for a given week, generation of the final baseline schedule of spacecraft tracking passes takes roughly 5 months from the initial requirements submission deadline, with several weeks of peer-to-peer negotiations in between. This paper proposes a deep reinforcement learning (RL) approach to generate candidate DSN schedules from mission requests and spacecraft ephemeris data with demonstrated capability to address real-world operational constraints. A deep RL agent is developed that takes mission requests for a given week as input, and interacts with a DSN scheduling environment to allocate tracks such that its reward signal is maximized. A comparison is made between an agent trained using Proximal Policy Optimization and its random, untrained counterpart. The results represent a proof-of-concept that, given a well-shaped reward signal, a deep RL agent can learn the complex heuristics used by experts to schedule the DSN. A trained agent can potentially be used to generate candidate schedules to bootstrap the scheduling process and thus reduce the turnaround cycle for DSN scheduling.

Wilson, Brian↗
NASA NTRS2022

SatNet: A Benchmark for Satellite Scheduling Optimization

Satellites provide essential services such as networking and weather tracking, and the number of near-earth and deep space satellites are expected to grow rapidly in the coming years. Communications with terrestrial ground stations is one of the critical functionalities of any space mission. Satellite scheduling is a problem that has been scientifically investigated since the 1970s. A central aspect of this problem is the need to consider resource contention and satellite visibility constraints as they require line of sight. Due to the combinatorial nature of the problem, prior solutions such as linear programs and evolutionary algorithms require extensive compute capabilities to output a feasible schedule for each scenario. Machine learning based scheduling can provide an alternative solution by training a model with historical data and generating a schedule quickly with model inference. We present SatNet, a benchmark for satellite scheduling optimization based on historical data from the NASA Deep Space Network. We propose formulation of the satellite scheduling problem as a Markov Decision Process and use reinforcement learning (RL) policies to generate schedules. The nature of constraints imposed by SatNet differ from other combinatorial optimization problems such as vehicle routing studied in prior literature. Our initial results indicate that RL is an alternative optimization approach that can generate candidate solutions of comparable quality to existing state-of-the-practice results. However, we also find that RL policies overfit to the training dataset and do not generalize well to new data, thereby necessitating continued research on reusable and generalizable agents.

Wilson, Brian↗
NASA NTRS1964

Photographic Study of Liquid-Oxygen Boiling and Gas Injection in the Injector of a Chugging Rocket Engine

High-speed motion pictures were taken of conditions in the injector liquid-oxygen cavity of an RL-10 rocket engine during throttled engine operation. Photographs were taken during operation of the engine in the chugging region as the helium gas was injected to stabilize combustion, during operation at rated thrust, and during transition into chugging conditions as the gas injection was discontinued. Results of the investigation indicate that, during chugging rocket operation of the RL-10 engine, a high population of fairly large bubbles formed and collapsed within the liquid-oxygen cavity at the same frequency as the chamber pressure oscillations. When gaseous helium was injected into the liquid-oxygen cavity, a fog rapidly spread over the entire field of view, and the system immediately became stable. The injection of gaseous helium at rated conditions produced a very slight increase in engine performance but not enough to produce a net gain in a typical mission payload with the extra equipment needed. The inherent low-frequency system instability associated with the fuel system at low thrust levels was reduced by injecting either gaseous helium or hydrogen. Complete stabilization was achieved in some cases, and a reduction in the severity of the oscillations in others. This was apparently due to the ·anchoring of the phase change front to the location of the gas injection.

Conrad, E. William↗
NASA NTRS1973

Design study of RL10 derivatives. Volume 2: Engine design characteristics

The design characteristics of the RL-10 rocket engine are discussed. The results from critical elements evaluation, baseline engine design, parametric and special study tasks are presented. Critical element evaluation established the feasibility of various engine features such as tank head idle, pumped idle, autogenous tank pressurization, and two-phase pumping. Three baseline engines, derived from the RL-10 were conceptually designed. Parametric life and performance data were generated. Special studies were conducted to establish the impact on the engine design of environment, safety, interchangeability, and maintenance.

Adams, A.↗
NASA NTRS1980

Orbit transfer vehicle engine study. Volume 2: Technical report

The orbit transfer vehicle (OTV) engine study provided parametric performance, engine programmatic, and cost data on the complete propulsive spectrum that is available for a variety of high energy, space maneuvering missions. Candidate OTV engines from the near term RL 10 (and its derivatives) to advanced high performance expander and staged combustion cycle engines were examined. The RL 10/RL 10 derivative performance, cost and schedule data were updated and provisions defined which would be necessary to accommodate extended low thrust operation. Parametric performance, weight, envelope, and cost data were generated for advanced expander and staged combustion OTV engine concepts. A prepoint design study was conducted to optimize thrust chamber geometry and cooling, engine cycle variations, and controls for an advanced expander engine. Operation at low thrust was defined for the advanced expander engine and the feasibility and design impact of kitting was investigated. An analysis of crew safety and mission reliability was conducted for both the staged combustion and advanced expander OTV engine candidates.

Source record↗