SEARCH · Engineering Papers

Results for “Reducing agent”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

NASA NTRS2003

Agent Based Software for the Autonomous Control of Formation Flying Spacecraft

Distributed satellite systems is an enabling technology for many future NASA/DoD earth and space science missions, such as MMS, MAXIM, Leonardo, and LISA [1, 2, 3]. While formation flying offers significant science benefits, to reduce the operating costs for these missions it will be essential that these multiple vehicles effectively act as a single spacecraft by performing coordinated observations. Autonomous guidance, navigation, and control as part of a coordinated fleet-autonomy is a key technology that will help accomplish this complex goal. This is no small task, as most current space missions require significant input from the ground for even relatively simple decisions such as thruster burns. Work for the NMP DS1 mission focused on the development of the New Millennium Remote Agent (NMRA) architecture for autonomous spacecraft control systems. NMRA integrates traditional real-time monitoring and control with components for constraint-based planning, robust multi-threaded execution, and model-based diagnosis and reconfiguration. The complexity of using an autonomous approach for space flight software was evident when most of its capabilities were stripped off prior to launch (although more capability was uplinked subsequently, and the resulting demonstration was very successful).

How, Jonathan P.↗

NASA NTRS2000

Structure, Hydrodynamics, and Phase Transition of Freely Suspended Liquid Crystals

Smectic liquid crystals are phases of rod shaped molecules organized into one dimensionally (1D) periodic arrays of layers, each layer being between one and two molecular lengths thick. In the least ordered smectic phases, the smectics A and C, each layer is a two dimensional (2D) liquid. Additionally there are a variety of more ordered smectic phases having hexatic short range translational order or 2D crystalline quasi long range translational order within the layers. The inherent fluid-layer structure and low vapor pressure of smectic liquid crystals enable the long term stabilization of freely suspended, single component, layered fluid films as thin as 30A, a single molecular layer. The layering forces the films to be an integral number of smectic layers thick, quantizing their thickness in layer units and forcing a film of a particular number of layers to be physically homogeneous with respect to its layer structure over its entire area. Optical reflectivity enables the precise determination of the number of layers. These ultrathin freely suspended liquid crystal films are structures of fundamental interest in condensed matter and fluid physics. They are the thinnest known stable condensed phase fluid structures and have the largest surface-to-volume ratio of any stable fluid preparation, making them ideal for the study of the effects of reduced dimensionality on phase behavior and on fluctuation and interface phenomena. Their low vapor pressure and quantized thickness enable the effective use of microgravity to extend the study of basic capillary phenomena to ultrathin fluid films. Freely suspended films have been a wellspring of new liquid crystal physics. They have been used to provide unique experimental conditions for the study of condensed phase transitions in two dimensions. They are the only system in which the hexatic has been unambiguously identified as a phase of matter, and the only physical system in which fluctuations of a 2D XY system and Kosterlitz Thouless phase transition has been observed and 2D XY quasi long range order verified. Smectic films have enabled the precise determination of smectic layer electron density and positional fluctuation profile and have been used to show that the interlayer interactions in anti-ferroelectric tilted smectics do not extend significantly beyond nearest neighbors. The interactions which are operative in liquid crystals are generally weak in comparison to those in crystalline phases, leading to the facile manipulation of the order in liquid crystals by external agents such as applied fields and surfaces. Effects arising from weak ordering are significantly enhanced in ultrathin free films and filaments wherein the intermolecular coupling is effectively reduced by loss of neighbors. Over the past four years this research, which we now detail, has produced a host of exciting new discoveries and unexpected results, maintaining the position of the study of freely suspended liquid crystal structures as one of most exciting and fruitful areas of complex fluid physics. In addition, several potentially interesting microgravity free film experiments have been identified.

Clark, Noel A.↗

NASA NTRS1979

Electrochemistry of lunar rocks

Electrolysis of silicate melts has been shown to be an effective means of producing metals from common silicate materials. No fluxing agents need be added to the melts. From solution in melts of diopside (CaMgSi2O6) composition, the elements Si, Ti, Ni, and Fe have been reduced to their metallic states. Platinum is a satisfactory anode material, but other cathode materials are needed. Electrolysis of compositional analogs of lunar rocks initially produces iron metal at the cathode and oxygen gas at the anode. Utilizing mainly heat and electricity which are readily available from sunlight, direct electrolysis is capable of producing useful metals from common feedstocks without the need for expendable chemicals. This simple process and the products obtained from it deserve further study for use in materials processing in space.

Lindstrom, D. J.↗

NASA NTRS2000

Autonomous Agents and Intelligent Assistants for Exploration Operations

Human exploration of space will involve remote autonomous crew and systems in long missions. Data to earth will be delayed and limited. Earth control centers will not receive continuous real-time telemetry data, and there will be communication round trips of up to one hour. There will be reduced human monitoring on the planet and earth. When crews are present on the planet, they will be occupied with other activities, and system management will be a low priority task. Earth control centers will use multi-tasking "night shift" and on-call specialists. A new project at Johnson Space Center is developing software to support teamwork between distributed human and software agents in future interplanetary work environments. The Engineering and Mission Operations Directorates at Johnson Space Center (JSC) are combining laboratories and expertise to carry out this project, by establishing a testbed for hWl1an centered design, development and evaluation of intelligent autonomous and assistant systems. Intelligent autonomous systems for managing systems on planetary bases will commuicate their knowledge to support distributed multi-agent mixed-initiative operations. Intelligent assistant agents will respond to events by developing briefings and responses according to instructions from human agents on earth and in space.

Malin, Jane T.↗

NASA NTRSDate not supplied

Intelligent Change Detection System: Autonomous Intelligent Machine Agent Model Development

NASA’s significant role in facilitating the harmonious integration of unmanned aircraft systems (UAS), with other aerial vehicles operating in the National Airspace System (NAS), has revealed a need for more advanced technological tools than are being utilized currently. This technology would lend itself to significantly assisting Direct-Action Aviation Personnel (DAAP) with the ingress and egress of UAS operations within the NAS. Providing research findings that would reduce technical barriers, associated with UAS-NAS integration, has been a persistent effort by both NASA and the FAA. One such research effort, pursued by NASA’s Transformative Tools and Technologies – Revolutionary Aviation Mobility (T3-RAM) project, is the development of an autonomous intelligent machine (AIM) agent that would aid DAAP functioning as implemented in remote ground control stations (RGCS). This evolution of the “human-machine” symbiosis, within the aviation environment, is necessary for many reasons. For example, there are projections of large increases to the 864,000 registered UAS and 45,000 aviation operations taking place in the NAS each day. With a data output range from 1 to 20 terabytes per flight or each aerial vehicle, which is projected to have proportional rate increase to that of registered UASs. It is evident, that due to the projected increase of UASs and their generated data, the human-agent’s data managing capabilities will be quickly overwhelmed by the enormous amounts of data emanating in the NAS. The research efforts presented in this paper puts forward results from the development, assessment, and verification of a previously conceptualized AIM-Agent that combats actionable-data (information) errors resulting from the visual perception phenomenon known as “change blindness” (CB). CB has been identified as one of the main culprits of information erroring encountered within the ground control station operator (GCSO) community.

Change Blindness↗

NASA NTRS2019

Spatiotemporal Monitoring of Hydrilla [Hydrilla verticillata (L. f.) Royle] to Aid Management Actions

Hydrilla is an invasive aquatic plant that has rapidly spread through many inland water bodies across the globe by outcompeting native aquatic plants. Drinking water supply, power generation, and recreational use of lakes are threatened by the spread of this invasive species. Moreover, hydrilla serves as a host for an epiphytic, toxic cyanobacteria known to be the causative agent of the neurodegenerative disease called avian vacuolar myelinopathy, which affects waterfowl, raptors, and amphibians. The early detection of hydrilla infestation is very important to reduce the costs associated with control and removal efforts of this invasive species. Therefore, this study was aimed at developing a rapid, frequent, and large-scale tool for monitoring and predicting spatial extent of hydrilla habitat. This was achieved by integrating in situ and Landsat 8 OLI satellite data for Lake J. Strom Thurmond, the largest US Army Corps of Engineers Lake east of the Mississippi River located on the Georgia/South Caroline border. The predictive model incorporated remote sensing reflectance, Secchi disk depth (SDD), light attenuation coefficient (Kd), maximum depth of colonization (Zc), and percentage of light available through water column (PLW). The model predicted ideal habitat for hydrilla featuring high SDD, low values of Kd, high Zc and PLW. Final analysis showed that hydrilla starts growing in April, reaches peak around October, begins retreating in the following months, and disappears in February. Additionally, physical meteorological factors were analyzed and were found to be closely associated with hydrilla extent. Management agencies can use these results to not only to plan future removal efforts, but also evaluate and adapt their current mitigation efforts.

Percentage of light available through the water co↗

NASA NTRSDate not supplied

Towards Autonomous Lunar Resource Excavation via Reinforcement Learning

To continue on a sustainable and flexible path, NASA needs to address the challenge of collecting and moving large amounts of regolith at the destination. NASA’s Regolith Advanced Surface Systems Operations Robot (RASSOR) is principally designed to mine and deliver regolith for In-Situ Resource Utilization (ISRU) processing. RASSOR’s design enables it to efficiently collect and deposit regolith, return collected material for processing, and myriad related ISRU activities. To reliably perform these operations on the lunar surface, RASSOR software and sensory systems need to be robust and maximize the information extracted from a reduced sensor payload. Herein, we present preliminary findings from the Intelligent Capabilities Enhanced RASSOR project. We created reduced-order simulation environments to develop autonomous trenching controllers via reinforcement learning and prototype state estimation architectures. The goal of reinforcement learning is for an agent to learn a policy (task strategy) through interactions with an environment. When the agent performs an action, a change occurs in environment state and a numerical reward is received which informs the agent whether the action performed was good or not. Since reinforcement learning algorithms learn through trial-and-error, a simulation is a desirable first environment for development and learning. We developed two simulations, the first is a 2D excavation simulation developed to facilitate parameter selection, and a 3D simulation developed using a game physics engine, to simulate simplified soil interactions and increase the fidelity of the dynamic models of the robotic agents. The development of this 3D simulation has enabled the training of additional sensing capabilities and research both at the granular mechanics and operations levels. We experimented with various virtual sensor payloads to identify a combination that enabled efficient excavation operation and learning. Our reward function is based on how much material is excavated per step. A penalty is also received for leaving the dig site and to smooth the acceleration of the drum arms. We implemented pseudo time-of-flight sensors to report distance from each drum to ground and the height above ground which was found to be more efficient than existing solutions. Our findings suggest that reinforcement learning for autonomous operations has learned viable trenching strategies within 3000 training episodes in our simplified 2D environment and helped identify desirable sensing capabilities, arrangements, and considerations such as the positioning of time-of-flight sensors. Future work includes expanding our simulation to more complex environments and scenarios, and transfer learning from simulation to RASSOR 2.0 hardware for deployment in the Regolith Test Bin at NASA's Kennedy Space Center.

rassor↗

NASA NTRS2004

Structure and Dynamics of Freely Suspended Liquid Crystals

Smectic liquid crystals are phases of rod shaped molecules organized into one dimensionally (1 D) periodic arrays of layers, each layer being between one and two molecular lengths thick. In the least ordered smectic phases, the smectics A and C, each layer is a two dimensional (2D) liquid. Additionally there are a variety of more ordered smectic phases having hexatic short range translational order or 2D crystalline or quasi long range translational order within the layers. The inherent fluid-layer structure and low vapor pressure of smectic liquid crystals enables the long term stabilization of freely suspended, single component, layered fluid films as thin as 30A, a single molecular layer. The layering forces the films to be an integral number of smectic layers thick, quantizing their thickness in layer units and forcing a film of a particular number of layers to be physically homogeneous with respect to its layer structure over its entire area. Optical reflectivity enables the precise determination of the number of layers. These ultrathin freely suspended liquid crystal films are structures of fundamental interest in condensed matter and fluid physics. They are the thinnest known stable fluid structures and have the largest surface-to-volume ratio of any stable fluid preparation, making them ideal for the study of the effects of reduced dimensionality on phase behavior and on fluctuation and interface phenomena. Their low vapor pressure and quantized thickness enable the effective use of microgravity to extend the study of basic capillary phenomena to ultrathin fluid films. Freely suspended films have been a wellspring of new LC physics. They have been used to provide unique experimental conditions for the study of condensed phase transitions in two dimensions. They are the only system in which the hexatic has been unambiguously identified as a phase of matter, and the only physical system in which fluctuations of a 2D XY system and Kosterlitz Thouless phase transition has been observed and 2D XY quasi long range order verified. Smectic films have enabled the precise determination of smectic layer electron density and positional fluctuation profiles and have been used to show that the interlayer interactions in antiferroelectric tilted smectics do not extend significantly beyond nearest neighbors. Freely suspended films played a pivotal role in the recent discovery of macroscopic chiral-polar ordering in fluids of achiral molecules. The interactions which are operative in liquid crystals are generally weak in comparison to those in crystalline phases, leading to the facile manipulation of the order in liquid crystals by external agents such as applied fields and surfaces. Effects arising from weak ordering are significantly enhanced in ultrathin free films and filaments, in which the intermolecular coupling is effectively further reduced by loss of neighbors. Over the past four years this research, which we now detail, has produced a host of exciting new discoveries and unexpected results, maintaining the study of freely suspended liquid crystal structures as one of most exciting and fruitful areas of complex fluid physics. In addition, a class of experiments on the behavior of 1D interfaces in 2D films have been pursued with results that point to potentially quite interesting effects in microgravity.

Clark, Noel A.↗

NASA NTRS2017

Flight Deck Surface Trajectory-Based Operations (STBO): A Four-Dimensional Trajectory (4DT) Simulation

Within human factors there is burgeoning interest in the Human-Autonomy Teaming (HAT) concept as away to address the challenges of interacting with complex, increasingly autonomous systems. The HAT concept comes out of an aspiration to interact with increasingly autonomous automation as a team member, rather than simply use automation as a tool. The authors, and others, have proposed core tenets for HAT that include bi-directional communication, automation and system transparency, and advanced coordination between human and automated teammates via predefined, dynamic task sequences known as plays (Shively et al., 2017). It is believed that, with proper implementation, HAT should foster appropriate teamwork, thus increasing trust and reliance on the system, which in turn will reduce workload, increase situation awareness, and improve performance. To this end, HAT has been demonstrated and/or studied in multiple applications including search and rescue operations (Nourbakhsh et al., 2005), healthcare and medicine (Tsui Yanco, 2007), autonomous vehicles (Parasuraman, Barnes, Cosenzo, Mulgund, 2007), photography (Lachter, Brandt, Sadler, Shively, in press), and aviation (Shively et al., in press). The current paper presents one such effort to apply HAT. It details the design of a R-HAT Agent developed as part of a NASA Research Agreement awarded to Human-Autonomy Teaming Solutions Inc. (HATS Inc), and developed in collaboration with the Human-Autonomy Teaming Laboratory at NASA Ames Research Center. The role of this Agent is to mediate interaction between the automation and the human operator of an advanced ground dispatch station, with this mediation based upon previously mentioned core tenets for HAT and the many lessons learned from the HAT research literature. This dispatch station was developed to support a NASA project investigating a concept called Reduced Crew Operations (RCO; Lachter, Brandt, Battiste, Matessa, Johnson, in press). Part of the RCO concept involves a ground operator providing enhanced support to a large number of aircraft with a single pilot on the flight deck. When assisted by the Agent, operators can monitor and support or manage a large number of aircraft and use plays to respond in real-time to complicated, workload-intensive events (e.g., an airport closure). A play is a plan that encapsulates goals, tasks, and a task allocation strategy appropriate for a particular situation. In the current implementation, when a play is initiated by a user, the Agent determines what tasks need to be done and has the ability to autonomously execute them (e.g., determining diversion options and uplinking new routes to aircraft) when it is safe and appropriate. The R-HAT Agent has been designed to both support end users and research in RCO and HAT. Additionally, the Agent and its underlying architecture were developed with generalizability in mind as a modular piece of software applicable outside of RCO aviation in domains such as those mentioned above. This paper will also discuss future further development and testing of the R-HAT Agent.

Bakowski, Deborah L.↗

NASA NTRS1996

Dextroamphetamine: a pharmacologic countermeasure for space motion sickness and orthostatic dysfunction

Dextroamphetamine has potential as a pharmacologic agent for the alleviation of two common health effects associated with microgravity. As an adjuvant to Space Motion Sickness (SMS) medication, dextroamphetamine can enhance treatment efficacy by reducing undesirable Central Nervous System (CNS) side effects of SMS medications. Secondly, dextroamphetamine may be useful for the prevention of symptoms of post-mission orthostatic intolerance caused by cardiovascular deconditioning during spaceflight. There is interest in developing an intranasal delivery form of dextroamphetanmine for use as a countermeasure in microgravity conditions. Development of this dosage form will require an analytical detection method with sensitivity in the low ng range (1 to 100 ng/mL). During the 1995 Summer Faculty Fellowship Program, two analytical methods were developed and evaluated for their suitability as quantitative procedures for dextroamphetamine in studies of product stability, bioavailability assessment, and pharmacokinetic evaluation. In developing some of the analytical methods, beta-phenylethylamine, a primary amine structurally similar to dextroamphetamine, was used. The first analytical procedure to be evaluated involved hexane extraction and subsequent fluorescamine labeling of beta-phenylethylamine. The second analytical procedure to be evaluated involved quantitation of dextroamphetamine by an Enzyme-Linked ImmunoSorbent Assay (ELISA).

Snow, L. Dale↗

NASA NTRS1985

Particle size distributions of several commonly used seeding aerosols

During the course of experimentation, no solid particle powder could be found which produced an aerosol with a narrow particle size distribution when fluidization was the only flow process used in producing the aerosol. The complication of adding particle size fractionation processes to the aerosol generation effort appears to be avoidable. In this regard, a simple sonic orifice is found to be effective in reducing the percentage of agglomerates in the several metal oxide powders tested. Marginally beneficial results are obtained for a 0.5/99.5 percent by weight mixture of the flow agent and metal oxide powder. However, agglomeration is observed to be enhanced when the flow agent percentage is increased to 5 percent. Liquid atomization using the Collison nebulizer as well as a version of the Laskin nozzle resulted in polydispersed aerosols with particle size distributions heavily weighted by the small particle end of the size spectrum. The aerosol particle size distributions produced by the vaporization/condensation seeder are closer to the ideal monodispersed aerosol than any of the other aerosols tested. In addition, this seeding approach affords a measure of control over particle size and particle production rate.

Crosswy, F. L.↗

NASA NTRS2005

Trusted Autonomy for Space Flight Systems

NASA has long supported research on intelligent control technologies that could allow space systems to operate autonomously or with reduced human supervision. Proposed uses range from automated control of entire space vehicles to mobile robots that assist or substitute for astronauts to vehicle systems such as life support that interact with other systems in complex ways and require constant vigilance. The potential for pervasive use of such technology to extend the kinds of missions that are possible in practice is well understood, as is its potential to radically improve the robustness, safety and productivity of diverse mission systems. Despite its acknowledged potential, intelligent control capabilities are rarely used in space flight systems. Perhaps the most famous example of intelligent control on a spacecraft is the Remote Agent system flown on the Deep Space One mission (1998 - 2001). However, even in this case, the role of the intelligent control element, originally intended to have full control of the spacecraft for the duration of the mission, was reduced to having partial control for a two-week non-critical period. Even this level of mission acceptance was exceptional. In most cases, mission managers consider intelligent control systems an unacceptable source of risk and elect not to fly them. Overall, the technology is not trusted. From the standpoint of those who need to decide whether to incorporate this technology, lack of trust is easy to understand. Intelligent high-level control means allowing software io make decisions that are too complex for conventional software. The decision-making behavior of these systems is often hard to understand and inspect, and thus hard to evaluate. Moreover, such software is typically designed and implemented either as a research product or custom-built for a particular mission. In the former case, software quality is unlikely to be adequate for flight qualification and the functionality provided by the system is likely driven largely by the need to publish innovative work. In the latter case, the mission represents the first use of the system, a risky proposition even for relatively simple software.

Freed, Michael↗

NASA NTRS1975

Effects of pharmacological agents on subcortical resistance shifts

Microliter quantities of tetrodotoxin, tetraethylammonium chloride, and picrotoxin injected into the inferior colliculus and superior olive of unanesthetized cats differentially affect the amplitude and waveform of click-evoked potentials and evoked resistance shifts. Tetrodotoxin simultaneously reduces the negative phase of the evoked potential and eliminates the evoked resistance shift. Tetraethylammonium enhances the negative evoked potential component, presumably of postsynaptic origin, without significantly altering evoked resistance shift amplitude. Picrotoxin also enhances the negative evoked potential wave but increases evoked resistance shift amplitude. These findings implicate events associated with postsynaptic membrane depolarization in the production of the evoked resistance shift.

Klivington, K. A.↗

NASA NTRS1972

Contamination threats to critical surfaces from handling and storage practices.

Review of the procedures and results of a program designed to remove the threat of sources of molecular and particulate contamination of critical optical, electrical, and mechanical elements in spacecraft. The results of recent contamination-probing thermal-vacuum tests indicate that some of the materials and practices commonly used to protect critical surfaces from molecular or particulate contamination can themselves represent significant threats. These contamination sources include clean-room and clean-tent materials, gloves, tissues, and covering or packaging materials. Mass and infrared spectral analyses of these materials and the environments and instruments exposed to them show that the contaminants are mostly plasticizers, slip or antistatic agents, and binders used in the manufacture of these products. Products of particular threat include vinyl gloves, boots, clean-tent walls, and some polyethylene sheets and bags. Techniques for reducing these threats are discussed.

Poehlmann, H. C.↗

NASA NTRS1998

In-Vitro Induced Immunosuppression in a Rotary Cell Culture System

The function of the innate immune system is to provide a first-line of defense against infectious organisms, via control of bacterial and viral growth using antigen nonspecific means. These nonspecific immune effectors include macrophages and Natural Killing (NK) cells, and certain cytokines elicited in response to "super antigens" on the infectious agents. This innate system usually keeps most infectious agents from rapidly growing while the adaptive immune system is generating a specific response complete with immunologic memory. Compelling evidence suggests that space flight results in various immunosuppressive effects, including reduced innate and adaptive immune responses. We were particularly concerned with reduced NK activity at landing, and have asked whether the microgravity component of space flight could be responsible for the previously observed NK defect. We have conclusively demonstrated that simulated microgravity as provided by the Synthecon bioreactors does not inhibit the NK function nor the IL-2 activation of lymphokine-activated killing (LAK). Interleukin-2 is the key cytokine responsible for activation of NK cells to express LAK, as well as to support differentiation of lymphocytes during adaptive immune responses. Therefore, we have disproved our original hypothesis based on poor NK in many of the astronauts upon landing.

Grimm, Elizabeth A.↗

NASA NTRS2005

Multi-Scale Modeling of Cross-Linked Nanotube Materials

The effect of cross-linking single-walled carbon nanotubes on the Young's modulus of a nanotube-reinforced composite is modeled with a multi-scale method. The Young's modulus is predicted as a function of nanotube volume fraction and cross-link density. In this method, the constitutive properties of molecular representative volume elements are determined using molecular dynamics simulation and equivalent-continuum modeling. The Young's modulus is subsequently calculated for cross-linked nanotubes in a matrix which consists of the unreacted cross-linking agent. Two different cross-linking agents are used in this study, one that is short and rigid (Molecule A), and one that is long and flexible (Molecule B). Direct comparisons between the predicted elastic constants are made for the models in which the nanotubes are either covalently bonded or not chemically bonded to the cross-linking agent. At a nanotube volume fraction of 10%, the Young's modulus of Material A is not affected by nanotube crosslinking, while the Young's modulus of Material B is reduced by 64% when the nanotubes are cross-linked relative to the non-cross-linked material with the same matrix.

Frankland, S. J. V.↗

NASA NTRS1998

Preliminary Work for Examining the Scalability of Reinforcement Learning

Researchers began studying automated agents that learn to perform multiple-step tasks early in the history of artificial intelligence (Samuel, 1963; Samuel, 1967; Waterman, 1970; Fikes, Hart & Nilsonn, 1972). Multiple-step tasks are tasks that can only be solved via a sequence of decisions, such as control problems, robotics problems, classic problem-solving, and game-playing. The objective of agents attempting to learn such tasks is to use the resources they have available in order to become more proficient at the tasks. In particular, each agent attempts to develop a good policy, a mapping from states to actions, that allows it to select actions that optimize a measure of its performance on the task; for example, reducing the number of steps necessary to complete the task successfully. Our study focuses on reinforcement learning, a set of learning techniques where the learner performs trial-and-error experiments in the task and adapts its policy based on the outcome of those experiments. Much of the work in reinforcement learning has focused on a particular, simple representation, where every problem state is represented explicitly in a table, and associated with each state are the actions that can be chosen in that state. A major advantage of this table lookup representation is that one can prove that certain reinforcement learning techniques will develop an optimal policy for the current task. The drawback is that the representation limits the application of reinforcement learning to multiple-step tasks with relatively small state-spaces. There has been a little theoretical work that proves that convergence to optimal solutions can be obtained when using generalization structures, but the structures are quite simple. The theory says little about complex structures, such as multi-layer, feedforward artificial neural networks (Rumelhart & McClelland, 1986), but empirical results indicate that the use of reinforcement learning with such structures is promising. These empirical results make no theoretical claims, nor compare the policies produced to optimal policies. A goal of our work is to be able to make the comparison between an optimal policy and one stored in an artificial neural network. A difficulty of performing such a study is finding a multiple-step task that is small enough that one can find an optimal policy using table lookup, yet large enough that, for practical purposes, an artificial neural network is really required. We have identified a limited form of the game OTHELLO as satisfying these requirements. The work we report here is in the very preliminary stages of research, but this paper provides background for the problem being studied and a description of our initial approach to examining the problem. In the remainder of this paper, we first describe reinforcement learning in more detail. Next, we present the game OTHELLO. Finally we argue that a restricted form of the game meets the requirements of our study, and describe our preliminary approach to finding an optimal solution to the problem.

Clouse, Jeff↗

NASA NTRS2003

An Effective Division of Labor Between Human and Robotic Agents Performing a Cooperative Assembly Task

NASA's Human Space Flight program depends heavily on spacewalks performed by human astronauts. These so-called extra-vehicular activities (EVAs) are risky, expensive and complex. Work is underway to develop a robotic astronaut's assistant that can help reduce human EVA time and workload by delivering human-like dexterous manipulation capabilities to any EVA worksite. An experiment is conducted to evaluate human-robot teaming strategies in the context of a simplified EVA assembly task in which Robonaut, a collaborative effort with the Defense Advanced Research Projects Agency (DARPA), an anthropomorphic robot works side-by-side with a human subject. Team performance is studied in an effort to identify the strengths and weaknesses of each teaming configuration and to recommend an appropriate division of labor. A shared control approach is developed to take advantage of the complementary strengths of the human teleoperator and robot, even in the presence of significant time delay.

Rehnmark, Fredrik↗