Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

Large-Scale Traveling Weather Systems in Mars Southern Extratropics

Between late fall and early spring, Mars' middle- and high-latitude atmosphere supports strong mean equator-to-pole temperature contrasts and an accompanying mean westerly polar vortex. Observations from both the MGS Thermal Emission Spectrometer (TES) and the MRO Mars Climate Sounder (MCS) indicate that a mean baroclinicity-barotropicity supports intense, large-scale eastward traveling weather systems (i.e., transient synoptic-period waves). Such extratropical weather disturbances are critical components of the global circulation as they serve as agents in the transport of heat and momentum, and generalized scalar/tracer quantities (e.g., atmospheric dust, water-vapor and ice clouds). The character of such traveling extratropical synoptic disturbances in Mars' southern hemisphere during late winter through early spring is investigated using a moderately high-resolution Mars global climate model (Mars GCM). This Mars GCM imposes interactively-lifted and radiatively-active dust based on a threshold value of the surface stress. The model exhibits a reasonable "dust cycle" (i.e., globally averaged, a dustier atmosphere during southern spring and summer occurs). Compared to the northern-hemisphere counterparts, the southern synoptic-period weather disturbances and accompanying frontal waves have smaller meridional and zonal scales, and are far less intense. Influences of the zonally asymmetric (i.e., east-west varying) topography on southern large-scale weather are investigated, in addition to large-scale up-slope/down-slope flows and the diurnal cycle. A southern storm zone in late winter and early spring presents in the western hemisphere via orographic influences from the Tharsis highlands, and the Argyre and Hellas impact basins. Geographically localized transient-wave activity diagnostics are constructed that illuminate dynamical differences amongst the simulations and these are presented.

Hollingsworth, Jeffery L.↗

Autonomous Power Controller for the NASA Gateway

Intelligent autonomous control of a spacecraft is an enabling technology that must be developed for deep space human exploration. NASA's current long term human space platform, the International Space Station which is in Low Earth Orbit, is in almost continuous communication with ground based mission control. This allows near real-time operation of all the vehicle core systems, including power, from the ground. As the focus shifts from Low Earth Orbit to deep space, challenges associated with communication time-lag and bandwidth limitations beyond geosynchronous orbit do not permit this type of ground based operation. These communication limitations motivate autonomous vehicle operations, including the vehicle subsystems such as power. This presentation will describe the ongoing development of an Autonomous Power Control (APC) system that can be used for a deep space exploration spacecraft. This work extends previous work on developing an autonomous power control which includes developing a control architecture for deep space vehicles, the use of software agents, and constructing a control simulation lab for demonstrating this capability. This presentation will begin with a discussion of the representative future power architectures that will be required for deep space exploration vehicles. The presentation then will describe how the power controller will integrate with the vehicle followed with a discussion of the autonomous power controller. Next it will describe the test setup used to evaluate the performance of the system and lastly show some of the test results. To develop the type of controller envisioned, it will be necessary to employ a detailed real-time simulation to evaluate its performance and ultimately verify its functionality.

Csank, Jeffrey↗

Autonomous Power Control

Intelligent autonomous control of a spacecraft is an enabling technology that must be developed for deep space human exploration. NASA's current long term human space platform, the International Space Station which is in Low Earth Orbit, is in almost continuous communication with ground based mission control. This allows near realtime operation of all the vehicle core systems, including power, from the ground. As the focus shifts from Low Earth Orbit to deep space, challenges associated with communication time-lag and bandwidth limitations beyond geosynchronous orbit do not permit this type of ground based operation. These communication limitations motivate autonomous vehicle operations, including the vehicle subsystems such as power. This presentation will describe the ongoing development of an Autonomous Power Control (APC) system that can be used for a deep space exploration spacecraft. This work extends previous work on developing an autonomous power control which includes developing a control architecture for deep space vehicles, the use of software agents, and constructing a control simulation lab for demonstrating this capability. This presentation will begin with a discussion of the representative future power architectures that will be required for deep space exploration vehicles. The presentation then will describe how the power controller will integrate with the vehicle followed with a discussion of the autonomous power controller. Next it will describe the test setup used to evaluate the performance of the system and lastly show some of the test results. To develop the type of controller envisioned, it will be necessary to employ a detailed real-time simulation to evaluate its performance and ultimately verify its functionality.

Csank, Jeffrey↗

Evolving Relationship between Humans and Machines

Traditional design typically consists of a master-servant relationship between humans and machines where the human directly controls what the machine will do and when it will do it through an interface. The current archetypical path encompasses moving from informational displays, where the human directly controls the machine based on information displayed, to automation where the human still directs the machine that then caries out the request using predefined set of instructions. Rapid pace of technological advancement makes it possible now, or in a near future, for machines to reach a level of intelligence that enables for systems to execute tasks/missions without predefined specific instructions; thus attaining a status of non- human autonomous agents. Now the course of human-machine interface technology changes from an information system to automation to an autonomous agent—essentially moving from a master-servant relationship to teammates. This paper discusses these changing relationships and challenges associated with progressing from a master-servant relationship with technology to more of an equal teammate. Examples of this progression includes current work encompassing rotorcraft noise minimization for urban air mobility.

Trujillo, Anna C.↗

A Unified Approach to Model-Based Planning and Execution

Writing autonomous software is complex, requiring the coordination of functionally and technologically diverse software modules. System and mission engineers must rely on specialists familiar with the different software modules to translate requirements into application software. Also, each module often encodes the same requirement in different forms. The results are high costs and reduced reliability due to the difficulty of tracking discrepancies in these encodings. In this paper we describe a unified approach to planning and execution that we believe provides a unified representational and computational framework for an autonomous agent. We identify the four main components whose interplay provides the basis for the agent's autonomous behavior: the domain model, the plan database, the plan running module, and the planner modules. This representational and problem solving approach can be applied at all levels of the architecture of a complex agent, such as Remote Agent. In the rest of the paper we briefly describe the Remote Agent architecture. The new agent architecture proposed here aims at achieving the full Remote Agent functionality. We then give the fundamental ideas behind the new agent architecture and point out some implication of the structure of the architecture, mainly in the area of reactivity and interaction between reactive and deliberative decision making. We conclude with related work and current status.

Muscettola, Nicola↗

The atmospheric structure and dynamical properties of Neptune derived from ground-based and IUE spectrophotometry

A wide range of recent full-disk spectral observations is used to constrain the atmospheric structure and dynamical properties of Neptune; analytical determinations are made of the abundances of such spectrally active gas species as the deep-atmosphere CH4 molar fraction and the mean ortho/para hydrogen ratio in the visible atmosphere, as well as stratospheric and tropospheric aerosol properties. Compared to Uranus, the greater abundance and shorter lifetimes of Neptunian particulates in the stratospheric region irradiated by the solar UV flux indicate that such radiation is the darkening agent of stratospheric aerosols on both planets.

Baines, Kevin H.↗

Pan-cancer proteogenomics characterization of tumor immunity

Despite the successes of immunotherapy in cancer treatment over recent decades, less than <10%–20% cancer cases have demonstrated durable responses from immune checkpoint blockade. To enhance the efficacy of immunotherapies, combination therapies suppressing multiple immune evasion mechanisms are increasingly contemplated. To better understand immune cell surveillance and diverse immune evasion responses in tumor tissues, we comprehensively characterized the immune landscape of more than 1,000 tumors across ten different cancers using CPTAC pan-cancer proteogenomic data. We identified seven distinct immune subtypes based on integrative learning of cell type compositions and pathway activities. We then thoroughly categorized unique genomic, epigenetic, transcriptomic, and proteomic changes associated with each subtype. Further leveraging the deep phosphoproteomic data, we studied kinase activities in different immune subtypes, which revealed potential subtype-specific therapeutic targets. Insights from this work will facilitate the development of future immunotherapy strategies and enhance precision targeting with existing agents.

60 APPLIED LIFE SCIENCES↗

Production of oxygen from lunar ilmenite

The overall objective of this project was to develop a novel carbothermal reduction process for production of oxygen from lunar ilmenite. The specific objective was to use a reaction sequence in which a wide variety of carbonaceous compounds (including carbonaceous wastes) can be used as reducing agents. During the first phase, two reactor systems were designed, constructed, and operated to study the reaction fundamental important in this process. One system is a small fluidized bed, and the other is a thermo-gravimetric reactor system. Preliminary experiments on synthetic ilmenite are conducted to study the effect of carbon type, carbon loading, temperature, and gas flow rate. Results indicate that a reaction path based on carbon gasification can be used to promote the overall kinetics. A unique temperature and concentration-programmed reaction procedure was being developed for rapid parametric study of the process.

Shadman, F.↗

Biosensors for Real-Time Monitoring of Radiation-Induced Biologic Effects in Space

This work seeks to develop cellular biosensors based on dendritic polymers. Nanoscale polymer structures less than 20 nm in diameter will be used as the basis of the biosensors. The structures will be designed to target into specific cells of an astronaut and be able to monitor health issues such as exposure to radiation. Multiple components can be assembled on the polymers including target directors, analytical devices (such as molecular probes), and reporting agents. The reporting will be accomplished through fluorescence signal monitoring, with the use of multispectral analysis for signal interpretation. These nanosensors could facilitate the success and increase the safety of extended space flight. The design and assembly of these devices has been pioneered at the Center for Biologic Nanotechnology in the University of Michigan. This period, synthesis of the test-bed biosensors continued. Studies were performed on the candidate fluorescent dyes to determine which might be suitable for the biosensor under development. Development continued on producing an artificial capillary bed as a tool for the use in the production of the fluorescence signal monitor. Work was also done on the in vitro multispectral analysis system, which uses the robotic microscope.

Baker, James R.↗

Human-Centric Teaming in a Multi-Agent EVA Assembly Task

NASA's Human Space Flight program depends heavily on spacewalks performed by pairs of suited human astronauts. These Extra-Vehicular Activities (EVAs) are severely restricted in both duration and scope by consumables and available manpower.An expanded multi-agent EVA team combining the information-gathering and problem-solving skills of human astronauts with the survivability and physical capabilities of highly dexterous space robots is proposed. A 1-g test featuring two NASA/DARPA Robonaut systems working side-by-side with a suited human subject is conducted to evaluate human-robot teaming strategies in the context of a simulated EVA assembly task based on the STS-61B ACCESS flight experiment.

Rehnmark, Fredrik↗

From Goal-Oriented Requirements to Event-B Specifications

In goal-oriented requirements engineering methodologies, goals are structured into refinement trees from high-level system-wide goals down to fine-grained requirements assigned to specific software/ hardware/human agents that can realise them. Functional goals assigned to software agents need to be operationalised into specification of services that the agent should provide to realise those requirements. In this paper, we propose an approach for operationalising requirements into specifications expressed in the Event-B formalism. Our approach has the benefit of aiding software designers by bridging the gap between declarative requirements and operational system specifications in a rigorous manner, enabling powerful correctness proofs and allowing further refinements down to the implementation level. Our solution is based on verifying that a consistent Event-B machine exhibits properties corresponding to requirements.

Aziz, Benjamin↗

Practical benzylation of N,N -substituted ethanolamines related to chemical warfare agents for analysis and detection by Electron Ionization Gas Chromatography-Mass Spectrometry

The benzylation of three, low-molecular weight N,N-disubstituted ethanolamines related to chemical warfare agents (CWAs) to furnish derivatives with improved Gas Chromatography-Mass Spectrometry (GC-MS) profiles is described. Due to their low molecular weight and polar nature, N,N-disubstituted ethanolamines are notoriously difficult to detect by routine GC-MS analyses during Organisation for the Prohibition of Chemical Weapons (OPCW) proficiency tests (PTs), particularly in scenarios when they are present at low levels (~1-10 ppm) amidst more abundant interferences. Our studies revealed that the optimal derivatization conditions involved the treatment of the ethanolamine with benzyl bromide in the presence of an inorganic base (e.g., Na 2 CO 3 ) in dichloromethane at 55 °C for 2 hours. This optimized set of conditions were then successfully applied to the derivatization of N,N-dimethylethanolamine, N,N-diethylethanolamine and N,N-diisopropylethanolamine present separately at 1 and 10 μg/mL concentrations in a glycerol-rich matrix sample featured in the 48 th OPCW PT. The benzylated derivatives of the three ethanolamines possessed retention times long enough to clear the massive glycerol-containing matrix interferences. In conclusion, the protocol herein is introduced as an alternative method for derivatization of these CWA and pharmaceutically important species and should find broad applicability in laboratories where their routine forensic analysis is carried out.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Reinforcement Learning for feedback-enabled cyber resilience

The rapid growth in the number of devices and their connectivity has enlarged the attack surface and made cyber systems more vulnerable. As attackers become increasingly sophisticated and resourceful, mere reliance on traditional cyber protection, such as intrusion detection, firewalls, and encryption, is insufficient to secure the cyber systems. Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and uncertainties in real-time and strategically responds to them to maintain the critical functions of the cyber systems in the event of successful attacks. Feedback architectures play a pivotal role in enabling the online sensing, reasoning, and actuation process of the CRM. Reinforcement Learning (RL) is an important gathering of algorithms that epitomize the feedback architectures for cyber resilience. It allows the CRM to provide dynamic and sequential responses to attacks with limited or without prior knowledge of the environment and the attacker. In this work, we review the literature on RL for cyber resilience and discuss the cyber-resilient defenses against three major types of vulnerabilities, i.e., posture-related, information-related, and human-related vulnerabilities. Here we introduce moving target defense, defensive cyber deception, and assistive human security technologies as three application domains of CRMs to elaborate on their designs. The RL algorithms also have vulnerabilities themselves. We explain the major vulnerabilities of RL and present develop several attack models where the attacker target the information exchanged between the environment and the agent: the rewards, the state observations, and the action commands. We show that the attacker can trick the RL agent into learning a nefarious policy with minimum attacking effort. The paper introduces several defense methods to secure the RL-enabled systems from these attacks. However, there is still a lack of works that focuses on the defensive mechanisms for RL-enabled systems. Last but not least, we discuss the future challenges of RL for cyber security and resilience and emerging applications of RL-based CRMs.

97 MATHEMATICS AND COMPUTING↗

Tunable and Cooperative Thermomechanical Properties of Protein-Metal-Organic Frameworks

We recently introduced protein-metal-organic frameworks (protein-MOFs) as chemically designed protein crystals, composed of ferritin nodes that predictably assemble into 3D lattices upon coordination of various metal ions and ditopic, hydrox-amate-based linkers. Owing to their unique tripartite construction, protein-MOFs possess extremely sparse lattice connectivi-ty, suggesting that they might display unusual thermomechanical properties. Leveraging the synthetic modularity of ferritin-MOFs, we investigated the temperature-dependent structural dynamics of six distinct frameworks. Our results show that the thermostabilities of ferritin-MOFs can be tuned through the metal component or the presence of crowding agents. Our stud-ies also reveal a framework that undergoes a reversible and isotropic first-order phase transition near-room temperature, corre-sponding to a 4% volumetric change within ≤1 deg C and a hysteresis window of ~10 deg C. Here, this highly cooperative crystal-to-crystal transformation, which stems from the soft crystallinity of ferritin-MOFs, illustrates the advantage of modular con-struction strategies in discovering tunable–and unpredictable–material properties.

36 MATERIALS SCIENCE↗

Enabling aqueous processing for LiNi 0.80 Co 0.15 Al 0.05 O 2 (NCA)-based lithium-ion battery cathodes using polyacrylic acid

Replacing N-methyl-2-pyrrolidone (NMP) with water in the production of lithium-ion battery cathodes is critical to realizing process cost savings and improved worker safety. LiNi 0.80 Co 0.15 Al 0.05 O 2 (NCA) is a poor fit for aqueous processing due to destructive Al current collector corrosion resulting from highly basic slurries and detrimental surface reconstruction reactions that occur in water. In this study, polyacrylic acid (PAA, MW = 450,000 g·mol -1 ) is examined as a corrosion-mitigating and surface-stabilizing agent. Adding PAA to an aqueous NCA slurry can provide a stable pH (4.0–8.5) for at least 4 h, since the carboxyl groups from PAA dissociate and increase the proton concentration in the slurry. Further, these groups can also adsorb to the surface of NCA particles and provide electrostatic stability from active material particle agglomeration, as revealed by zeta potential measurements. Minor cracking does occur at the electrode surface; this cracking likely caused poorer adhesion to the Al current collector in the aqueous-processed film when compared to the NMP-processed baseline. Electrochemically, the leached Li does cause a lower initial capacity for the aqueous-processed cathode, though the capacity retention of the aqueous-processed cathode is better than the baseline. The cracks in the coating led to a rise in charge transfer resistance that hindered rate capability above 1C.

25 ENERGY STORAGE↗

High-Resolution Mapping of Lunar Crustal Magnetic Fields: Correlations with Albedo Markings of the Reiner Gamma Class

During the last eight months of the Lunar Prospector mission (December 1999-July 1999), the spacecraft was placed in a relatively low-altitude (15-30-km perapsis), near-polar orbit that allowed high-resolution mapping of crustal magnetic fields. We report here initial studies of the correlation of locally strong magnetic anomalies with unusual, swirl-like albedo markings of the Reiner Gamma class. Based on this correlation, which is known from earlier studies of Apollo subsatellite magnetometer data, it has been proposed that the swirls represent regions whose higher albedos have been preserved via deflection of the solar-wind ion bombardment by strong crustal fields. This model in turn depends on the hypothesis that solar-wind implanted H is at least one component of the process that optically matures exposed silicate surfaces in the inner solar system . Specifically, it is hypothesized that implanted H acts as an effective reducing agent to enhance the rate of production of nanophase metallic Fe particles from preexisting silicates during micrometeoroid impacts. According to the model, the curvilinear shapes of these albedo markings are caused, at least in part, by the geometry of ion deflections in a magnetic field. The improved resolution and coverage of the Prospector data allow more detailed mapping of the fields, especially on the lunar farside. This permits a more quantitative test of whether all albedo markings of this class are associated with strong local magnetic fields.Only if the latter condition is met can the solar-wind deflection hypothesis he valid. The basic procedure for mapping crustal magnetic fields using Lunar Prospector magnetometer data follows that developed for analysis of Apollo subsatellite magnetometer data. The specific mapping steps are (1) selection of mission time intervals suitable for mapping crustal fields; these are limited essentially either to times when the Moon is in a lobe of the geomagnetic tail or to times when the Moon is in the solar wind but the spacecraft is in the lunar wake; the data are transformed to a radial, east, and north coordinate system with measurements given as a function of spacecraft latitude, longitude, and altitude; (2) visual editing of individual orbit segments selected for minimal external field disturbances; (3) minimization of remaining low-frequency external fields for individual orbit data segments by quadratic detrending; and (4) two-dimensional filtering of individual orbit segments to produce a vector field map along the slightly curved surface defined by the spacecraft altitude; maps of the three field components (radial, east, and north), the field magnitude, and the spacecraft altitude are constructed. For data obtained at low to middle latitudes, the horizontal resolution of the field maps is limited by the orbit-track separation (about 30 km at the equator). Maps of the field magnitude have been constructed within limited selenographic regions based mainly on data acquired in March, April, and May of 1999. This was a time period when the orbit plane was nearly aligned with the Sun-Moon line so that field mapping was possible at times when the Moon was in the solar wind as well as when the Moon was in the geomagnetic tail. Most of the coverage is across the lunar farside. However, a shows an example of a field map produced from solar-wind wake data for a region including Reiner Gamm on western Oceanus Procellarum (location: 58.5W, 7.5N). The contour interval is 3 nT and the mean spacecraft altitude is 18 km to within the accuracy allowed by the resolution of the map (30 km or about 1 deg.); strong magnetic anomalies correlate closely with swirl locations. Individual orbit profiles (whose resolution along the orbit track is comparable to the spacecraft altitude of 18 km) also demonstrate a good correlation of field magnitude with surface albedo. In order to investigate the correlation of magnetic fields with the location of swirl features, we have reexamined available lunar imagery (Lunar Orbiter, Apollo, and Clementine) to identify and map swirl locations within regions where swirls have previously been mapped. In these images, swirls were distinguished from other high-albedo features such as crater rays by their curvilinear shapes and increased visibility in forward-scattered light. Digital maps of swirls identified by all available imagery were then superposed on maps of the field magnitude at the spacecraft altitude. Based upon analysis of these composite magnetic/geologic maps, we draw the preliminary conclusion that swirl features are associated with magnetic anomalies revealed by Lunar Prospector. Detailed maps of these swirl features are currently being constructed for the magnetically strong regions antipodal to the Imbrium, Serenitatis, and Crisium Basins. Additional information contained in the original,

Hood, L. L.↗

Manipulation of amorphous precursors to enhance zeolite nucleation

Crystallization in media comprised of amorphous precursors is becoming a more common phenomenon for numerous synthetic, biological, and natural materials that grow by a combination of classical and nonclassical pathways. Amorphous phases can exhibit a wide range of physicochemical properties that may evolve during the course of nucleation and crystal growth. This creates challenges for establishing causal relationships between amorphous precursor properties and their effect(s) on the selection of mechanistic pathways of crystallization and ultimately the properties of the crystalline product. In this study, we examine ways to manipulate the composition and colloidal stability of amorphous (alumino)silicate precursors that are prevalent in nanoporous zeolite syntheses. Changes in the amorphous precursor properties are evaluated on the basis of their ability to enhance rates of crystal formation. Here, we use fumed silica as the primary silicon source and examine the effects of infusing the source or growth medium with additional alkali metal, which serves as an inorganic structure-directing agent to facilitate the formation of porous crystal structures. We also assess the impact of adding a polymer additive, which reduces the colloidal stability of precursors, wherein we posit that the confined pockets of solution within the interstitial spaces of the precursor aggregates play an important role in regulating the rate of zeolite crystallization. Three commercially relevant zeolites (mordenite, SSZ-13, and ZSM-5) were selected for this study based on their diverse frameworks and methods of preparation. Here our findings reveal that alkali infusion significantly reduces the crystallization times for mordenite and SSZ-13, but has little impact on ZSM-5 synthesis. Conversely, we find that polymer addition markedly enhanced the rates of crystallization among all three zeolites, suggesting that this method may be a general approach to reduce zeolite synthesis times. Given the relatively high costs associated with commercial zeolite production, identifying new methods to improve the efficiency of hydrothermal syntheses can have significant practical implications beyond the fundamental benefits of developing new routes to tailor nonclassical crystallization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Preliminary Work for Examining the Scalability of Reinforcement Learning

Researchers began studying automated agents that learn to perform multiple-step tasks early in the history of artificial intelligence (Samuel, 1963; Samuel, 1967; Waterman, 1970; Fikes, Hart & Nilsonn, 1972). Multiple-step tasks are tasks that can only be solved via a sequence of decisions, such as control problems, robotics problems, classic problem-solving, and game-playing. The objective of agents attempting to learn such tasks is to use the resources they have available in order to become more proficient at the tasks. In particular, each agent attempts to develop a good policy, a mapping from states to actions, that allows it to select actions that optimize a measure of its performance on the task; for example, reducing the number of steps necessary to complete the task successfully. Our study focuses on reinforcement learning, a set of learning techniques where the learner performs trial-and-error experiments in the task and adapts its policy based on the outcome of those experiments. Much of the work in reinforcement learning has focused on a particular, simple representation, where every problem state is represented explicitly in a table, and associated with each state are the actions that can be chosen in that state. A major advantage of this table lookup representation is that one can prove that certain reinforcement learning techniques will develop an optimal policy for the current task. The drawback is that the representation limits the application of reinforcement learning to multiple-step tasks with relatively small state-spaces. There has been a little theoretical work that proves that convergence to optimal solutions can be obtained when using generalization structures, but the structures are quite simple. The theory says little about complex structures, such as multi-layer, feedforward artificial neural networks (Rumelhart & McClelland, 1986), but empirical results indicate that the use of reinforcement learning with such structures is promising. These empirical results make no theoretical claims, nor compare the policies produced to optimal policies. A goal of our work is to be able to make the comparison between an optimal policy and one stored in an artificial neural network. A difficulty of performing such a study is finding a multiple-step task that is small enough that one can find an optimal policy using table lookup, yet large enough that, for practical purposes, an artificial neural network is really required. We have identified a limited form of the game OTHELLO as satisfying these requirements. The work we report here is in the very preliminary stages of research, but this paper provides background for the problem being studied and a description of our initial approach to examining the problem. In the remainder of this paper, we first describe reinforcement learning in more detail. Next, we present the game OTHELLO. Finally we argue that a restricted form of the game meets the requirements of our study, and describe our preliminary approach to finding an optimal solution to the problem.

Clouse, Jeff↗