Nonzero-sum differential games - Concepts and models
Differential games controlling inputs to single dynamic system as extension of optimal control theory
Starr, A. W.↗
SEARCH · Engineering Papers
Results for “Game theory”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cognitive Modeling of Video Game Player User Experience
This paper argues for the use of cognitive modeling to gain a detailed and dynamic look into user experience during game play. Applying cognitive models to game play data can help researchers understand a player's attentional focus, memory status, learning state, and decision strategies (among other things) as these cognitive processes occurred throughout game play. This is a stark contrast to the common approach of trying to assess the long-term impact of games on cognitive functioning after game play has ended. We describe what cognitive models are, what they can be used for and how game researchers could benefit by adopting these methods. We also provide details of a single model - based on decision field theory - that has been successfUlly applied to data sets from memory, perception, and decision making experiments, and has recently found application in real world scenarios. We examine possibilities for applying this model to game-play data.
Bohil, Corey J.↗
Simulation, models, and games - sources of measurement.
Sources of measurement through analogy in simulation, models and games
MAN-MACHINE SYSTEM↗
Longer term consequences of the Short Take-Off and Landing (STOL) aircraft system
An assessment of the STOL aircraft and the various means of employing it are discussed in the light of a research study to evaluate the efficacy of such analyses. It was determined that current approaches to assessment are generally inadequate for investigating the full social consequences of implementing a new technology. It is stated that a meaningful methodology of technology assessment must reflect mechanisms underlying the relationship of technology to social change. Interrelated methods which are discussed are: (1) gaming and simulation as heurisitic approaches in analysis and inquiry, (2) long range planning and questions of the future, (3) planning theory as a background for critical analysis of policy planning, and (4) social theory, with particular emphasis on social change and systems theories.
Laporte, T. R.↗
Distributed Control by Lagrangian Steepest Descent
Often adaptive, distributed control can be viewed as an iterated game between independent players. The coupling between the players mixed strategies, arising as the system evolves from one instant to the next, is determined by the system designer. Information theory tells us that the most likely joint strategy of the players, given a value of the expectation of the overall control objective function, is the minimizer of a function o the joint strategy. So the goal of the system designer is to speed evolution of the joint strategy to that Lagrangian mhimbhgpoint,lowerthe expectated value of the control objective function, and repeat Here we elaborate the theory of algorithms that do this using local descent procedures, and that thereby achieve efficient, adaptive, distributed control.
Wolpert, David H.↗
Item Retrieval as Utility Estimation
Retrieval systems have greatly improved over the last half century, estimating relevance to a latent user need in a wide variety of areas. One area that has not enjoyed such advancements is searching for items by attribute values, a common activity in e-commerce and science, particularly given numeric values. Existing item retrieval systems assume the user has a firm grasp of their own desires and can formulate a good Boolean or SQL-style query to retrieve items, as one would do with a database. A contrasting approach would be to estimate how well items match the user?s latent desires and return items ranked by this estimation. Towards this end, we present a retrieval model inspired by multi-criteria decision making theory, concentrating on numeric attributes. We evaluate our novel approach, the de-facto standard of Boolean retrieval, and several models proposed in the literature, in two user studies using Amazon Mechanical Turk. We use a competitive game to motivate test subjects and compare methods based on the results of the subjects? initial query and their success in the game. In our experiments, our new method signi cantly outperformed the others, whereas the Boolean approaches had the worst performance.
operations research↗
Theory of Collective Intelligence
In this chapter an analysis of the behavior of an arbitrary (perhaps massive) collective of computational processes in terms of an associated "world" utility function is presented We concentrate on the situation where each process in the collective can be viewed as though it were striving to maximize its own private utility function. For such situations the central design issue is how to initialize/update the collective's structure, and in particular the private utility functions, so as to induce the overall collective to behave in a way that has large values of the world utility. Traditional "team game" approaches to this problem simply set each private utility function equal to the world utility function. The "Collective Intelligence" (COIN) framework is a semi-formal set of heuristics that recently have been used to construct private utility. functions that in many experiments have resulted in world utility values up to orders of magnitude superior to that ensuing from use of the team game utility. In this paper we introduce a formal mathematics for analyzing and designing collectives. We also use this mathematics to suggest new private utilities that should outperform the COIN heuristics in certain kinds of domains. In accompanying work we use that mathematics to explain previous experimental results concerning the superiority of COIN heuristics. In that accompanying work we also use the mathematics to make numerical predictions, some of which we then test. In this way these two papers establish the study of collectives as a proper science, involving theory, explanation of old experiments, prediction concerning new experiments, and engineering insights.
Nash equilibrium↗
A finite state, finite memory minimum principle, part 2
The development of the theory of the finite - state, finite - memory (FSFM) stochastic control problem is discussed. The sufficiency of the FSFM minimum principle (which is in general only a necessary condition) was investigated. By introducing the notion of a signaling strategy as defined in the literature on games, conditions under which the FSFM minimum principle is sufficient were determined. This result explicitly interconnects the information structure of the FSFM problem with its optimality conditions. The min-H algorithm for the FSFM problem was studied. It is demonstrated that a version of the algorithm always converges to a particular type of local minimum termed a person - by - person extremal.
Sandell, N. R., Jr.↗
Optimal helicopter trajectory planning for terrain following flight
Helicopters operating in high threat areas have to fly close to the earth surface to minimize the risk of being detected by the adversaries. Techniques are presented for low altitude helicopter trajectory planning. These methods are based on optimal control theory and appear to be implementable onboard in realtime. Second order necessary conditions are obtained to provide a criterion for finding the optimal trajectory when more than one extremal passes through a given point. A second trajectory planning method incorporating a quadratic performance index is also discussed. Trajectory planning problem is formulated as a differential game. The objective is to synthesize optimal trajectories in the presence of an actively maneuvering adversary. Numerical methods for obtaining solutions to these problems are outlined. As an alternative to numerical method, feedback linearizing transformations are combined with the linear quadratic game results to synthesize explicit nonlinear feedback strategies for helicopter pursuit-evasion. Some of the trajectories generated from this research are evaluated on a six-degree-of-freedom helicopter simulation incorporating an advanced autopilot. The optimal trajectory planning methods presented are also useful for autonomous land vehicle guidance.
Menon, P. K. A.↗
Collectives for Multiple Resource Job Scheduling Across Heterogeneous Servers
Efficient management of large-scale, distributed data storage and processing systems is a major challenge for many computational applications. Many of these systems are characterized by multi-resource tasks processed across a heterogeneous network. Conventional approaches, such as load balancing, work well for centralized, single resource problems, but breakdown in the more general case. In addition, most approaches are often based on heuristics which do not directly attempt to optimize the world utility. In this paper, we propose an agent based control system using the theory of collectives. We configure the servers of our network with agents who make local job scheduling decisions. These decisions are based on local goals which are constructed to be aligned with the objective of optimizing the overall efficiency of the system. We demonstrate that multi-agent systems in which all the agents attempt to optimize the same global utility function (team game) only marginally outperform conventional load balancing. On the other hand, agents configured using collectives outperform both team games and load balancing (by up to four times for the latter), despite their distributed nature and their limited access to information.
Tumer, K.↗
Time-Extended Payoffs for Collectives of Autonomous Agents
A collective is a set of self-interested agents which try to maximize their own utilities, along with a a well-defined, time-extended world utility function which rates the performance of the entire system. In this paper, we use theory of collectives to design time-extended payoff utilities for agents that are both aligned with the world utility, and are "learnable", i.e., the agents can readily see how their behavior affects their utility. We show that in systems where each agent aims to optimize such payoff functions, coordination arises as a byproduct of the agents selfishly pursuing their own goals. A game theoretic analysis shows that such payoff functions have the net effect of aligning the Nash equilibrium, Pareto optimal solution and world utility optimum, thus eliminating undesirable behavior such as agents working at cross-purposes. We then apply collective-based payoff functions to the token collection in a gridworld problem where agents need to optimize the aggregate value of tokens collected across an episode of finite duration (i.e., an abstracted version of rovers on Mars collecting scientifically interesting rock samples, subject to power limitations). We show that, regardless of the initial token distribution, reinforcement learning agents using collective-based payoff functions significantly outperform both natural extensions of single agent algorithms and global reinforcement learning solutions based on "team games".
Tumer, Kagan↗
Qualitative differential games with two targets
So-called differential games of kind (qualitative games) were considered involving two or more players each of whom possesses a target toward which he wished to steer the response of a dynamical system that was under the control of all players. Sufficient conditions were derived, which assure termination on a particular player's target. In general, these conditions were constructive in that they permited construction of a winning (terminating) strategy for a player. The theory is illustrated by a pursuit-evasion problem.
Getz, W. M.↗
Preliminary Work for Examining the Scalability of Reinforcement Learning
Researchers began studying automated agents that learn to perform multiple-step tasks early in the history of artificial intelligence (Samuel, 1963; Samuel, 1967; Waterman, 1970; Fikes, Hart & Nilsonn, 1972). Multiple-step tasks are tasks that can only be solved via a sequence of decisions, such as control problems, robotics problems, classic problem-solving, and game-playing. The objective of agents attempting to learn such tasks is to use the resources they have available in order to become more proficient at the tasks. In particular, each agent attempts to develop a good policy, a mapping from states to actions, that allows it to select actions that optimize a measure of its performance on the task; for example, reducing the number of steps necessary to complete the task successfully. Our study focuses on reinforcement learning, a set of learning techniques where the learner performs trial-and-error experiments in the task and adapts its policy based on the outcome of those experiments. Much of the work in reinforcement learning has focused on a particular, simple representation, where every problem state is represented explicitly in a table, and associated with each state are the actions that can be chosen in that state. A major advantage of this table lookup representation is that one can prove that certain reinforcement learning techniques will develop an optimal policy for the current task. The drawback is that the representation limits the application of reinforcement learning to multiple-step tasks with relatively small state-spaces. There has been a little theoretical work that proves that convergence to optimal solutions can be obtained when using generalization structures, but the structures are quite simple. The theory says little about complex structures, such as multi-layer, feedforward artificial neural networks (Rumelhart & McClelland, 1986), but empirical results indicate that the use of reinforcement learning with such structures is promising. These empirical results make no theoretical claims, nor compare the policies produced to optimal policies. A goal of our work is to be able to make the comparison between an optimal policy and one stored in an artificial neural network. A difficulty of performing such a study is finding a multiple-step task that is small enough that one can find an optimal policy using table lookup, yet large enough that, for practical purposes, an artificial neural network is really required. We have identified a limited form of the game OTHELLO as satisfying these requirements. The work we report here is in the very preliminary stages of research, but this paper provides background for the problem being studied and a description of our initial approach to examining the problem. In the remainder of this paper, we first describe reinforcement learning in more detail. Next, we present the game OTHELLO. Finally we argue that a restricted form of the game meets the requirements of our study, and describe our preliminary approach to finding an optimal solution to the problem.
Clouse, Jeff↗
Pressure-Sensitive Paints Advance Rotorcraft Design Testing
The rotors of certain helicopters can spin at speeds as high as 500 revolutions per minute. As the blades slice through the air, they flex, moving into the wind and back out, experiencing pressure changes on the order of thousands of times a second and even higher. All of this makes acquiring a true understanding of rotorcraft aerodynamics a difficult task. A traditional means of acquiring aerodynamic data is to conduct wind tunnel tests using a vehicle model outfitted with pressure taps and other sensors. These sensors add significant costs to wind tunnel testing while only providing measurements at discrete locations on the model's surface. In addition, standard sensor solutions do not work for pulling data from a rotor in motion. "Typical static pressure instrumentation can't handle that," explains Neal Watkins, electronics engineer in Langley Research Center s Advanced Sensing and Optical Measurement Branch. "There are dynamic pressure taps, but your costs go up by a factor of five to ten if you use those. In addition, recovery of the pressure tap readings is accomplished through slip rings, which allow only a limited amount of sensors and can require significant maintenance throughout a typical rotor test." One alternative to sensor-based wind tunnel testing is pressure sensitive paint (PSP). A coating of a specialized paint containing luminescent material is applied to the model. When exposed to an LED or laser light source, the material glows. The glowing material tends to be reactive to oxygen, explains Watkins, which causes the glow to diminish. The more oxygen that is present (or the more air present, since oxygen exists in a fixed proportion in air), the less the painted surface glows. Imaged with a camera, the areas experiencing greater air pressure show up darker than areas of less pressure. "The paint allows for a global pressure map as opposed to specific points," says Watkins. With PSP, each pixel recorded by the camera becomes an optical pressure tap. "Instead of having 100 or 200 pressure taps, you can have in theory several million, up to whatever the resolution of your camera is." Watkins explains that typical wind tunnel testing requires two models: one with very little instrumentation, and a pressure model with a significant amount of sensors applied. "If you can make all of your measurements on one model with PSP, you've decreased your model costs by at least a factor of two and preferably your testing costs by about that much," he says. PSP technology has been around for almost 20 years, but a PSP solution for gathering instantaneous dynamic pressure data from surfaces moving at high speeds, such as rotor blades, was not available until a NASA partnership led to a game-changing innovation.
Source record↗
Independent Technical Assessment of NASA and External Quantum Sensing Capability
The recent FY 2020 federal Research and Development Budget Priorities memo addresses the leadership need in Quantum Information Science (QIS) directing agencies to “prioritize QIS research and development (R&D), which will build the technical and scientific base necessary to explore the next generation of QIS theory, devices, and applications.” Quantum Sensing (QS) is an integral part of QIS. NASA is a key part of the directive to forward American space exploration and commercialization by providing “capabilities that have broad potential applications in space and on Earth.” QS provides an arena for NASA to demonstrate leadership in both areas of the administrative directive. QS uses quantum properties to achieve unprecedented measurement sensitivity and performance, including quantum-enhanced methodologies that outperform their classical counterparts. Typical quantum sensors exploit techniques such as atomic systems, matter waves, quantum entanglement, quantum superposition of states, quantum illumination methods, and manipulation of photons and atoms, in general. Guided by advancements in our ability to generate, manipulate, and control quantum systems, the emerging quantum sensing technologies promise unrivalled sensitivity, resolution, and precision, potentially leading to game-changing applications. Significant gains include technologies important for a range of NASA missions such as remote sensing, in situ measurements, metrology, interferometry, quantum communication, ranging, imaging, radar and lidar receivers, and gravity measurements. NASA Engineering and Safety Center has convened an independent external panel, comprising of Quantum Sensing Experts from Government, DoD, academia, and Federally funded Research and Development Center to conduct an independent technical assessment of the agency's capabilities in Quantum Sensing to understand NASA's internal needs and competencies related to Quantum Sensing and compare agency capabilities with those available externally including industry, academia, and other government agencies. The outcomes of the assessment will help the agency in establishing appropriate strategies and investments to develop and maintain the state-of-the-art sensing competence and capabilities required to meet the agency’s future needs. The External Experts Panel (EEP) is collaborating with HQ, various NASA Center and the representatives in the NASA Quantum Sensing Community of Practice (QS CoP), a part of NASA Sensors and Instrumentation Technical Fellow Technical Discipline Team, in obtaining common, current understanding of agency mission needs where QS can be an important enabler for future needs, and any programs, projects, assets, and technologists working in QS. The EEP organized a Quantum Sensing Workshop of practitioners from industry, academia, other government agencies, external experts, and interested NASA personnel to gather the assessment information. EEP also conducted information gathering on the industry at large, educational institutions, and other government agency research efforts for capture in the assessment database. At the conclusion of this assessment, EEP team will develop findings and conclusions describing NASA’s capabilities for the mission needs, NASA's relative position on new, enabling technologies, and an analysis of the gaps that may present any risks to near-term or far-term mission needs. This presentation will give details of the NASA and External Quantum Sensing Assessment outcomes, findings, observations, and its recommendation to NASA as how it can advance Quantum Sensing technologies and techniques for its science and explorations related missions.
External Quantum Sensing Capability↗
Temporal Fluctuations in a 100-A LaB6 Hollow Cathode
The temporal fluctuations in the near plume of a 100-Amperes LaB6 (Lanthanum hexaboride) hollow cathode are experimentally investigated. At high currents, turbulent oscillations may contribute to two of the anomalous processes of hollow cathode operation - anomalous resistivity and the production of energetic ions. A detailed study of the properties of the oscillations in a high current cathode is necessary to determine the impact of the fluctuation spectrum on these two effects. In this investigation, a probe array is employed to measure the amplitude and dispersion of axial modes in the plume while a retarding potential analyzer yields estimates of the radial ion energy distribution. The onset of the ion acoustic turbulence (IAT) is observed at high current values, and the character of the turbulent spectrum is shown to agree with weak turbulent theory: the amplitude of the spectrum decreases with flow rate but increases with discharge current. Estimates of the anomalous collision frequency based on experimental observations indicate that the IAT collision frequency can exceed the classical collision frequency at sufficiently high discharge current densities. Additionally, the onset of the IAT is shown to be correlated with the appearance of a high energy ion tail and that the energy in this tail is comparable to the energy in the experimentally-observed IAT.
production of ions↗
Demonstration of Capability to Simulate Particle Irregular Shape and Poly-Disperse Mixtures Within Lunar Lander Plume-Surface Interaction
Plume-Surface Interaction (PSI) between lander engine plumes and regolith soil creates hazards in obscuration and contamination by particle clouds, high-energy ejecta streams, and landing area cratering damage. The MSFC Fluid Dynamics Branch is developing simulation tools to offer a predictive PSI capability to NASA customers such as the Human Lander System (HLS) and Commercial Lunar Payload Services (CLPS). The Gas-Granular Flow Solver (GGFS) is the main application tool for coupled gas-particle two-phase flow simulations to predict the range of PSI effects from onset of surface erosion to deep crater formation. GGFS features an Eulerian-Eulerian modeling approach, treating both gas and granular material as interacting continuum phases. Modeling the lunar regolith granular material fluidic characteristics poses special challenges due to complex particle shapes and mixture composition. The lunar regolith is poorly sorted with broad particle size distributions and large fines content. It has significant cohesion, due to interlocking jagged particle shapes. Eulerian granular material flow modeling requires closure formulations for the granular material constitutive models (stress, friction, collisional and kinetic energy dissipation, drag, etc.). While closure models for spherical particles are available from particle kinetic theory, closure models for realistic non-spherical particles must be extracted from unit physics Discrete Element Model (DEM) particle interaction simulations and provided in the form of tabular datasets. The effects of particle irregular shape (non-spherical shape factors, angular particle surface roughness, and interlocking features) are simulated by approximating the particle features in the form of grouped elemental spheres to form composite particles in the DEM simulations. The effects of the wide range of regolith mixture particle sizes and the strong effects of the presence of the small particle sizes results in high cohesion and low porosity of the regolith mixture. The range of particle sizes is simulated by binning the particle sizes into an appropriate finite number of particle-size species and solving the problem as a species mixture. Combining these two modeling approaches enables simulations to capture both, the contributions of the irregular particle shape and the particle size distribution. The integration and maturation of the DEM-based constitutive model database generation process and poly-disperse mixture binning approach into the GGFS simulation framework are proceeding under funding by the NASA Game Changing Development program. The status of current capabilities will be presented in comparisons of crater characteristics resulting for spherical and irregular shape particles, and for mono-, bi-, and tri-disperse mixture simulations of Apollo LM plume-surface interaction. The computational results confirm the significance of including the particle shape and mixture effects. Going forward plans for the full implementation of the general poly-disperse regolith modeling capability and maturation towards NASA project application readiness under the GCD program will be presented.
Peter A Liever↗
Demonstration of Capability to Simulate Particle Irregular Shape and Poly-Disperse Mixtures Within Lunar Lander Plume-Surface Interaction
Plume-Surface Interaction (PSI) between lander engine plumes and regolith soil creates hazards in obscuration and contamination by particle clouds, high-energy ejecta streams, and landing area cratering damage. The MSFC Fluid Dynamics Branch is developing simulation tools to offer a predictive PSI capability to NASA customers such as the Human Lander System (HLS) and Commercial Lunar Payload Services (CLPS). The Gas-Granular Flow Solver (GGFS) is the main application tool for coupled gas-particle two-phase flow simulations to predict the range of PSI effects from onset of surface erosion to deep crater formation. GGFS features an Eulerian-Eulerian modeling approach, treating both gas and granular material as interacting continuum phases. Modeling the lunar regolith granular material fluidic characteristics poses special challenges due to complex particle shapes and mixture composition. The lunar regolith is poorly sorted with broad particle size distributions and large fines content. It has significant cohesion, due to interlocking jagged particle shapes. Eulerian granular material flow modeling requires closure formulations for the granular material constitutive models (stress, friction, collisional and kinetic energy dissipation, drag, etc.). While closure models for spherical particles are available from particle kinetic theory, closure models for realistic non-spherical particles must be extracted from unit physics Discrete Element Model (DEM) particle interaction simulations and provided in the form of tabular datasets. The effects of particle irregular shape (non-spherical shape factors, angular particle surface roughness, and interlocking features) are simulated by approximating the particle features in the form of grouped elemental spheres to form composite particles in the DEM simulations. The effects of the wide range of regolith mixture particle sizes and the strong effects of the presence of the small particle sizes results in high cohesion and low porosity of the regolith mixture. The range of particle sizes is simulated by binning the particle sizes into an appropriate finite number of particle-size species and solving the problem as a species mixture. Combining these two modeling approaches enables simulations to capture both, the contributions of the irregular particle shape and the particle size distribution. The integration and maturation of the DEM-based constitutive model database generation process and poly-disperse mixture binning approach into the GGFS simulation framework are proceeding under funding by the NASA Game Changing Development program. The status of current capabilities will be presented in comparisons of crater characteristics resulting for spherical and irregular shape particles, and for mono-, bi-, and tri-disperse mixture simulations of Apollo LM plume-surface interaction. The computational results confirm the significance of including the particle shape and mixture effects. Going forward plans for the full implementation of the general poly-disperse regolith modeling capability and maturation towards NASA project application readiness under the GCD program will be presented.
Peter A. Liever↗