Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Game theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Product Distributions for Distributed Optimization

With connections to bounded rational game theory, information theory and statistical mechanics, Product Distribution (PD) theory provides a new framework for performing distributed optimization. Furthermore, PD theory extends and formalizes Collective Intelligence, thus connecting distributed optimization to distributed Reinforcement Learning (FU). This paper provides an overview of PD theory and details an algorithm for performing optimization derived from it. The approach is demonstrated on two unconstrained optimization problems, one with discrete variables and one with continuous variables. To highlight the connections between PD theory and distributed FU, the results are compared with those obtained using distributed reinforcement learning inspired optimization approaches. The inter-relationship of the techniques is discussed.

Bieniawski, Stefan R.↗
NASA NTRS2005

Distributed Adaptive Control: Beyond Single-Instant, Discrete Variables

In extensive form noncooperative game theory, at each instant t, each agent i sets its state x, independently of the other agents, by sampling an associated distribution, q(sub i)(x(sub i)). The coupling between the agents arises in the joint evolution of those distributions. Distributed control problems can be cast the same way. In those problems the system designer sets aspects of the joint evolution of the distributions to try to optimize the goal for the overall system. Now information theory tells us what the separate q(sub i) of the agents are most likely to be if the system were to have a particular expected value of the objective function G(x(sub 1),x(sub 2), ...). So one can view the job of the system designer as speeding an iterative process. Each step of that process starts with a specified value of E(G), and the convergence of the q(sub i) to the most likely set of distributions consistent with that value. After this the target value for E(sub q)(G) is lowered, and then the process repeats. Previous work has elaborated many schemes for implementing this process when the underlying variables x(sub i) all have a finite number of possible values and G does not extend to multiple instants in time. That work also is based on a fixed mapping from agents to control devices, so that the the statistical independence of the agents' moves means independence of the device states. This paper also extends that work to relax all of these restrictions. This extends the applicability of that work to include continuous spaces and Reinforcement Learning. This paper also elaborates how some of that earlier work can be viewed as a first-principles justification of evolution-based search algorithms.

Wolpert, David H.↗
NASA NTRS2005

A Comparative Study of Probability Collectives Based Multi-agent Systems and Genetic Algorithms

We compare Genetic Algorithms (GA's) with Probability Collectives (PC), a new framework for distributed optimization and control. In contrast to GA's, PC-based methods do not update populations of solutions. Instead they update an explicitly parameterized probability distribution p over the space of solutions. That updating of p arises as the optimization of a functional of p. The functional is chosen so that any p that optimizes it should be p peaked about good solutions. The PC approach works in both continuous and discrete problems. It does not suffer from the resolution limitation of the finite bit length encoding of parameters into GA alleles. It also has deep connections with both game theory and statistical physics. We review the PC approach using its motivation as the information theoretic formulation of bounded rationality for multi-agent systems. It is then compared with GA's on a diverse set of problems. To handle high dimensional surfaces, in the PC method investigated here p is restricted to a product distribution. Each distribution in that product is controlled by a separate agent. The test functions were selected for their difficulty using either traditional gradient descent or genetic algorithms. On those functions the PC-based approach significantly outperforms traditional GA's in both rate of descent, trapping in false minima, and long term optimization.

Huang, Chien-Feng↗
NASA NTRS2011

Modeling Humans as Reinforcement Learners: How to Predict Human Behavior in Multi-Stage Games

This paper introduces a novel framework for modeling interacting humans in a multi-stage game environment by combining concepts from game theory and reinforcement learning. The proposed model has the following desirable characteristics: (1) Bounded rational players, (2) strategic (i.e., players account for one anothers reward functions), and (3) is computationally feasible even on moderately large real-world systems. To do this we extend level-K reasoning to policy space to, for the first time, be able to handle multiple time steps. This allows us to decompose the problem into a series of smaller ones where we can apply standard reinforcement learning algorithms. We investigate these ideas in a cyber-battle scenario over a smart power grid and discuss the relationship between the behavior predicted by our model and what one might expect of real human defenders and attackers.

Lee, Ritchie↗
NASA NTRSDate not supplied

Aerial Vehicle Routing and Scheduling for UAS Traffic Management: A Monte Carlo Tree Search Approach

Numerous unmanned aircraft systems operating at low altitudes to deliver goods and services may one day become ubiquitous in our cities. In the Unmanned Aircraft Systems (UAS) Traffic Management (UTM) framework, such a concept is envisioned, where aerial vehicles operate beyond visual line of sight (BVLOS) within specifically reserved and time stamped “corridors” in the airspace. For example, these corridors or operational intent volumes can connect an aerial vehicle’s origin site to its destination site for package delivery operations. There may also be more than one corridor available for an aerial vehicle to choose from and often different corridors may intersect with one another. Thus, it is imperative to ensure flight trajectories belonging to different aerial vehicles are not in conflict. Per the UTM CONOPs, we assume that a vehicle almost always stays inside its corridor or operational volume. This work provides a framework for strategic deconfliction of UTM or package delivery drones, where we schedule the departure time of all vehicles subject to various temporal constraints (including the corridor deconfliction at the intersections). We present the “multi-route weighted package delivery problem” which serves as an exemplifying model for strategic deconfliction in UTM. In the multi-route weighted package delivery problem, a graph network is given which consists of a set of depots (source) and drop-off (destination) nodes, with multiple routes (defined as a sequence of waypoints) connecting the depots to drop-off nodes. In addition, routes are weighted by the associated ground risk and total travel distance for package delivery. The goal is for a known set of aerial vehicles to depart from the depots, choose a route and take off time, while avoiding conflicts with other aerial vehicles, and minimizing both risk and distance traveled. We provide a mixed integer linear programming (MILP) formulation of the problem, as well as a heuristic solution based on Monte Carlo Tree Search (MCTS) – a method used in game theory and artificial intelligence – to overcome limitations inherent to optimal solvers. Computational results show the advantages of using MCTS over the MILP formulation; the former can provide a sub-optimal solution quickly, and may sometimes even reach an optimal solution, whereas the latter may not even produce a solution in reasonable time. Furthermore, results from both the MILP formulation and MCTS methods were validated using a preliminary agent-based simulator implementing the UTM concept of operations. Thus, the MCTS method can be seen as a scalable solution to the complex multi-route weighted package delivery problem and may possibly be extended to similar complex optimization problems.

Kenny Chour↗
NASA NTRSDate not supplied

Off-Nominal Event Analysis in Autonomous Flights Based on Explainable Artificial Intelligence

A key objective in the Urban Air Mobility program at NASA is to intelligently perform an autonomous flight in a complex urban environment under all weather conditions with guaranteed levels of safety. To accomplish this, the mission manager (central decision-making module) of the vehicle needs to make informed decisions between various Courses of Action (CoA) based on its' interpretation of the inputs it receives. If an off-nominal event is detected either based on the amalgamation of sensor data or the use of machine learning models, the mission manager may greatly benefit from identification of the input features that most likely contributed to that specific event. Such an understanding is usually not possible to obtain from the classical machine learning models (deep learning) due to the inherent black box like structure. However, this understanding is achieved using eXplainable Artificial Intelligence (XAI) models that provide a human interpretable rationale for the predictions made. This work presents a game theory inspired XAI model for the off-nominal assessment of autonomous flights. The proposed approach based on Shapley values is model agnostic, provides local as well as global explanation and satisfies the four axioms (efficiency, symmetry, dummy, additivity) to achieve fair contribution. The versatility of the approach is first demonstrated on a simulated dataset in which the significance of each input to flight phase prediction is clearly identified. Subsequently, data from simulated flight trajectories are fed into the model which reveal the input features that most likely contributed to a rotor failure event thereby empowering the mission manager to take the appropriate CoA.

autonomy↗
NASA NTRSDate not supplied

Off-Nominal Event Analysis in Autonomous Flights Based on Explainable Artificial Intelligence

A key objective in the Urban Air Mobility program at NASA is to intelligently perform an autonomous flight in a complex urban environment under all weather conditions with guaranteed levels of safety. To accomplish this, the mission manager (central decision-making module) of the vehicle needs to make informed decisions between various Courses of Action (CoA) based on its' interpretation of the inputs it receives. If an off-nominal event is detected either based on the amalgamation of sensor data or the use of machine learning models, the mission manager may greatly benefit from identification of the input features that most likely contributed to that specific event. Such an understanding is usually not possible to obtain from the classical machine learning models (deep learning) due to the inherent black box like structure. However, this understanding is achieved using eXplainable Artificial Intelligence (XAI) models that provide a human interpretable rationale for the predictions made. This work presents a game theory inspired XAI model for the off-nominal assessment of autonomous flights. The proposed approach based on Shapley values is model agnostic, provides local as well as global explanation and satisfies the four axioms (efficiency, symmetry, dummy, additivity) to achieve fair contribution. The versatility of the approach is first demonstrated on a simulated dataset in which the significance of each input to flight phase prediction is clearly identified. Subsequently, data from simulated flight trajectories are fed into the model which reveal the input features that most likely contributed to a rotor failure event thereby empowering the mission manager to take the appropriate CoA.

autonomy↗
NASA NTRSDate not supplied

Multi-Party Flight Trajectory Negotiation for Upper Class E Traffic Management

A new operational concept has been proposed in Upper Class E airspace at or above 60,000 feet (Flight Level / FL600), which will allow operators of diverse vehicle characteristics to cooperatively manage and share their operational intents with neighboring operators to avoid conflict. There is a consensus in the community that negotiation for strategic deconfliction is needed, but there are no specific guidelines for how the negotiation should be conducted. There is a need for a structured and cooperative way to resolve the conflict between a wide variety of aircraft projected to be operating in Upper Class E for the negotiation to be carried out routinely. The use of negotiation models are a promising solution that can resolve conflict risks during flight in real-time while taking into account the uncertainty of future vehicle positions and dynamic business considerations. A two-party negotiation model has been researched, but as the traffic demand grows, there is a higher likelihood of conflict involving multiple aircraft that would require a method to handle multi-party conflict. This paper proposes a cooperative multi-party negotiation model inspired by game theory concepts for application to flight trajectory negotiation in Upper Class E traffic management. This model can be applied in flight with operators communicating directly after a potential conflict is detected. Some of the model’s benefits include allowing business costs to be private to operators, allowing operators to collaborate together to find conflict-free flight trajectories, and being compatible with different aircraft and operation types. This model provides a structured procedure for conflict resolution that can handle conflict involving multiple parties, assuming each operator is willing to take on a small cost to themselves in order to reduce the total cost to the group.

Upper Class E Traffic Management (ETM)↗
NASA NTRS2024

Generating Dominating Sets Using Locally Defined Centrality Measures

The dominating set problem has many practical applications but is well-known to be NP-hard. Therefore, there is a need for efficient heuristic algorithms, especially in applications such as ad hoc wireless networks. Most distributed algorithms proposed in the literature assume that each node has knowledge of the network structure. We propose a distributed heuristic algorithm that uses two rounds of communication, and where each node has only local information, both in terms of network structure and dominating set assignment. First, each node calculates a local centrality measure to determine whether it is part of the dominating set D. The second round guarantees D is a dominating set by adding any non-dominated nodes. We compare several centrality measures and show that the Shapley centrality, derived from the Shapley value in game theory, is theoretically motivated and performs well in practice on several synthetic and real-world networks.

Network↗
DOE OSTI2020

AI-Science for Performance Optimization and Diagnosis of Science Instrument Federations

Next generation of science workflows are expected to be executed over complex federations composed of supercomputers, science instruments, storage systems and networks, with new additions of the edge and cloud systems and services. The sheer complexity of these multi-domain federations makes it hard to manage them and optimize their performance, as small impedance mismatches (that can dynamically develop between systems) could drastically degrade the entire federation performance. Recent proliferation of Software Defined Everything (SDX) technologies combined with containerization frameworks provide custom instruments that can monitor and collect critical measurements at various levels to support diagnoses and performance optimization; but their data too enormous for human operators and analysts to process and generate decisions. Machine Learning (ML) methods that extract critical parameters, relationships and trends from the data offer general solutions. Artificial Intelligence (AI) and ML methods must be custom-developed for these problems based on solid, rigorous foundations, since black-box approaches are often ineffective and unsound.We propose to develop comprehensive AI-Science for the performance of science federations to (i) monitor and control storage, networks, experiments, and computing systems across multiple domains via softwarization layers, at speeds and scales orders of magnitude superior to current practice, (ii) optimally realize and orchestrate complex workflows with high performance by using dynamic state and performance estimation methods, and (iii) aggregate measurements across sites and time to develop infrastructure-level profiles, optimizations and diagnoses using AI-Science based on foundational principles from ML, game theory, and information fusion areas.

Rao, Nageswara↗
NASA NTRS2008

Game-Based Learning Theory

Persistent Immersive Synthetic Environments (PISE) are not just connection points, they are meeting places. They are the new public squares, village centers, malt shops, malls and pubs all rolled into one. They come with a sense of 'thereness" that engages the mind like a real place does. Learning starts as a real code. The code defines "objects." The objects exist in computer space, known as the "grid." The objects and space combine to create a "place." A "world" is created, Before long, the grid and code becomes obscure, and the "world maintains focus.

Laughlin, Daniel↗
DOE OSTI2025

Game theoretic modeling and optimization of competition and collaboration in dual channel electronic waste supply chains

The rapid growth of electronic waste (e-waste) presents critical challenges for sustainable resource recovery and environmental protection. This study develops a dual-channel closed-loop supply chain (CLSC) model formulated as a hierarchical Stackelberg game, that integrates dynamic pricing and cost-sharing mechanisms to optimize both economic and environmental outcomes. The model explicitly captures strategic interactions between manufacturer-led and third-party recycling channels, accounting for consumer behavior, regulatory incentives, and market competition. Numerical simulations conducted (implemented over a four-iteration horizon using a commercial optimization solver) show that, relative to the baseline equilibrium, manufacturer profit increases from 11.6 thousand USD to 37.9 thousand USD (+226.8%), total recycled volume rises from 7,848 to 7,942 units (+1.2%), and collector profit nearly doubles under cost-sharing, enabling more equitable profit distribution. Furthermore, scenario-based simulations across Sub-Saharan Africa, high-income economies, and emerging Asian industrial countries reveal that infrastructure quality, policy intensity, and labor costs critically shape recycling efficiency and profit allocation. These findings demonstrate that subsidies alone are insufficient to ensure system efficiency. Instead, coordinated strategies that integrate internal incentive alignment with context-sensitive policy support are required. Overall, this study offers a robust framework for designing resilient, efficient, and regionally adaptable e-waste management systems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS1972

Differential games.

General discussion of the theory of differential games with two players and zero sum. Games starting at a fixed initial state and ending at a fixed final time are analyzed. Strategies for the games are defined. The existence of saddle values and saddle points is considered. A stochastic version of a differential game is used to examine the synthesis problem.

Varaiya, P. P.↗
DOE OSTI2023

A game-theoretic approach to nuclear fuel cycle transition analysis under uncertainty

We present a novel methodology for optimizing nuclear fuel cycle transitions that incorporates a game-theoretic approach and captures interactions among multiple decision makers. The methodology is demonstrated using a two-person sequential game with uncertainty, where the two players represent a policy maker and an electric utility company, though the method generalizes to any number and type of individual decision making entities. Coupled with a sophisticated nuclear fuel cycle simulator, rich transition scenarios may be analyzed to identify robust transition strategies. These strategies explicitly treat uncertainties using a stochastic programming approach, devising optimal near-term hedging strategies that simultaneously consider all possible states of the world, maintaining flexibility to allow for intelligent recourse decisions once uncertainties are resolved. In the demonstration game, reactor technology and fuel cycle scheme adopted by the electric utility are shown to depend on both the policy maker’s decisions and the distributions over uncertain technological and economic outcomes.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

The role of information structures in game-theoretic multi-agent learning

Multi-agent learning (MAL) studies how agents learn to behave optimally and adaptively from their experience when interacting with other agents in dynamic environments. The outcome of a MAL process is jointly determined by all agents’ decision-making. Hence, each agent needs to think strategically about others’ sequential moves, when planning future actions. The strategic interactions among agents makes MAL go beyond the direct extension of single-agent learning to multiple agents. With the strategic thinking, each agent aims to build a subjective model of others decision-making using its observations. Such modeling is directly influenced by agents’ perception during the learning process, which is called the information structure of the agent’s learning. As it determines the input to MAL processes, information structures play a significant role in the learning mechanisms of the agents. This review creates a taxonomy of MAL and establishes a unified and systematic way to understand MAL from the perspective of information structures. Here we define three fundamental components of MAL: the information structure (i.e., what the agent can observe), the belief generation (i.e., how the agent forms a belief about others based on the observations), as well as the policy generation (i.e., how the agent generates its policy based on its belief). In addition, this taxonomy enables the classification of a wide range of state-of-the-art algorithms into four categories based on the belief-generation mechanisms of the opponents, including stationary, conjectured, calibrated, and sophisticated opponents. We introduce Value of Information (VoI) as a metric to quantify the impact of different information structures on MAL. Finally, we discuss the strengths and limitations of algorithms from different categories and point to promising avenues of future research.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Constrained Turret Defense with Fixed Final Time

In this paper, we extend existing turret defense differential game formulations involving a turn-constrained turret and mobile agent to include specified final time and a constraint. For the purposes of this analysis, the specified final time may represent some exogenous input, perhaps representing the time at which some other event will take place. As for the constraint, it represents a no-fly zone for the mobile agent. The scenario is formulated as a two-player, zero-sum differential game and solved via the method of characteristics (i.e., back-propagation of equilibrium trajectories). Three different trajectory types make up the solution: trajectories that end with the turret aligned with the mobile agent, trajectories that end with the mobile agent on the constraint boundary, and regular trajectories.

differential game, game theory↗
DOE OSTI2020

Simulating policy interventions in the interfirm diffusion of low-carbon technologies: An agent-based evolutionary game model

Policy interventions are frequently used by authorities around the world to mitigate carbon emissions. However, economic systems are essentially complex adaptive systems, which often exhibit unexpected responses to exogenous interventions and leave intervenors in a dilemma, even if the interventions are elaborately designed. To explore the behaviors of the diffusion system regarding low-carbon technologies, this study builds an agent-based model (ABM) to simulate enterprises' reactions to multiple policy interventions aimed at spurring low-carbon technology diffusion. The simulated enterprises are in a complex network where they play evolutionary games with their neighbors, which enables the model to possess two critical features of economic systems: adaptiveness and equilibrium. The model reveals a dilemma of policy interventions: intuitively, carbon taxes, asymmetric penalties (only imposed on un-low-carbon enterprises) and subsidies can improve the diffusion, which is consistent with previous research; counter-intuitively, all these policies turn out to be inefficient or even harmful to low-carbon enterprises because of the diffusion system's high adaptiveness. Specifically, when carbon taxes and penalties increase, both the low-carbon and un-low-carbon enterprises end up with the equilibrium of equivalent but lower profits. In contrast, all the enterprises earn equivalent but more profits even though subsidies are only given to the low-carbon enterprises, which implies that the un-low-carbon enterprises (indirectly) grab a portion of the subsidies and accordingly weaken the incentive effect of subsidies. These system behaviors are summarized as "equalizing effect" because the system tends to equalize the impact of both positive and negative interventions among all enterprises even though the policy interventions are asymmetrically imposed on one type of the enterprises. The findings also indicate that policies implemented to enlarge green market sizes can help policymakers bypass the dilemma.

Agent-based modeling↗