Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “games”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Unobtrusive Sensor-Based Occupancy Facing Direction Detection and Tracking Using Advanced Machine Learning Algorithms

Facing direction detection plays a critical role in human-computer interaction and has a wide application in surveillance systems, driving awareness recognition, smart home appliances, computer games, and so on. Current detection methods are mainly focused on extracting specific patterns from user's optical images, which raises concerns on privacy invasion and these detection techniques do not usually work in a dark environment. To address these concerns, here this paper proposes an activity recognition system guided by an unobtrusive sensor (ARGUS). By using a low pixel infrared thermopile array sensor, ARGUS is capable of identifying five facing directions (left 45°/90°, right 45°/90°, and front) through the support vector machine classifier. Also two feature extraction methods are compared. One is manually-defined and the other is based on a pre-trained convolutional neural network (CNN) model. The facing direction detection accuracy resulting from manually-defined features reaches 85.3%, 90.6%, and 85.2% at detection distance of 0.6, 1.2, and 1.8 m, respectively. The level of accuracy resulted from using pre-trained CNN features demonstrates a much more reliable performance (89.1%, 95.3%, and 95.1% at distance of 0.6, 1.2, and 1.8 m, respectively). In addition, ARGUS has been successfully applied for occupancy tracking with a root mean square error of 0.19 m.

42 ENGINEERING↗
DOE OSTI2018

Dynamic Power Distribution System Management With a Locally Connected Communication Network

Coordinated optimization and control of distribution-level assets can enable a reliable and optimal integration of massive amount of distributed energy resources (DERs) and facilitate distribution system management (DSM). Accordingly, the objective is to coordinate the power injection at the DERs to maintain certain quantities across the network, e.g., voltage magnitude, line flows, or line losses, to be close to a desired profile. By and large, the performance of the DSM algorithms has been challenged by two factors: i) the possibly non-strongly connected communication network over DERs that hinders the coordination; ii) the dynamics of the real system caused by the DERs with heterogeneous capabilities, time-varying operating conditions, and real-time measurement mismatches. In this paper, we investigate the modeling and algorithm design and analysis with the consideration of these two factors. In particular, a game theoretic characterization is first proposed to account for a locally connected communication network over DERs, along with the analysis of the existence and uniqueness of the Nash equilibrium (NE) therein. To achieve the equilibrium in a distributed fashion, a projected-gradient-based asynchronous DSM algorithm is then advocated. The algorithm performance, including the convergence speed and the tracking error, is analytically guaranteed under the dynamic setting. In conclusion, extensive numerical tests on both synthetic and realistic cases corroborate the analytical results derived.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

Defense strategies for asymmetric networked systems under composite utilities

We consider an infrastructure of networked systems with discrete components that can be reinforced at certain costs to guard against attacks. The communications network plays a critical, asymmetric role of providing the vital connectivity between the systems. We characterize the correlations within this infrastructure at two levels using (a) aggregate failure correlation function that specifies the infrastructure failure probability giventhe failure of an individual system or network, and (b) first order differential conditions on system survival probabilities that characterize component-level correlations. We formulate an infrastructure survival game between an attacker and a provider, who attacks and reinforces individual components, respectively. They use the composite utility functions composed of a survival probability term and a cost term, and the previously studiedsum-form and product-form utility functions are their special cases. At Nash Equilibrium, we derive expressions for individual system survival probabilities and the expected total number of operational components. We apply and discuss these estimates for a simplified model of distributed cloud computing infrastructure

Rao, Nageswara S.↗
DOE OSTI2015

A Distributed Cooperative Power Allocation Method for Campus Buildings

We propose a coordination algorithm for cooperative power allocation among a collection of commercial buildings within a campus. We introduced thermal and power models of a typical commercial building Heating, Ventilation, and Air Conditioning (HVAC) system, and utilize model predictive control to characterize their power flexibility. The power allocation problem is formulated as a cooperative game using the Nash Bargaining Solution (NBS) concept, in which buildings collectively maximize the product of their utilities subject to their local flexibility constraints and a total power limit set by the campus coordinator. To solve the optimal allocation problem, a distributed protocol is designed using dual decomposition of the Nash bargaining problem. Numerical simulations are performed to demonstrate the efficacy of our proposed allocation method

Hao, He↗
DOE OSTI2017

Using simulators to assess knowledge and behavior of “novice” operators of critical infrastructure under cyberattack events

—The transition from the traditional power grid to the smart grid improves reliability, performance, and management, while simultaneously increasing its susceptibility to cyberattacks. One of the biggest weaknesses in grid security is the human agent responsible for its maintenance and operations. As such, it is crucial to embed hands-on understanding of grid operations and security, especially for students in the Electrical and Computer Engineering (ECE) fields. This paper presents a case study where ECE students from Temple University used an interactive microgrid simulator, "Grid Game", a program developed by Idaho National Laboratory. This case study offers insights into the ECE students’ understanding of key engineering principles (microgrid stability, generation control system, generator inertia, energy storage, and grid security) gained by using the simulator program. Furthermore, the human behavior (strategies to prepare for and respond to cyberattacks, and cooperation and conflict in decision-making) of defenders/ECE students as they experience cyberattacks are also discussed. The paper then offers some limitations and possible suggestions for future research.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

Stochastic Dual Algorithm for Voltage Regulation in Distribution Networks with Discrete Loads

This paper considers power distribution networks with distributed energy resources and designs an incentive-based algorithm that allows the network operator and customers to pursue given operational and economic objectives while concurrently ensuring that voltages are within prescribed limits. Heterogeneous DERs with continuous and discrete control commands are considered. We address four major challenges: discrete decision variables, non-convexity due to a Stackelberg game structure, unavailability of private information from customers, and asynchronous operation. Starting from a non-convex setting, we develop a distributed stochastic dual algorithm that solves a relaxed problem, and prove that the proposed algorithm achieves the global optimal solution of the original problem on average. Feasible values for discrete decision variables are also recovered. Stability of the algorithm is analytically established and numerically corroborated.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

On defense strategies for system of systems using aggregated correlations

We consider a System of Systems (SoS) wherein each system Si, i = 1; 2; ... ;N, is composed of discrete cyber and physical components which can be attacked and reinforced. We characterize the disruptions using aggregate failure correlation functions given by the conditional failure probability of SoS given the failure of an individual system. We formulate the problem of ensuring the survival of SoS as a game between an attacker and a provider, each with a utility function composed of asurvival probability term and a cost term, both expressed in terms of the number of components attacked and reinforced. The survival probabilities of systems satisfy simple product-form, first-order differential conditions, which simplify the Nash Equilibrium (NE) conditions. We derive the sensitivity functions that highlight the dependence of SoS survival probability at NE on cost terms, correlation functions, and individual system survival probabilities.We apply these results to a simplified model of distributed cloud computing infrastructure.

Rao, Nageswara S.↗
DOE OSTI2021

Explainable AI for a No-Teardown Vehicle Component Cost Estimation: A Top-Down Approach

The broader ambition of this article is to popularize an approach for the fair distribution of the quantity of a system's output to its subsystems while allowing for underlying complex subsystem level interactions. Particularly, we present the use of this framework on a very specific (but generalizable) application, interesting for a more general AI audience. Here, we detail a data-driven approach to vehicle price modeling and its component price estimation by leveraging a combination of concepts from machine learning and game theory. We show an alternative to common teardown methodologies and surveying approaches for component and vehicle price estimation at the manufacturer's suggested retail price (MSRP) level that has the advantage of bypassing uncertainties involved in gathering teardown data, the need to perform expensive and biased surveying, and the need to perform retail price equivalent or indirect cost multiplier adjustments to mark up direct manufacturing costs to MSRP. This novel exercise not only provides accurate pricing of the technologies at the customer level, but also shows the, a priori known, large gaps in pricing strategies between manufacturers, vehicle classes, market segments, etc. There is also clear interaction between the price of technologies and other specifications present in vehicles. Those results are indication that old methods of manufacturer-level component costing, aggregation, and application of flat and rigid adjustment factors should be carefully examined. The findings are based on a database developed by Argonne, which includes over 64,000 vehicles covering MY1990 to MY2020 with hundreds of vehicle specs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Suppressing Screening-Current-Induced Strain in a 72-mm-Bore REBCO Insert for a 20-T Magnet: A Numerical Study

Rare-earth barium copper oxide (REBCO) high-temperature superconducting (HTS) magnets are considered a game changer for the capability of generating magnetic fields exceeding 20 T at or above liquid helium temperature, combined with the potential for substantially reduced manufacturing and operational costs. Princeton Plasma Physics Laboratory is dedicated to developing large-bore high-field superconducting magnet systems to support forefront physics research. Here, our current project focuses on a φ72-mm cold-bore REBCO insert comprising 21 dry-wound double-pancake coils, designed to generate at least 8 T when nested within a 12-T outsert magnet. A significant challenge for high-field REBCO magnets is the time-varying-magnetic-field-induced screening currents (SC) in the REBCO conductors, which can cause localized strain and stress concentrations. This paper presents a numerical study of the SC-induced strain in the REBCO insert magnet, confirming that the SC-induced strain can be substantially suppressed by energizing the REBCO insert before the outsert magnets. We discuss and reveal the mechanism behind this reduction. By applying this strategy, we expect to unleash the potential of the REBCO insert magnet to generate up to 12 T in a 12-T background field.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2015

Quantifying Mixed Uncertainties in Cyber Attacker Payoffs

Representation and propagation of uncertainty in cyber attacker payoffs is a key aspect of security games. Past research has primarily focused on representing the defender’s beliefs about attacker payoffs as point utility estimates. More recently, within the physical security domain, attacker payoff uncertainties have been represented as Uniform and Gaussian probability distributions, and intervals. Within cyber-settings, continuous probability distributions may still be appropriate for addressing statistical (aleatory) uncertainties where the defender may assume that the attacker’s payoffs differ over time. However, systematic (epistemic) uncertainties may exist, where the defender may not have sufficient knowledge or there is insufficient information about the attacker’s payoff generation mechanism. Such epistemic uncertainties are more suitably represented as probability boxes with intervals. In this study, we explore the mathematical treatment of such mixed payoff uncertainties.

Chatterjee, Samrat↗
DOE OSTI2024

Buyers Collusion in Incentivized Forwarding Networks: A Multi-Agent Reinforcement Learning Study

We present the issue of monetarily incentivized forwarding in a multi-hop mesh network architecture from an economic perspective. It is anticipated that credit-incentivized forwarding and relaying will be a simple method of exchanging transmission power and spectrum for connectivity. However, gateways and forwarding nodes, like any other free market, may create an oligopolistic market for the users they serve. In this study, a coalition scheme between buyers aims to address price control by gateways or nodes closer to gateways. In a Stackelberg competition game, buyer agents (users) and sellers (gateways) make decisions using reinforcement learning (RL), with decentralized Deep Q-Networks to buy and sell forwarding resources. We allow communication links between the buyers with a limited messaging space, without defining a collusion mechanism. The idea is to demonstrate that through messaging, and RL tacit collusion can emerge between agents in a decentralized setup. The multi-agent reinforcement learning (MARL) system is presented and analyzed from a machine-learning perspective. Moreover, MARL dynamics are discussed via mean field analysis to better understand divergence causes and make implementation recommendations for such systems. Finally, the simulation results show the results of coordination among the users.

42 ENGINEERING↗
DOE OSTI2022

Interference Moral Hazard in Large Multihop Networks

Cooperation between network nodes is critical for supporting services in ad hoc networks. Cooperation, however, is an idealized assumption that may not always be present. This assumption can fail because of moral hazard, a scenario in part caused by misaligned incentives between the requesting node and supporting node. In this paper, we characterize a moral hazard that perversely incentivizes nodes to increase their routing payments by transmitting interference into the multi-hop network. We refer to this as the interference moral hazard (IMH) problem which is inherent to strategyproof mechanisms with low overpayments. We investigate IMH as a non-cooperative game played by network nodes on a random graph. For large networks, we show that IMH can be solved in the network design space. Finally, we provide sufficient conditions on the network distribution that guarantee an equilibrium path with interference-free play. This is achieved by 1) lower-bounding the number of nodes and 2) bounding the network density slightly above the 2-connectedness threshold and below a proposed upper-bound. Simulations suggest that density plays a fundamental role in IMH.

42 ENGINEERING↗
DOE OSTI2023

Challenges and Opportunities in Deep Reinforcement Learning With Graph Neural Networks: A Comprehensive Review of Algorithms and Applications

Deep reinforcement learning (DRL) has empowered a variety of artificial intelligence fields, including pattern recognition, robotics, recommendation-systems, and gaming. Similarly, graph neural networks (GNN) have also demonstrated their superior performance in supervised learning for graph-structured data. In recent times, the fusion of GNN with DRL for graph-structured environments has attracted a lot of attention. Here, this paper provides a comprehensive review of these hybrid works. These works can be classified into two categories: (1) algorithmic enhancement, where DRL and GNN complement each other for better utility; (2) application-specific enhancement, where DRL and GNN support each other. This fusion effectively addresses various complex problems in engineering and life sciences. Based on the review, we further analyze the applicability and benefits of fusing these two domains, especially in terms of increasing generalizability and reducing computational complexity. Finally, the key challenges in integrating DRL and GNN, and potential future research directions are highlighted, which will be of interest to the broader machine learning community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Resource Adequacy in Electricity Markets With Renewable Energy

This paper presents a market-based resource adequacy assessment framework to analyze the system generation portfolio that results in a competitive market environment. In particular, we apply the modeling framework to investigate the impact of high penetration levels of variable renewable energy resources and different market designs on achieving resource adequacy. The model captures the strategic capacity investment and retirement decision-making of profit-maximizing generation companies. In contrast to many previous studies, this paper considers markets for capacity, energy, and reserve products in detail; thus, the modeling framework allows improved analysis of generation expansion and revenue sufficiency in a competitive market environment. The modeling framework is based on Stackelberg leader-follower games and is formulated as a bi-level optimization problem, which is then transformed into a mathematical program with equilibrium constraints. Furthermore, we employ a Lagrangian decomposition algorithm to enhance computational performance. In a case study of the electric reliability council of texas (ERCOT) system, we compare the results from the proposed model with those obtained from a traditional centralized least-cost planning model. Finally, the results demonstrate the effectiveness of the proposed modeling framework and highlight the importance of considering strategic behavior in a competitive market framework when assessing resource adequacy.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2019

Mechanism Design for Demand Response Programs

Demand response (DR) programs serve to reduce the consumption of electricity at times when the supply is scarce and expensive. Consumers or agents with flexible consumption profiles are recruited by an aggregator who manages the DR program. The aggregator calls on a subset of its pool of recruited agents to reduce their electricity use during DR events. Agents are paid for reducing their energy consumption from contractually established baselines. Baselines are counter-factual consumption estimates of the energy an agent would have consumed if they were not participating in the DR program. Baselines are used to determine payments to agents. This creates an incentive for agents to inflate their baselines in order to increase the payments they receive. There are several newsworthy cases of agents gaming their baseline for economic benefit. We propose a novel self-reported baseline mechanism (SRBM) where each agent reports its baseline and marginal utility. These reports are strategic and need not be truthful. Based on the reported information, the aggregator selects or calls on agents with a certain probability to meet the load reduction target. Called agents are paid for observed reductions from their self-reported baselines. Agents who are not called face penalties for consumption shortfalls below their baselines. Under SRBM, we show that truthful reporting of baseline consumption and marginal utility is a dominant strategy. Thus, SRBM eliminates the incentive for agents to inflate baselines. SRBM is assured to meet the load reduction target. To conclude, we show that SRBM is almost optimal in the metric of average cost of DR provision faced by the aggregator.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several subnetworks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each subnetwork is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33-and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Optimal Transactive Energy Trading of Electric Vehicle Charging Stations With On-Site PV Generation in Constrained Power Distribution Networks

This paper presents a two-level transactive energy market framework, that enables energy trading among electric vehicle charging stations (EVCSs). At the lower level, the discharging capability of EVs and on-site PV generation are leveraged by individual EVCS for participating in the transactive trading with their peers. Once the lower-level trading is completed, EVCSs trade energy at the upper level through the power grid network managed by the distribution system operator (DSO). The upper-level market is cleared while satisfying the power distribution network constraints. A cooperative game-based model is proposed to model the energy trading among EVCSs. To this end, the asymmetric Nash bargaining method is applied to allocate the grand coalition's payoff to each EVCS at the upper-level market, while a weighted proportional allocation method is used to allocate individual EVCS's payoff to its respective EVs at the lower-level market. In this work, the upper-level market formulation is further decomposed into two subproblems representing an energy scheduling and trading subproblem which maximizes EVCS payoffs, and a bargaining subproblem which allocates EVCS payoffs. The effectiveness of the proposed framework for incentivizing transactive trades among EVs and EVCSs is validated in case studies.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs

This paper proposes attention enabled multi-agent deep reinforcement learning (MADRL) framework for active distribution network decentralized Volt-VAR control. Using the unsupervised clustering, the whole distribution system can be decomposed into several sub-networks according to the voltage and reactive power sensitivity relationships. Then, the distributed control problem of each sub-network is modeled as Markov games and solved by the improved MADRL algorithm, where each sub-network is modeled as an adaptive agent. An attention mechanism is developed to help each agent focus on specific information that is mostly related to the reward. All agents are centrally trained offline to learn the optimal coordinated Volt-VAR control strategy and executed in a decentralized manner to make online decisions with only local information. Compared with other distributed control approaches, the proposed method can effectively deal with uncertainties, achieve fast decision makings, and significantly reduce the communication requirements. Comparison results with model-based and other data-driven methods on IEEE 33-bus and 123-bus systems demonstrate the benefits of the proposed approach.

distribution network↗