Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Games”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Interference Moral Hazard in Large Multihop Networks

Cooperation between network nodes is critical for supporting services in ad hoc networks. Cooperation, however, is an idealized assumption that may not always be present. This assumption can fail because of moral hazard, a scenario in part caused by misaligned incentives between the requesting node and supporting node. In this paper, we characterize a moral hazard that perversely incentivizes nodes to increase their routing payments by transmitting interference into the multi-hop network. We refer to this as the interference moral hazard (IMH) problem which is inherent to strategyproof mechanisms with low overpayments. We investigate IMH as a non-cooperative game played by network nodes on a random graph. For large networks, we show that IMH can be solved in the network design space. Finally, we provide sufficient conditions on the network distribution that guarantee an equilibrium path with interference-free play. This is achieved by 1) lower-bounding the number of nodes and 2) bounding the network density slightly above the 2-connectedness threshold and below a proposed upper-bound. Simulations suggest that density plays a fundamental role in IMH.

42 ENGINEERING↗
DOE OSTI2023

Challenges and Opportunities in Deep Reinforcement Learning With Graph Neural Networks: A Comprehensive Review of Algorithms and Applications

Deep reinforcement learning (DRL) has empowered a variety of artificial intelligence fields, including pattern recognition, robotics, recommendation-systems, and gaming. Similarly, graph neural networks (GNN) have also demonstrated their superior performance in supervised learning for graph-structured data. In recent times, the fusion of GNN with DRL for graph-structured environments has attracted a lot of attention. Here, this paper provides a comprehensive review of these hybrid works. These works can be classified into two categories: (1) algorithmic enhancement, where DRL and GNN complement each other for better utility; (2) application-specific enhancement, where DRL and GNN support each other. This fusion effectively addresses various complex problems in engineering and life sciences. Based on the review, we further analyze the applicability and benefits of fusing these two domains, especially in terms of increasing generalizability and reducing computational complexity. Finally, the key challenges in integrating DRL and GNN, and potential future research directions are highlighted, which will be of interest to the broader machine learning community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several subnetworks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each subnetwork is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33-and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Optimal Transactive Energy Trading of Electric Vehicle Charging Stations With On-Site PV Generation in Constrained Power Distribution Networks

This paper presents a two-level transactive energy market framework, that enables energy trading among electric vehicle charging stations (EVCSs). At the lower level, the discharging capability of EVs and on-site PV generation are leveraged by individual EVCS for participating in the transactive trading with their peers. Once the lower-level trading is completed, EVCSs trade energy at the upper level through the power grid network managed by the distribution system operator (DSO). The upper-level market is cleared while satisfying the power distribution network constraints. A cooperative game-based model is proposed to model the energy trading among EVCSs. To this end, the asymmetric Nash bargaining method is applied to allocate the grand coalition's payoff to each EVCS at the upper-level market, while a weighted proportional allocation method is used to allocate individual EVCS's payoff to its respective EVs at the lower-level market. In this work, the upper-level market formulation is further decomposed into two subproblems representing an energy scheduling and trading subproblem which maximizes EVCS payoffs, and a bargaining subproblem which allocates EVCS payoffs. The effectiveness of the proposed framework for incentivizing transactive trades among EVs and EVCSs is validated in case studies.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs

This paper proposes attention enabled multi-agent deep reinforcement learning (MADRL) framework for active distribution network decentralized Volt-VAR control. Using the unsupervised clustering, the whole distribution system can be decomposed into several sub-networks according to the voltage and reactive power sensitivity relationships. Then, the distributed control problem of each sub-network is modeled as Markov games and solved by the improved MADRL algorithm, where each sub-network is modeled as an adaptive agent. An attention mechanism is developed to help each agent focus on specific information that is mostly related to the reward. All agents are centrally trained offline to learn the optimal coordinated Volt-VAR control strategy and executed in a decentralized manner to make online decisions with only local information. Compared with other distributed control approaches, the proposed method can effectively deal with uncertainties, achieve fast decision makings, and significantly reduce the communication requirements. Comparison results with model-based and other data-driven methods on IEEE 33-bus and 123-bus systems demonstrate the benefits of the proposed approach.

distribution network↗
DOE OSTI2022

Secure mmWave Spectrum Sharing with Autonomous Beam Scheduling for 5G and Beyond

Spectrum Sharing (SS) has seen a renewed set of initiatives in 5G with the availability of shared and unlicensed spectrum bands that can be used by multiple cellular service providers and private cellular networks. Beam based transmission, instead of the traditional sector based transmission in conjunction with the spectrum agility of the 5G New Radio (NR) has brought new opportunities to optimized sharing of spectrum. Currently in the U.S., a centralized Spectrum Access Server (SAS) is used to co-ordinate spectrum sharing among networks sharing the same spectrum band. However, SAS becomes a focal point for security attacks and a performance bottleneck. In addition, SAS relies on an Environmental Sensor Network (ESN), separate from the 5G network. Without trusted spectral occupancy information, false reporting of spectrum sensing data can create sub-optimal and unfair spectrum usage. This paper summarizes our recent research findings in using a decentralized scheme for multiple networks to securely share spectrum with autonomous beam scheduling : 1) A new stochastic network framework based on Lyapunov Optimization approach is developed to optimize scheduling at the base stations; 2) Game theoretic (GT) approach is used to formulate the distributed scheduler; 3) Another distributed scheduler with Q-learning is presented that utilizes the Reinforcement Learning (RL) approach; 4) The performance and convergence rate of these distributed solutions to use shared and unlicensed spectrum are compared with existing solutions. Conditions under which the performance of these schedulers approach the theoretical upper bound, which is the performance possible with no interference among the operators sharing the spectrum, are presented; 5) The ability of a base station to use its own user equipment as sensors, for optimal spectrum sharing with base stations in other operator networks, is demonstrated to be an effective approach.

5G↗
DOE OSTI2020

Effects of social structure and management on risk of disease establishment in wild pigs

Contact heterogeneity among hosts determines invasion and spreading dynamics of infectious disease, thus its characterization is essential for identifying effective disease control strategies. Yet, little is known about the factors shaping contact networks in many 28 wildlife species and how wildlife management actions might affect contact networks. Wild pigs in North America are an invasive, socially-structured species that pose a health concern for domestic swine given their ability to transmit numerous devastating diseases 2 such as African swine fever (ASF). Using proximity loggers and GPS data from 48 wild pigs in Florida and South Carolina, USA, we employed a probabilistic framework to estimate weighted contact networks. We determined the effects of sex, social group, and spatial distribution (monthly home range overlap and distance) on wild pig contact. We also estimated the impacts of management-induced perturbations on contact and inferred their effects on ASF establishment in wild pigs with simulation. Social group membership was the primary factor influencing contacts. Between-group contacts depended primarily on space use characteristics, with fewer contacts among groups separated by >2 km and no contacts among groups >4 km apart within a month. Modeling ASF dynamics on the contact network demonstrated that indirect contacts resulting from baiting (a typical method of attracting wild pigs or game species to a site to enhance recreational hunting) increased the risk of disease establishment by ~33% relative to direct contact. Low-intensity population reduction (<5.9% of the population) had no detectable impact on contact structure but reduced predicted ASF establishment risk relative to no population reduction. Here, we demonstrate an approach for understanding the relative role of spatial, social, and individual-level characteristics in shaping contact networks and predicting their effects on disease establishment risk, thus providing insight for optimizing disease control in spatially- and socially-structured wildlife species.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Age and growth of early‐life‐stage Atlantic tarpon ( Megalops atlanticus) from the northcentral Gulf of Mexico

Abstract Age and growth of early‐life‐stage Atlantic tarpon Megalops atlanticus collected from Mississippi coastal waters in the northcentral Gulf of Mexico (GOM) are described using otolith microstructure analysis. Tarpon leptocephali ( n = 95, 16.0—27.8 mm standard length, L S ) collected from June throughOctober 2013—2018, ranged in age from 22 to 43 days (mean = 30.9 ± 0.5 days). Leptocephalus somatic growth rates ranged 0.46—1.24 mm day −1 (mean = 0.76 ± 0.02 mm day −1 ), and leptocephalus otolith growth rates ranged 1.78—3.97 μm day −1 (mean = 2.58 ± 0.04 μm day −1 ). Growth rates were inversely correlated to leptocephalus age, indicating the shrinkage phase associated with leptocephalus metamorphosis. Juvenile tarpon ( n = 358, 50—359 mm fork length, L F ) were collected from August through December 2007—2018. Juveniles exhibited a positive allometric relationship (adjusted R 2 = 0.99, P < 0.001) between length and mass. The age of 100 juveniles (71—277 mm L F ) ranged from 76 to 174 days. Juvenile growth rate was estimated as 1.56 ± 0.11 mm day −1 . Significant ( P < 0.001) linear relationships were found between juvenile age and otolith metrics, including otolith mass ( R 2 = 0.81) and radius ( R 2 = 0.68). Evaluation of the backcalculated hatch dates suggests that specimens in the collection hatched from late May through mid‐September with slight peaks during July and August. A Rao's Spacing Test of Uniformity indicates the presence of significant lunar periodicity in leptocephalus hatch dates ( n = 95, U = 250.1, P < 0.05), with 50% of the leptocephali hatched within 5 days (before or after) of the full moon. This study fills critical gaps in the scientific knowledge of tarpon and provides estimates of early‐life‐history metrics for an iconic game fish at the northernmost extent of its GOM range.

Graham, Patrick M.↗
DOE OSTI2024

Effects of human presence on African mammal waterhole attendance and temporal activity patterns

Abstract Human impacts on the environment and wildlife populations are increasing globally, threatening thousands of species with extinction. While wildlife‐based tourism is beneficial for educating tourists, generating income for conservation efforts, and providing local employment, more information is needed to understand how this industry may impact wildlife. In this study, we used motion‐activated cameras at 12 waterholes on a private game reserve in northern Namibia to determine if the presence of humans and permanent infrastructure affected mammal visits by examining their (1) number of visits, (2) time spent, and (3) diel activity patterns. Our results revealed no differences in the number of visits based on human presence for any of the 17 mammal species studied. However, giraffes ( Giraffe camelopardalis ) spent more time at waterholes before observer presence compared to during. Additionally, several species changed diel activity patterns when human observers were present. Notably, several carnivore and ungulate species increased overlap in their activity patterns during periods while humans were present relative to when humans were absent. These modifications of mammal temporal activity patterns due to human presence could eventually lead to changes in community structure and trophic dynamics because of altered predator–prey interactions. As humans continue to expand into wildlife habitats, and wildlife‐based tourism increases globally, it is imperative that we fully understand the effects of anthropogenic pressures on mammal behavior. Monitoring of wildlife behavioral changes in response to human activity is crucial to further develop wildlife tourism opportunities in a way that optimizes the impact of conservation goals.

Patterson, J. R. [Savannah River Ecology Lab, Warn↗
DOE OSTI2021

The effects of allelochemical transfer on the dynamics of hosts, parasitoids, and competing hyperparasitoids

Allelochemicals produced by plants may be ingested by herbivorous insects and transferred to higher trophic levels with potentially deleterious effects. We develop a system of differential equations to investigate the effect of the transfer of allelochemicals, such as nicotine, on the population dynamics of a system of hosts, parasitoids, and two competing hyperparasitoids that attack different life stages of the parasitoids. We find both somewhat deleterious effects of nicotine on the larvae-attacking hyperparasitoids and increased attack rates for the pupae-attacking hyperparasitoids can promote coexistence. We also use an evolutionary game-theoretic approach to determine the optimal distribution of hyperparasitoid attacks among nicotine-producing and nicotine-free plants. With strong deleterious effects of nicotine and increased attack rates for the pupae-attacking hyperparasitoid, we find both species attack parasitoids on the nicotine-free plant but only pupae-attacking hyperparasitoids attack parasitoids on the nicotine-producing plant.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

BI-LEVEL OPTIMIZATION FOR ELECTRICITY TRANSACTION IN SMART COMMUNITY WITH MODULAR PUMP HYDRO STORAGE

Grid integration of the increasing distributed energy resources could be challenging in terms of new infrastructure investment, power grid stability, etc. To resolve more renewables locally and reduce the need for extensive electricity transmission, a community energy transaction market is assumed with market operator as the leader whose responsibility is to generate local energy prices and clear the energy transaction payment among the prosumers (followers). The leader and multi-followers have competitive objectives of revenue maximization and operational cost minimization. This non-cooperative leader-follower (Stackelberg) game is formulated using a bi-level optimization framework, where a novel modular pump hydro storage technology (GLIDES system) is set as an upper level market operator, and the lower level prosumers are nearby commercial buildings. The best responses of the lower level model could be derived by necessary optimality conditions, and thus the bi-level model could be transformed into single level optimization model via replacing the lower level model by its Karush-Kuhn-Tucker (KKT) necessary conditions. Several experiments have been designed to compare the local energy transaction behavior and profit distribution with the different demand response levels and different local price structures. The experimental results indicate that the lower level prosumers could benefit the most when local buying and selling prices are equal, while maximum revenue potential for the upper level agent could be reached with non-equal trading prices.

Chen, Yang↗
DOE OSTI2025

Neuromorphic ionic computing in droplet interface synapses

Ionic devices with memory capabilities can emulate neural functionality, enabling neuromorphic computing and biomedical applications. In this study, we report an ionic spiking synapse based on aqueous droplet interface bilayer assembly. Under stepwise triangular voltages, the device displays coupled memcapacitive-memristive behavior, showing noncrossing pinched hysteretic I-V loops. This hysteretic ion dynamics can be regulated by modifying bilayer components, reconstituting protein channels, or adjusting droplet assembly configuration. Droplet interface synapses (DIS) exhibit fundamental neuromorphic behaviors such as paired-pulse facilitation/depression, spike rate–dependent plasticity, Hebbian learning, and short-term associative learning under classical conditioning. We also used reservoir computing with DIS to implement two learning algorithms: a classification algorithm that recognizes handwritten digits and a reinforcement learning algorithm that learns to play a board game of tic-tac-toe.

Li, Zhongwu [Lawrence Livermore National Laborator↗
DOE OSTI2022

Perovskite solar cells can take the heat

Solar panels must endure extreme conditions during their service lifetime. There are few other products, let alone pristine semiconductor electronics, that are manufactured today and expected to withstand decades of abuse from full exposure to outdoor elements, including harsh ultraviolet sunlight, rain and hail, and daily and seasonal thermal cycles. Over the past decade, a new class of semiconductors, known as metal halide perovskites, has emerged, which constitutes the light-absorbing material in solar panels and has made incredible advancements, potentially taking photovoltaics in new directions. Perovskite materials have been touted as a potential game changer for solar energy production.

14 SOLAR ENERGY↗
DOE OSTI2024

Leveraging Multitime Hamilton–Jacobi PDEs for Certain Scientific Machine Learning Problems

Hamilton-Jacobi partial differential equations (HJ PDEs) have deep connections with a wide range of fields, including optimal control, differential games, and imaging sciences. By considering the time variable to be a higher dimensional quantity, HJ PDEs can be extended to the multi-time case. In this paper, we establish a novel theoretical connection between specific optimization problems arising in machine learning and the multi-time Hopf formula, which corresponds to a representation of the solution to certain multi-time HJ PDEs. Through this connection, we increase the interpretability of the training process of certain machine learning applications by showing that when we solve these learning problems, we also solve a multi-time HJ PDE and, by extension, its corresponding optimal control problem. As a first exploration of this connection, we develop the relation between the regularized linear regression problem and the Linear Quadratic Regulator (LQR). We then leverage our theoretical connection to adapt standard LQR solvers (namely, those based on the Riccati ordinary differential equations) to design new training approaches for machine learning. Lastly, we provide some numerical examples that demonstrate the versatility and possible computational advantages of our Riccati-based approach in the context of continual learning, post-training calibration, transfer learning, and sparse dynamics identification.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Foundations of automatic feature extraction at LHC–point clouds and graphs

Abstract Deep learning algorithms will play a key role in the upcoming runs of the Large Hadron Collider (LHC), helping bolster various fronts ranging from fast and accurate detector simulations to physics analysis probing possible deviations from the Standard Model. The game-changing feature of these new algorithms is the ability to extract relevant information from high-dimensional input spaces, often regarded as “replacing the expert” in designing physics-intuitive variables. While this may seem true at first glance, it is far from reality. Existing research shows that physics-inspired feature extractors have many advantages beyond improving the qualitative understanding of the extracted features. In this review, we systematically explore automatic feature extraction from a phenomenological viewpoint and the motivation for physics-inspired architectures. We also discuss how prior knowledge from physics results in the naturalness of the point cloud representation and discuss graph-based applications to LHC phenomenology.

Bhardwaj, Akanksha↗
DOE OSTI2023

Automated Adversary-in-the-Loop Cyber-Physical Defense Planning

Security of cyber-physical systems (CPS) continues to pose new challenges due to the tight integration and operational complexity of the cyber and physical components. To address these challenges, this article presents a domain-aware, optimization-based approach to determine an effective defense strategy for CPS in an automated fashion—by emulating a strategic adversary in the loop that exploits system vulnerabilities, interconnection of the CPS, and the dynamics of the physical components. Our approach builds on an adversarial decision-making model based on a Markov Decision Process (MDP) that determines the optimal cyber (discrete) and physical (continuous) attack actions over a CPS attack graph. The defense planning problem is modeled as a non-zero-sum game between the adversary and defender. We use a model-free reinforcement learning method to solve the adversary’s problem as a function of the defense strategy. We then employ Bayesian optimization (BO) to find an approximate best-response for the defender to harden the network against the resulting adversary policy. This process is iterated multiple times to improve the strategy for both players. We demonstrate the effectiveness of our approach on a ransomware-inspired graph with a smart building system as the physical process. Numerical studies show that our method converges to a Nash equilibrium for various defender-specific costs of network hardening.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Sudoku Online Testing Framework v.0.2

The Sudoku Online Testing Framework provides a configurable basis for testing user interactions to solve visualization-related problems with sudoku puzzles and games. It consists of a server, a client, and test code. The python server implements sudoku puzzle logic and board selection functions. The JavaScript client implements a graphical user interface (GUI) front-end to display the puzzles and facilitate interactions with users. Within the GUI, logical rules may be selected. Cells may be selected to “pivot” on to make a board for each possible value in the cell. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-6409 O

Leger, Michelle↗
DOE OSTI2021

Exploration ToolKit (ExTK)

The Exploration Toolkit (ExTK) is a reusable Extended Reality (XR) system developed for incorporating and exploring 3-Dimensional (3-D) computer aided design (CAD) models in XR, with a primary focus on Augmented Reality (AR). The ExTK consists of a Developer Mode and a User Mode. In Developer Mode, ExTK provides developers with the ability to easily import 3-D CAD models and activate desired exploration functionality and layout. Multiple models can be added to a single instantiation of the ExTK using Unity's Scene capability. Exploration functions include scaling, rotating, explode/contract, animations, hiding parts, submodules, and measurement functions. In User Mode, ExTK provides a menu system that allows users to select models and initiate exploration functions. ExTK is architected for reusability and developers can customize the ExTK layout and functions according to application needs. ExTK is designed to be hardware agnostic, although initial development focused on the Microsoft HoloLens as the primary deployment platform. The ExTK is developed using the Unity Game Engine Development Platform. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-1506 O

Klein, BrandonThorin↗