Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A Sequential Model Predictive and Deep Reinforcement Learning-Based Controller for Distribution System Outage Mitigation under Hurricane Events

This paper proposes a proactive outage mitigation framework for power distribution networks to withstand hurricane-induced disruptions. It leverages Model Predictive Control (MPC) to identify safe lines for proactive switching during hurricanes, minimizing the risk of cascading failures and voltage violations. The switching strategies optimized by MPC are sequentially integrated with a Deep Reinforcement Learning agent using the Advantage Actor-Critic algorithm, enabling dynamic line switching to maximize connected buses and minimize voltage violations in real time. Using a probabilistic hurricane model, the framework predicts line failures and adapts to varying conditions to enhance grid resilience. Simulations on the IEEE 123-bus system demonstrate its effectiveness in maintaining high connectivity and minimizing disruptions. Real-time testing with an RTDS confirms the practicality and reliability of the proposed approach.

Selim, Alaa [University of Connecticut]↗
DOE OSTI2026

Small-molecule modulation of β-arrestins

β-Arrestins are multifunctional regulators of G-protein-coupled receptor (GPCR) signalling and orchestrate diverse downstream signalling events and physiological responses across the GPCR superfamily. Although GPCR pharmacology has advanced to target orthosteric and allosteric sites, as well as G proteins and GPCR kinases, direct chemical tools to modulate β-arrestin activities have remained conspicuously absent. Here we report the identification of small-molecule inhibitors that selectively target β-arrestins and delineate their mechanism of action through integrated pharmacological, biochemical, biophysical and structural analyses. These inhibitors disrupt β-arrestin engagement with agonist-activated GPCRs, impairing desensitization, internalization and β-arrestin-dependent physiological functions while sparing G protein–receptor coupling. Cryo-electron microscopy, molecular dynamics simulations and structure-guided mutagenesis reveal that one modulator, Cmpd-5, engages a pocket within the central crest of β-arrestin1 formed by the middle, C and lariat loops, a critical receptor-binding interface, stabilizing a distinct conformation that is incompatible with full β-arrestin–receptor engagement. Together, these findings establish a mechanistic framework for β-arrestin modulation, reveal a novel allosteric site for structure-based drug design, and open new avenues for transducer-targeted, pathway-specific GPCR therapeutic agents.

Kahsai, Alem W. [Duke University, Durham, NC (Unit↗
DOE OSTI2024

Reinforcement learning pulses for transmon qubit entangling gates

The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Distributed deep reinforcement learning for simulation control

Abstract Several applications in the scientific simulation of physical systems can be formulated as control/optimization problems. The computational models for such systems generally contain hyperparameters, which control solution fidelity and computational expense. The tuning of these parameters is non-trivial and the general approach is to manually ‘spot-check’ for good combinations. This is because optimal hyperparameter configuration search becomes intractable when the parameter space is large and when they may vary dynamically. To address this issue, we present a framework based on deep reinforcement learning (RL) to train a deep neural network agent that controls a model solve by varying parameters dynamically. First, we validate our RL framework for the problem of controlling chaos in chaotic systems by dynamically changing the parameters of the system. Subsequently, we illustrate the capabilities of our framework for accelerating the convergence of a steady-state computational fluid dynamics solver by automatically adjusting the relaxation factors of the discretized Navier–Stokes equations during run-time. The results indicate that the run-time control of the relaxation factors by the learned policy leads to a significant reduction in the number of iterations for convergence compared to the random selection of the relaxation factors. Our results point to potential benefits for learning adaptive hyperparameter learning strategies across different geometries and boundary conditions with implications for reduced computational campaign expenses 4 4 Data and codes available at https://github.com/Romit-Maulik/PAR-RL . .

42 ENGINEERING↗
DOE OSTI2021

Reduced Order Model of Transactive Bidding Loads

Transactive energy (TE) has been identified to provide better grid efficiency and reliability by market-based transactive exchanges between energy producers and energy consumers. Simulations of TE systems are crucial to evaluate the benefits and impacts of different transactive mechanisms. However, such simulations can be time consuming due to the information exchange between various participants and complex co-simulation environments. In this paper, we develop a reduced order model to speed up the simulation of transactive systems in TE simulation platform (TESP) while achieving very low error between the reduced order and full model results. Specifically, the developed reduced order model consists of an aggregate responsive load agent which utilizes two Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTMs) to enable transactive elements to collectively participate in the TE system. The proposed aggregate responsive load (ARL) agent is able to produce similar transactive behaviors to the full simulation model while achieving significant simulation time reduction. Finally, we also show that the developed model enables generalization of simulation results across different dates and across different number of loads included in the simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2025

Localization of Ad-Hoc Lunar Constellations in Communication Failure Modes for Distributed Spacecraft Autonomy

As Lunar missions increase in complexity, inspired by NASA’s Artemis Program, they will require reliable and sufficient Position, Navigation, and Timing (PNT) capability to support the upcoming Lunar users. The navigation service should also be compatible with the smaller platforms, like CubeSats, being sent by the public and private sectors. A non-dedicated, ad-hoc Lunar navigation constellation can provide PNT services on-demand using the non-dedicated swarm assets. Swarm members cooperatively and autonomously localize themselves with minimal interaction from Earth, freeing up valuable bandwidth and ground segment resources. The autonomous localization of Lunar constellations utilizes neighbor two-way intersatellite link (ISL) measurements in a distributed extended Kalman filter (DEKF) system to minimize operating costs. Because the decentralized Lunar PNT system relies on relay communication amongst the agents, network failures or loss of assets among ad-hoc Lunar constellations may impact localization performance. This study presents an evaluation of localization performance under increasing levels of network degradation. A simulation of an ad-hoc Lunar PNT swarm is augmented to include system faults and the impacts of intermittent and permanent failures on localization performance are evaluated. We investigate three potential causes of network degradation: single spacecraft loss, multiple spacecraft loss, and antenna failure. The numerical assessments from the simulation show that the LPNT system under study, based on an autonomous decentralized concept of operation, is highly robust and resilient to communication failures. Minor faults, such as single spacecraft loss, solar interference, technical malfunctions, message delays, and antenna outages, have minimal impact on state estimation, with only a 4.47% and 3.75% degradation in median position error for assets and a representative ground user, respectively, compared to an ideal communication scenario. However, major faults, such as hardware failures or meteor strikes leading to the loss of multiple spacecrafts, are more concerning. The permanent loss of three spacecraft results in a more severe performance degradation, with median position error increasing by 23.3% for assets and 11.7% for a representative ground user, despite the Lunar PNT system remaining functional.

Yeji Kim↗
NASA NTRS2004

Design of Hybrid Mobile Communication Networks for Planetary Exploration

The Mobile Exploration System Project (MEX) at NASA Ames Research Center has been conducting studies into hybrid communication networks for future planetary missions. These networks consist of space-based communication assets connected to ground-based Internets and planetary surface-based mobile wireless networks. These hybrid mobile networks have been deployed in rugged field locations in the American desert and the Canadian arctic for support of science and simulation activities on at least six occasions. This work has been conducted over the past five years resulting in evolving architectural complexity, improved component characteristics and better analysis and test methods. A rich set of data and techniques have resulted from the development and field testing of the communication network during field expeditions such as the Haughton Mars Project and NASA Mobile Agents Project.

Alena, Richard L.↗
NASA NTRS2018

Playbook for UAS: UX of Goal-Oriented Planning & Execution

We are evaluating Playbook for CASAS (Connected Autonomous Smart Aerospace Systems), a tool designed to aid first responders in disaster relief efforts. We are adapting an existing tool, Playbook, to support a future unmanned aircraft system (UAS) swarm demonstration. Playbook for CASAS will be used to plan, edit, and monitor simulated UAS swarms, and we are interested in evaluating the user experience of this prototype as well as developing recommendations for future UAS interfaces. Allocation of roles and responsibilities between human-automation systems is key to promoting productive cooperation between users and automation. Future interfaces, however, must allow for adaptive management of the swarm not a constant split in human-automation control. Our early research indicates that when a single pilot is controlling swarms of robotic agents, such as UAS or ground rovers, operators require a higher level, goal-based interface with usability at its core. Along with that high-level control, users can leverage sensors within the swarm to be notified when lower level actions must be taken by the pilot. First responders working in disaster relief efforts require a high level of situational awareness (SA) and precise control at key moments within a mission. This balance in operator workload paired with SA can lead to improved safety and mission outcomes. Our research below outlines leverage points as well as the balance between human involvement and autonomy in UAS interfaces.

UAS↗
NASA NTRS2004

Web-Based Toxic Gas Dispersion Model for Shuttle Launch Operations

During the launch of the Space Shuttle vehicle, the burning of liquid hydrogen fuel with liquid oxygen at extreme high temperatures inside the three space shuttle main engines, and the burning of the solid propellant mixture of ammonium perchlorate oxidizer, aluminum fuel, iron oxide catalyst, polymer binder, and epoxy curing agent in the two solid rocket boosters result in the formation of a large cloud of hot, buoyant toxic exhaust gases near the ground level which subsequently rises and entrains into ambient air until the temperature and density of the cloud reaches an approximate equilibrium with ambient conditions. In this paper, toxic gas dispersion for various gases are simulated over the web for varying environmental conditions which is provided by rawinsonde data. The model simulates chemical concentration at ground level up to 10 miles (1 KM grids) in downrange up to an hour after launch. The ambient concentration of the gas dispersion and the deposition of toxic particles are used as inputs for a human health risk assessment model. The advantage of the present model is the accessibility and dissemination of model results to other NASA centers over the web. The model can be remotely operated and various scenarios can be analyzed.

Bardina, Jorge↗
DOE OSTI2023

Autonomous Cyber Defense Against Dynamic Multi-strategy Infrastructural DDoS Attacks

Dynamic Infrastructural Distributed Denial of Service (I-DDoS) attacks constantly change attack vectors to congest core backhaul links and disrupt critical network availability while evading end-system defenses. To effectively counter these highly dynamic attacks, defense mechanisms need to exhibit adaptive decision strategies for real-time mitigation. This paper presents a novel Autonomous DDoS Defense framework that employs model-based reinforcement agents. The framework continuously learns attack strategies, predicts attack actions, and dynamically determines the optimal composition of defense tactics such as filtering, limiting, and rerouting for flow diversion. Our contributions include extending the underlying formulation of the Markov Decision Process (MDP) to address simultaneous DDoS attack and defense behavior, and accounting for environmental uncertainties. We also propose a fine-grained action mitigation approach robust to classification inaccuracies in Intrusion Detection Systems (IDS). Additionally, our reinforcement learning model demonstrates resilience against evasion and deceptive attacks. Evaluation experiments using real-world and simulated DDoS traces demonstrate that our autonomous defense framework ensures the delivery of approximately 96 - 98% of benign traffic despite the diverse range of attack strategies.

Dutta, Ashutosh↗
NASA NTRS2022

The Turning Point of the Aerosol Era

Over the CMIP6 historical period (1850–2014), aerosols provided the largest negative forcing compared to all other climate forcings via their ability to absorb or scatter solar radiation and alter clouds. Aerosols played an important role in counterbalancing warming by greenhouse gases (GHGs). Here we study aerosol forcing trends in the CMIP6 simulations of the NASA Goddard Institute for Space Studies (GISS) ocean-atmosphere ModelE version 2.1 (GISS-E2.1-G) using a fully coupled atmospheric composition configuration, including interactive gas-phase chemistry, and either an aerosol microphysical (MATRIX) or a mass-based aerosol (OMA) module. Simulations of the CMIP6 historical period are analyzed as well as four Shared Socioeconomic Pathway (SSP) future scenarios for 2015–2100: SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5. The main conclusion of this study is that aerosol forcing in the GISS model has reached its turning point, switching from globally increasing to a decreasing trend in the first decade of the 21st century. This result is robust, independent of which aerosol module or SSP scenario is used. Non-linear aerosol-cloud interactions dominate as a forcing agent over aerosol-radiation interactions. Aerosols' ability to counterbalance GHG forcing on the global scale is today at a level comparable to that at the beginning of the last century. In the 1980s, the decade of largest global aerosol loads, aerosols balanced up to 80% of GHG forcing. As a consequence, global warming of the last decades, which is primarily driven by greenhouse gases, has been augmented by the effect of decreasing aerosol cooling in our model. By the end of this century, following the SSP scenarios, aerosols will only counterbalance 0%–20% of GHG forcing, depending on model and on scenario.

Aerosol forcing↗
DOE OSTI2023

Reinforcement Learning of Structured Stabilizing Control for Linear Systems With Unknown State Matrix

This paper delves into designing feedback control gains for a continuous-time linear quadratic regulator (LQR) problem that is constrained to certain predefined structure with unknown state matrix. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. Here we first formulate a model-based framework using dynamic programming (DP) to embed the structural constraint to the LQR gain computation in the continuous-time setting, and then subsequently, formulate a policy iteration RL algorithm that can alleviate the requirement of known state matrix in conjunction with maintaining the feedback gain structure. The design enables a distributed learning control design which is necessary for many large-scale cyber-physical systems. Theoretical guarantees are provided for stability and convergence of the structured reinforcement learning (SRL) algorithm. We validate our theoretical results with numerical simulations on a multi-agent networked linear time-invariant (LTI) dynamic system.

42 ENGINEERING↗
DOE OSTI2023

AI-Based Faster-Than-Real-Time Stability Assessment of Large Power Systems with Applications on WECC System

Achieving clean energy goals will require significant advances in regard to addressing the computational needs for next-generation renewable-dominated power grids. One critical obstacle that lies in the way of transitioning today’s power grid to a renewable-dominated power grid is the lack of a faster-than-real-time stability assessment technology for operating a fast-changing power grid. This paper proposes an artificial intelligence (AI) -based method that predicts the system’s stability margin information (e.g., the frequency nadir in the frequency stability assessment and the critical clearing time (CCT) value in the transient stability assessment) directly from the system operating conditions without performing the conventional time-consuming time-domain simulations over detailed dynamic models. Since the AI method shifts the majority of the computational burden to offline training, the online evaluation is extremely fast. This paper has tested the AI-based stability assessment method using multiple dispatch cases that are converted and tuned from actual dispatch cases of the Western Electricity Coordinating Council (WECC) system model with more than 20,000 buses. The results show that the AI-based method could accurately predict the stability margin of such a large power system in less than 0.2 milliseconds using the offline-trained AI agent. Therefore, the proposed method has great potential to achieve faster-than-real-time stability assessment for practical large power systems while preserving sufficient accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2001

Human Centered Autonomous and Assistant Systems Testbed for Exploration Operations

The Engineering and Mission Operations Directorates at NASA Johnson Space Center are combining laboratories and expertise to establish the Human Centered Autonomous and Assistant Systems Testbed for Exploration Operations. This is a testbed for human centered design, development and evaluation of intelligent autonomous and assistant systems that will be needed for human exploration and development of space. This project will improve human-centered analysis, design and evaluation methods for developing intelligent software. This software will support human-machine cognitive and collaborative activities in future interplanetary work environments where distributed computer and human agents cooperate. We are developing and evaluating prototype intelligent systems for distributed multi-agent mixed-initiative operations. The primary target domain is control of life support systems in a planetary base. Technical approaches will be evaluated for use during extended manned tests in the target domain, the Bioregenerative Advanced Life Support Systems Test Complex (BIO-Plex). A spinoff target domain is the International Space Station (ISS) Mission Control Center (MCC). Prodl}cts of this project include human-centered intelligent software technology, innovative human interface designs, and human-centered software development processes, methods and products. The testbed uses adjustable autonomy software and life support systems simulation models from the Adjustable Autonomy Testbed, to represent operations on the remote planet. Ground operations prototypes and concepts will be evaluated in the Exploration Planning and Operations Center (ExPOC) and Jupiter Facility.

Malin, Jane T.↗
DOE OSTI2025

A comparison of ceramic and carbon-based reductants for vitrification of low-activity waste

Sucrose is the current baseline additive at the Hanford Waste Treatment and Immobilization Plant in Washington, USA to control foaming during waste feed to glass transitions and the redox state of the glass melt. Alternative reductants are being investigated to alleviate strain on effluent treatment from toxic acetonitrile production from incomplete combustion of sucrose. This study evaluates ceramic additive options including B4C, B6Si, SiC and VB2 in simulated low-activity waste feed, as well as coke dust, probing the feed volume expansion during melting as well as the gas evolution. All alternative reductant options examined significantly reduced acetonitrile production, however there was variability in their effectiveness as foam-reducing agents. VB2 and coke at the appropriate ratios were similarly effective as sucrose in controlling both foam volume and glass redox state, but with considerably less acetonitrile production. B4C, B6Si and SiC showed more promising foam control and very little acetonitrile production, however all of the final glasses were over reduced, i.e., Fe2+/FeT = 0.5. These alternative reductant studies provide operational flexibility to the operation of the vitrification plant, as well as options for alternative raw materials in industrial glass melting.

Rigby, Jessica C. (ORCID:0000000235719977)↗
DOE OSTI2026

Bounded-Confidence Models of Multidimensional Opinions with Topic-Weighted Discordance

People’s opinions on a wide range of topics often evolve over time through their interactions with others. Models of opinion dynamics primarily focus on one-dimensional opinions, which represent opinions on one topic. However, opinions on various topics are rarely isolated; instead, they can be interdependent and correlated. In a bounded-confidence model (BCM) of opinion dynamics, agents are receptive to each other only if their opinions are sufficiently similar. Here, we extend classical agent-based BCMs—namely, the Hegselmann–Krause BCM, which has synchronous interactions, and the Deffuant–Weisbuch BCM, which has asynchronous interactions—to a multidimensional setting, in which the opinions are multidimensional vectors representing opinions of different topics and opinions on different topics are interdependent. To measure opinion differences between agents, we introduce topic-weighted discordance functions that account for opinion differences in all topics. We define regions of receptiveness for our models, and we use them to characterize the steady-state opinion clusters and provide an analytical approach to compute these regions. In addition, we numerically simulate our models on various networks with initial opinions drawn from a variety of distributions. When initial opinions are correlated across different topics, our topic-weighted BCMs yield significantly different results in both transient and steady states compared to baseline models, where the dynamics of each opinion topic are independent.

Mathematics and Computing↗
DOE OSTI2021

Isomer of linker for NU-1000 yields a new she -type, catalytic, and hierarchically porous, Zr-based metal–organic framework

The well-known MOF (metal–organic framework) linker tetrakis(p-benzoate)pyrene (TBAPy 4– ) lacks steric hindrance between its benzoates. Changing the 1,3,6,8-siting of benzoates in TBAPy 4– to 4,5,9,10-siting introduces substantial steric hindrance and, in turn, enables the synthesis of a new hierarchically porous, she-type MOF Zr 6 (μ 3 -O)4(μ 3 -OH) 4 (C 6 H 5 COO) 3 (COO) 3 (TBAPy-2) 3/2 (NU-601), where TBAPy-2 4– is the 4,5,9,10 isomer of TBAPy 4– . In conclusion, NU-601 shows high catalytic activity for degradative hydrolysis of a simulant for G-type fluoro-phosphorus nerve agents.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2012

Evaluation of the Absolute Regional Temperature Potential

The Absolute Regional Temperature Potential (ARTP) is one of the few climate metrics that provides estimates of impacts at a sub-global scale. The ARTP presented here gives the time-dependent temperature response in four latitude bands (90-28degS, 28degS-28degN, 28-60degN and 60-90degN) as a function of emissions based on the forcing in those bands caused by the emissions. It is based on a large set of simulations performed with a single atmosphere-ocean climate model to derive regional forcing/response relationships. Here I evaluate the robustness of those relationships using the forcing/response portion of the ARTP to estimate regional temperature responses to the historic aerosol forcing in three independent climate models. These ARTP results are in good accord with the actual responses in those models. Nearly all ARTP estimates fall within +/-20%of the actual responses, though there are some exceptions for 90-28degS and the Arctic, and in the latter the ARTP may vary with forcing agent. However, for the tropics and the Northern Hemisphere mid-latitudes in particular, the +/-20% range appears to be roughly consistent with the 95% confidence interval. Land areas within these two bands respond 39-45% and 9-39% more than the latitude band as a whole. The ARTP, presented here in a slightly revised form, thus appears to provide a relatively robust estimate for the responses of large-scale latitude bands and land areas within those bands to inhomogeneous radiative forcing and thus potentially to emissions as well. Hence this metric could allow rapid evaluation of the effects of emissions policies at a finer scale than global metrics without requiring use of a full climate model.

Shindell, D. T.↗