SEARCH · Engineering Papers

Results for “agent based model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

DOE OSTI2022

A Hybrid Reinforcement Learning-MPC Approach for Distribution System Critical Load Restoration

This paper proposes a hybrid control approach for distribution system critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while MPC models grid operations incorporating RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. We formulate the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The RL algorithm is trained offline using a historical forecast of renewable generation and load demand. The method is tested using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery. Case studies demonstrated that the proposed method outperforms other policies with static operating reserves.

distribution system↗

DOE OSTI2020

Reinforcement learning based automated history matching for improved hydrocarbon production forecast

History matching aims to find a numerical reservoir model that can be used to predict the reservoir performance. An engineer and model calibration (data inversion) method are required to adjust various parameters/properties of the numerical model in order to match the reservoir production history. In this study, we develop deep neural networks within the reinforcement learning framework to achieve automated history matching that will reduce engineers’ efforts, human bias, automatically and intelligently explore the parameter space, and remove the need of large set of labeled training data. To that end, a fast-marching-based reservoir simulator is encapsulated as an environment for the proposed reinforcement learning. The deep neural-network-based learning agent interacts with the reservoir simulator within reinforcement learning framework to achieve the automated history matching. Reinforcement learning techniques, such as discrete Deep Q Network and continuous Deep Deterministic Policy Gradients, are used toth, used to train the learning agents. The continuous actions enable the Deep Deterministic Policy Gradients to explore more states at each iteration in a a learning episode; consequently, a better history matching is achieved using this algorithm as compared to Deep Q Network. For simplified dual-target composite reservoir models, the best history-matching performances of the discrete and continuous learning methods in terms of normalized root mean square errors are 0.0447 and 0.0038, respectively. Furthermore, our study shows that continuous action space achieved by the deep deterministic policy gradient drastically outperforms deep Q network.

42 ENGINEERING↗

DOE OSTI2022

The development and use of decision support framework for informing selection of select agent toxins with modelling studies to inform permissible toxin amounts

Many countries have worked diligently to establish and implement policies and processes to regulate high consequence pathogens and toxins that could have a significant public health impact if misused. In the United States, the Antiterrorism and Effective Death Penalty Act of 1996 (Public Law 104-132, 1996), as amended by the Bioterrorism Preparedness and Response Act of 2002 (Public Law 107-188, 2002) requires that the Department of Health and Human Services (HHS) [through the Centers for Disease Control and Prevention (CDC)] establish a list of bacteria, viruses, and toxins that have the potential to pose a severe threat to public health and safety. Currently, this list is reviewed and updated on a biennial basis using input from subject matter experts (SMEs). We have developed decision support framework (DSF) approaches to facilitate selection of select toxins and, where toxicity data are known, conducted modelling studies to inform selection of toxin amounts that should be excluded from select agent regulations. Exclusion limits allow laboratories to possess toxins under an established limit to support their research or teaching activities without the requirement to register with the Federal Select Agent Program. Fact sheets capturing data from a previously vetted SME workshop convened by CDC, literature review and SME input were developed to assist in evaluating toxins using the DSF approach. The output of the DSF analysis agrees with the current select toxin designations, and no other toxins evaluated in this study were recommended for inclusion on the select agent and toxin list. To inform the selection of exclusion limits, attack scenarios were developed to estimate the amount of toxin needed to impact public health. Scenarios consisted of simulated aerosol releases of a toxin in high-population-density public facilities and the introduction of a toxin into a daily consumable product supply chain. Using published inhalation and ingestion median toxic dose (TD 50 ) and median lethal dose (LD 50 ) values, where available, a range of toxin amounts was examined to estimate the number of people exposed to these amounts in these scenarios. Based on data generated by these models, we proposed toxin exclusion values corresponding to levels below those that would trigger a significant public health response (i.e., amounts estimated to expose up to ten people by inhalation or one hundred people by ingestion to LD 50 or TD 50 levels of toxin in the modeled scenarios).

60 APPLIED LIFE SCIENCES↗

NASA NTRS1998

RIACS

The Research Institute for Advanced Computer Science (RIACS) was established by the Universities Space Research Association (USRA) at the NASA Ames Research Center (ARC) on June 6, 1983. RIACS is privately operated by USRA, a consortium of universities that serves as a bridge between NASA and the academic community. Under a five-year co-operative agreement with NASA, research at RIACS is focused on areas that are strategically enabling to the Ames Research Center's role as NASA's Center of Excellence for Information Technology. The primary mission of RIACS is charted to carry out research and development in computer science. This work is devoted in the main to tasks that are strategically enabling with respect to NASA's bold mission in space exploration and aeronautics. There are three foci for this work: (1) Automated Reasoning. (2) Human-Centered Computing. and (3) High Performance Computing and Networking. RIACS has the additional goal of broadening the base of researcher in these areas of importance to the nation's space and aeronautics enterprises. Through its visiting scientist program, RIACS facilitates the participation of university-based researchers, including both faculty and students, in the research activities of NASA and RIACS. RIACS researchers work in close collaboration with NASA computer scientists on projects such as the Remote Agent Experiment on Deep Space One mission, and Super-Resolution Surface Modeling.

Moore, Robert C.↗

NASA NTRS1998

RIACS

The Research Institute for Advanced Computer Science (RIACS) was established by the Universities Space Research Association (USRA) at the NASA Ames Research Center (ARC) on June 6, 1983. RIACS is privately operated by USRA, a consortium of universities that serves as a bridge between NASA and the academic community. Under a five-year co-operative agreement with NASA, research at RIACS is focused on areas that are strategically enabling to the Ames Research Center's role as NASA's Center of Excellence for Information Technology. Research is carried out by a staff of full-time scientist,augmented by visitors, students, post doctoral candidates and visiting university faculty. The primary mission of RIACS is charted to carry out research and development in computer science. This work is devoted in the main to tasks that are strategically enabling with respect to NASA's bold mission in space exploration and aeronautics. There are three foci for this work: Automated Reasoning. Human-Centered Computing. and High Performance Computing and Networking. RIACS has the additional goal of broadening the base of researcher in these areas of importance to the nation's space and aeronautics enterprises. Through its visiting scientist program, RIACS facilitates the participation of university-based researchers, including both faculty and students, in the research activities of NASA and RIACS. RIACS researchers work in close collaboration with NASA computer scientists on projects such as the Remote Agent Experiment on Deep Space One mission, and Super-Resolution Surface Modeling.

Moore, Robert C.↗

NASA NTRS2001

Physical and Chemical Aspects of Fire Suppression in Extraterrestrial Environments

A fire, whether in a spacecraft or in occupied spaces on extraterrestrial bases, can lead to mission termination or loss of life. While the fire-safety record of US space missions has been excellent, the advent of longer duration missions to Mars, the moon, or aboard the International Space Station (ISS) increases the likelihood of fire events, with more limited mission termination options. The fire safety program of NASA's manned space flight program is based largely upon the principles of controlling the flammability of on-board materials and greatly eliminating sources of ignition. As a result, very little research has been conducted on fire suppression in the microgravity or reduced-gravity environment. The objectives of this study are: to obtain fundamental knowledge of physical and chemical processes of fire suppression, using gravity and oxygen concentration as independent variables to simulate various extraterrestrial environments, including spacecraft and surface bases in Mars and moon missions; to provide rigorous testing of analytical models, which include comprehensive descriptions of combustion and suppression chemistry; and to provide basic research results useful for technological advances in fire safety, including the development of new fire-extinguishing agents and approaches, in the microgravity environment associated with ISS and in the partial-gravity Martian and lunar environments.

Takahashi, F.↗

NASA NTRS1996

A dual-input nonlinear system analysis of autonomic modulation of heart rate

Linear analyses of fluctuations in heart rate and other hemodynamic variables have been used to elucidate cardiovascular regulatory mechanisms. The role of nonlinear contributions to fluctuations in hemodynamic variables has not been fully explored. This paper presents a nonlinear system analysis of the effect of fluctuations in instantaneous lung volume (ILV) and arterial blood pressure (ABP) on heart rate (HR) fluctuations. To successfully employ a nonlinear analysis based on the Laguerre expansion technique (LET), we introduce an efficient procedure for broadening the spectral content of the ILV and ABP inputs to the model by adding white noise. Results from computer simulations demonstrate the effectiveness of broadening the spectral band of input signals to obtain consistent and stable kernel estimates with the use of the LET. Without broadening the band of the ILV and ABP inputs, the LET did not provide stable kernel estimates. Moreover, we extend the LET to the case of multiple inputs in order to accommodate the analysis of the combined effect of ILV and ABP effect on heart rate. Analyzes of data based on the second-order Volterra-Wiener model reveal an important contribution of the second-order kernels to the description of the effect of lung volume and arterial blood pressure on heart rate. Furthermore, physiological effects of the autonomic blocking agents propranolol and atropine on changes in the first- and second-order kernels are also discussed.

NASA Discipline Regulatory Physiology↗

DOE OSTI2025

A2SD: Accelerating Scientific Innovation Through Autonomous Discovery Systems

The 2025 Advancing Autonomous Scientific Discovery (A2SD) workshop convened researchers from academia, national laboratories, and industry to explore the transformative role of autonomy in scientific discovery. The workshop highlighted a convergence of artificial intelligence, robotics, and computational workflows into autonomous systems capable of accelerating the scientific process. Presentations and discussions spanned autonomous experimentation, intelligent workflow orchestration, digital twins, and agent-based systems for managing complex research ecosystems. Key challenges discussed included interoperability across heterogeneous infrastructures, near real-time data management under FAIR principles, reproducibility, and the integration of human oversight. The workshop also emphasized the need for modular software interfaces, federated learning models, and education initiatives to support a next-generation scientific workforce.

Taufer, Michela [University of Tennessee, Knoxvill↗

DOE OSTI2022

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Deep reinforcement learning (DRL) algorithms have successfully solved many challenging problems in various power system control scenarios. However, their decision-making process is usually regarded as black-boxes. Furthermore, how DRL models interact with human intelligence remains an open problem. Thus, this paper proposes a policy extraction framework to extract a complex DRL model into an explainable policy. This framework includes three parts: 1) DRL training and data generation. We train an agent for a specific control task and generate data, which contains the control policy of the agent. 2) Policy extraction. We propose an information gain rate based weighted oblique decision tree (IGR-WODT) for DRL policy extraction. 3) Policy evaluation. We define three metrics to evaluate the performance of the proposed approach. A case study for the under-voltage load shedding problem shows that the IGR-WODT presents a performance enhancement compared with DRL, weighted oblique decision tree, and univariate decision tree. The proposed policy extraction method could provide an intuitive explanation of the neural network decision-making process to the dispatchers when making final decisions on power grid operation. Also, the resulted rule-based controller could replace the deep neural network-based controller in many field edge devices with limited computing resources, providing comparable performance.

deep reinforcement learning↗

DOE OSTI2026

Optimization and stabilization of Fermilab Booster using hybrid Bayesian/RL framework

PIPII project will raise Fermilab Booster intensity and ramp rate. Beam losses will limit average power and are hard to simulate. Presently, Booster uses operator-guided empirical tuning. This task is challenging due to high dimensionality, multiple objectives, critical safety constraints, and drifts. We developed a synergistic suite of Bayesian optimization (BO) and reinforcement learning (RL) tools to optimize and stabilize beam losses. First, active learning was used to build a rough model. Data was collected parasitically using two novel safety constraint types – nonlinear input space restrictions (based on optics model), and uncertainty constraints (to stop bad steps/beam aborts). We then applied online multi-objective BO with scalarized objectives and fitting to improve/rebalance losses, increasing safety margins by 25%. Using BO model as a safety veto, we tried several on/off-policy RL agents for long term stabilization; SAC had best performance. We found that adding contextual (state) information further improved performance, eventually integrating key knobs like linac phase and temperature into the parameter space. Long term testing is ongoing to enable operational use.

Kuklev, Nikita [Fermilab]↗

NASA NTRS1993

Applications of artificial intelligence 1993: Knowledge-based systems in aerospace and industry; Proceedings of the Meeting, Orlando, FL, Apr. 13-15, 1993

The present volume on applications of artificial intelligence with regard to knowledge-based systems in aerospace and industry discusses machine learning and clustering, expert systems and optimization techniques, monitoring and diagnosis, and automated design and expert systems. Attention is given to the integration of AI reasoning systems and hardware description languages, care-based reasoning, knowledge, retrieval, and training systems, and scheduling and planning. Topics addressed include the preprocessing of remotely sensed data for efficient analysis and classification, autonomous agents as air combat simulation adversaries, intelligent data presentation for real-time spacecraft monitoring, and an integrated reasoner for diagnosis in satellite control. Also discussed are a knowledge-based system for the design of heat exchangers, reuse of design information for model-based diagnosis, automatic compilation of expert systems, and a case-based approach to handling aircraft malfunctions.

Fayyad, Usama M.↗

DOE OSTI2025

Distributed fiber-optic sensing in a subscale high-temperature superconducting dipole magnet

High-temperature superconductors, such as REBa2Cu3O7−x (REBCO, RE = rare earth), are becoming pivotal for high-field magnet technology for future circular colliders and compact fusion reactors. The U.S. Magnet Development Program, in collaboration with industry, is developing REBCO magnet technology using round conductors consisting of multiple REBCO tapes. For these multi-tape cables, traditional instrumentation, such as voltage taps and resistive strain gauges, become insufficient to help measure and understand the performance-limiting factors in these model magnets. Distributed fiber-optic sensing (DFOS) is a potential solution to address this challenge. Although DFOS is well established for various applications, measuring temperature and strain in high-temperature superconducting magnets is in its infancy. Here we report the detailed implementation and test results of DFOS based on Rayleigh scattering in a subscale canted cosθ (CCT) dipole magnet using high-temperature superconducting CORC® wires. We co-wound optical fibers in each layer of the CCT magnet and compared different types of commercial fibers and mold-release agents to reduce the power attenuation in the fibers. The DFOS allowed us to measure mechanical deformation and temperature along the conductor during tests at 77 and 4.2 K. The measured strain agreed quantitively with a finite-element mechanical model of the subscale magnet. Our results indicate that DFOS can effectively identify locations of strain and temperature changes, offering unique insight into magnet performance that can advance our understanding and development of the REBCO magnet technology for high-energy physics and fusion applications.

Luo, Linqing↗

DOE OSTI2026

GridCoPilot for Thermal Events: An LLM-Based Platform for Power Grid Reliability Analysis

Large Language Models show promise for translating natural language into database queries, but deploying such systems in safety-critical domains requires high reliability. We present an application of GridCoPilot to thermal event analysis (heatwaves and coldwaves) that affect power grid reliability. Our approach uses a LangChain SQL Agent to translate natural language queries into auditable SQL statements, with deterministic visualization routines that parse the structured query results. We introduce structural framing as a design principle, we integrate a NERC-region-level event library with county-level meteorology and decompose the combined data into three relational tables (event metadata, county-level event details, and a county-to-NERC subregion mapping), using prompt-guided joins to direct the model toward correct multi-table queries. For two core analytical patterns (identifying worst events by region and by region-year), the system achieved 100% SQL accuracy across all 16 NERC subregions and both event types (64 queries total). These results validate the approach for target use cases, though performance on diverse natural language formulations requires further investigation. We discuss design trade-offs, failure modes including JSON output truncation, and pathways for extending this approach to other hazard domains.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2024

Aera-Mip: Emission Pathways, Remaining Budgets, and Carbon Cycle Dynamics Compatible With 1.5 and 2 °C Global Warming Stabilization

While international climate policies now focus on limiting global warming to well below 2 °C or pursuing a 1.5 °C level of global warming, the climate modelling community has not provided an experimental design in which all Earth system models (ESMs) converge and stabilize at the same prescribed global warming levels. This gap hampers accurate estimations based on comprehensive ESMs of the carbon emission pathways and budgets needed to meet such agreed warming levels and of the associated climate impacts under temperature stabilization. Here, we apply the Adaptive Emission Reduction Approach (AERA) with ESMs to provide such simulations in which all models converge at 1.5 and 2.0 °C warming levels by adjusting their emissions over time. These emission-driven simulations provide a wide range of emission pathways and resulting atmospheric CO 2 projections for a given warming level, uncovering uncertainty ranges that were previously missing in the traditional Coupled Model Intercomparison Project (CMIP) scenarios with prescribed greenhouse gas concentration pathways. Meeting the 1.5 °C warming level requires a 40 % (full model range: 7 % to 76 %) reduction in multi-model mean CO 2 -forcing-equivalent (CO 2 -fe) emissions from 2025 to 2030, a 98 % (57 % to 127 %) reduction from 2025 to 2050, and a stabilization at 1.0 (−1.7 to 2.9) PgC yr−1 from 2100 onward after the 1.5 °C global warming level is reached. Meeting the 2.0 °C warming level requires a 47 % (8 % to 92 %) reduction in multi-model mean CO 2 -fe emissions until 2050 and a stabilization at 1.7 (−1.5 to 2.7) PgC yr−1 from 2100 onward. The on-average positive emissions under stabilized global temperatures are the result of a decreasing transient climate response to cumulative CO 2 -fe emissions over time under stabilized global warming. This evolution is consistent with a slightly negative zero emissions commitment – initially assumed to be zero – and leads to an increase in the post-2025 CO 2 -fe emission budget by a factor of 2.2 (−0.8 to 6.9) by 2150 for the 1.5 °C warming level and a factor of 1.4 (0.9 to 2.4) for the 2.0 °C warming level compared to its first estimate in 2025. The median CO 2 -only carbon budget by 2150, relative to 2020, is 800 GtCO 2 for the 1.5 °C warming level and 2250 GtCO 2 for the 2.0 °C warming level. These median values exceed the median IPCC AR6 estimates by 60 % for the 1.5 °C warming level and 67 % for 2.0 °C. Some of the differences may be explained by the choice of the mitigation scenario for non-CO 2 radiative agents. Our simulations highlight shifts in carbon uptake dynamics under stabilized temperature, such as a cessation of the carbon sinks in the North Atlantic and in tropical forests. On the other hand, the Southern Ocean remains a carbon sink centuries after temperatures stabilize. Overall, this new type of warming-level-based emission-driven simulation offers a more coherent assessment across climate models and opens up a wide range of possibilities for studying both the carbon cycle and climate impacts, such as extreme events, under climate stabilization.

Adaptive Emission Reduction Approach↗

DOE OSTI2020

Transactive Energy Systems: The Market-based Coordination of Distributed Energy Resources

This paper synthesizes a unifying framework that captures a large class of transactive energy systems studied in the literature. The framework consists of four key elements: agent preferences, control decisions, information structure, and solution concept. Among them, agent preference specifies the preferred outcome by each agent and models the conflict of interest in decision making. Control decision specifies the set of decisions for each agent to maximize its payoff. Information structure describes the information available to each agent before it makes a decision. Solution concept encodes the rationality assumptions of each agent in the system. These elements are important in identifying and distinguishing different transactive energy systems. In this paper, we use them to analyze a number of important ones, including the competitive equilibrium, Stackelberg game, reverse Stackelberg game, and mechanism design. Each class of these problems is formulated as a special case of the proposed transactive energy systems. Their connections and differences are discussed. Available tools and results for each category of problems are surveyed.

Li, Sen↗

DOE OSTI2023

Inverse reinforcement learning control for building energy management

Reinforcement learning (RL) based control is widely considered a promising approach in building automation and control as it has demonstrated the potential to deal with complex objectives in adjacent domains like robotics, autonomous vehicles, gaming applications, and advertisement recommendations. When applied to any environment, model-free RL learns to improve its control performance over time without requiring a control model, by receiving and then analyzing feedback from the building environment after each control action. Operational objectives are becoming increasingly complex through the simultaneous consideration of thermal comfort, carbon emissions, grid services, and indoor air quality. In this context, conventional rule-based control approaches are proving sub-optimal, mostly heuristic, and inadequate. The model-free and self learning nature of RL appears promising and attractive as it may address the scalability issues associated with advanced control approaches. However, it suffers from long training times and unstable control behavior during the early stages of its learning process, which makes it unsuitable to be applied directly to buildings. This paper addresses these issues using an inverse reinforcement learning approach (IRL), a technique utilized to learn the objective of a controller agent which is considered an expert in its respective domain. Here, we consider a rule-based control as the expert demonstrator. IRL is different from a direct imitation (i.e., direct mapping of states to actions) of control actions as it tries to find the underlying intent of an expert's policy, providing the controller with a better-generalized policy for unseen states or environments with slightly different dynamics. This approach propels the RL controller's policy to levels similar to or better than that of a rule-based policy before it starts learning by interacting with the building. This makes RL for building energy management applications more practical as it prevents the erratic and exploratory behavior in the initial training period, simultaneously speeding up the learning process when compared to applying an untrained RL agent directly to a building environment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2020

OpenABLext: An automatic code generation framework for agent-based simulations on CPU-GPU-FPGA heterogeneous platforms

The execution of agent-based simulations (ABSs) on hardware accelerator devices such as graphics processing units (GPUs) has been shown to offer great performance potentials. However, in heterogeneous hardware environments, it can become increasingly difficult to find viable partitions of the simulation and provide implementations for different hardware devices. To automate this process, we present OpenABLext, an extension to OpenABL, a model specification language for ABSs. By providing a device-aware OpenCL backend, OpenABLext enables the co-execution of ABS on heterogeneous hardware platforms consisting of central processing units, GPUs, and field programmable gate arrays (FPGAs).We present a novel online dispatching method that efficiently profiles partitions of the simulation during run-time to optimize the hardware assignment while using the profiling results to advance the simulation itself. In addition, OpenABLext features automated conflict resolution based on user-specified rules, supports graph-based simulation spaces, and utilizes an efficient neighbor search algorithm. We show the improved performance of OpenABLext and demonstrate the potential of FPGAs in the context of ABS. We illustrate how co-execution can be used to further lower execution times. OpenABLext can be seen as an enabler to tap the computing power of heterogeneous hardware platforms for ABS.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Spatial Impacts of Electric Vehicle Charging on Power Grid Stability: A Downtown Atlanta Case Study

The rapid increase in electric vehicle (EV) charging demand poses a potential risk to power grid stability, particularly as the spatial distribution of this demand remains underexplored. Existing research often focuses on technical optimization models while overlooking the geographic and human dynamics that affect energy consumption. This study addresses this gap by incorporating mobility data to estimate both building energy use and EV charging demand while also considering geographic factors for a better understanding of grid load. Using agent-based simulations and the Open-Source Distribution System Simulator, the study evaluates the effect of various EV penetration scenarios on grid voltage and unbalance. The results show that, although voltage remains within acceptable limits at lower EV penetration rates, significant voltage drop and unbalance occur as EV penetration exceeds 40%, particularly in residential areas with high charging demand. This study offers a framework for integrating spatial analysis and mobility data in power network simulations, providing insights for future EV infrastructure planning.

Pan, Melrose [ORNL] (ORCID:000000031627448X)↗