Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Hamilton: Flexible, Open Source $10 Wireless Sensor System for Energy Efficient Building Operation

Sensors for improving building performance are rapidly populating the market, driven in part by the drive to reduce greenhouse gas emissions resulting from energy production as well as improve the interior environment for healthy and more productive spaces. UC Berkeley has led wireless sensor development over the past 25 years (e.g., Telos mote), with the Hamilton (named after Alexander Hamilton on the US $10 bill) as the most recent. The Hamilton sensor was designed as a low-cost high-performance sensor that is modular and interoperable. The objective of the Hamilton project was to create, evaluate and establish the technological foundations for secure and easy to deploy building energy efficiency applications utilizing pervasive, low-cost wireless sensors integrated with traditional Building Management Systems (BMS), consumer-sector building components, and powerful data analytics. The project included iterative hardware design, incorporating a high-performance database (BTrDb, http://btrdb.io/), creating and iterating the development of secure data middleware (BOSSwave, WAVE/WAVEMQ), working with and pushing the development of an open-source tiny operating system RiotOS, and implementing and improving protocols such as Thread/OpenThread and TCP/IP. The hardware benefited from careful design to drive down the cost; the design included a System-on-a-Chip (SoC), chip antenna, single crystal and five passive components. Careful design of the operating system created a low-power design to enable a long life with small batteries. The hardware included several sensors: temperature, radiant temperature, relative humidity, magnetometer, accelerometer, and light, with an optional occupancy (Passive InfraRed) sensor. The project was the basis of several applications, both internal to the research team and other researchers and professionals at other institutions. Several applications used the sensor hardware as the basis for other complex devices. Other applications used the sensors to improve building performance through interoperating with the building Heating Ventilation and Air-Conditioning (HVAC) system, such as using occupancy and/or distributed temperature sensing to reduce HVAC zone energy while still providing thermal comfort and to reduce peak loads in small commercial buildings. We demonstrated cloud-based energy analytics, implemented a schedule and a Model Predictive Controller in a small commercial building to optimize HVAC energy, occupancy and electricity price. Initial integration of these technological innovations was performed through the creation of execution containers containing the WAVE agent and various driver, proxy, or building system function logic. The research added to the understanding of efficient sensor hardware, secure middleware, time-series data management (high performance database), efficient communication protocols, and interoperating with applications and building systems. The project showed the technical effectiveness and economic feasibility of creating a low-cost, modular, and easy-to-deploy sensor. Through conversations with multiple end users, the research team discovered that many customers wanted data management and services in addition to the sensors. HamiltonIOT developed packages of sensors, border router, and data services to provide a seamless “plug-and-play” sensor deployment. Some customers were willing to pay for higher quality sensors (such as light); some customers wanted a robust enclosure (waterproof).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Cooperating intelligent systems

Some of the issues connected to the development of a bureaucratic system are discussed. Emphasis is on a layer multiagent approach to distributed artificial intelligence (DAI). The division of labor in a bureaucracy is considered. The bureaucratic model seems to be a fertile model for further examination since it allows for the growth and change of system components and system protocols and rules. The first part of implementing the system would be the construction of a frame based reasoner and the appropriate B-agents and E-agents. The agents themselves should act as objects and the E-objects in particular should have the capability of taking on a different role. No effort was made to address the problems of automated failure recovery, problem decomposition, or implementation. Instead what has been achieved is a framework that can be developed in several distinct ways, and which provides a core set of metaphors and issues for further research.

Rochowiak, Daniel↗

Predicting Pilot Behavior in Medium Scale Scenarios Using Game Theory and Reinforcement Learning

Effective automation is critical in achieving the capacity and safety goals of the Next Generation Air Traffic System. Unfortunately creating integration and validation tools for such automation is difficult as the interactions between automation and their human counterparts is complex and unpredictable. This validation becomes even more difficult as we integrate wide-reaching technologies that affect the behavior of different decision makers in the system such as pilots, controllers and airlines. While overt short-term behavior changes can be explicitly modeled with traditional agent modeling systems, subtle behavior changes caused by the integration of new technologies may snowball into larger problems and be very hard to detect. To overcome these obstacles, we show how integration of new technologies can be validated by learning behavior models based on goals. In this framework, human participants are not modeled explicitly. Instead, their goals are modeled and through reinforcement learning their actions are predicted. The main advantage to this approach is that modeling is done within the context of the entire system allowing for accurate modeling of all participants as they interact as a whole. In addition such an approach allows for efficient trade studies and feasibility testing on a wide range of automation scenarios. The goal of this paper is to test that such an approach is feasible. To do this we implement this approach using a simple discrete-state learning system on a scenario where 50 aircraft need to self-navigate using Automatic Dependent Surveillance-Broadcast (ADS-B) information. In this scenario, we show how the approach can be used to predict the ability of pilots to adequately balance aircraft separation and fly efficient paths. We present results with several levels of complexity and airspace congestion.

Game Theory↗

A Hybrid Reinforcement Learning-MPC Approach for Distribution System Critical Load Restoration

This paper proposes a hybrid control approach for distribution system critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while MPC models grid operations incorporating RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. We formulate the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The RL algorithm is trained offline using a historical forecast of renewable generation and load demand. The method is tested using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery. Case studies demonstrated that the proposed method outperforms other policies with static operating reserves.

distribution system↗

Reinforcement learning based automated history matching for improved hydrocarbon production forecast

History matching aims to find a numerical reservoir model that can be used to predict the reservoir performance. An engineer and model calibration (data inversion) method are required to adjust various parameters/properties of the numerical model in order to match the reservoir production history. In this study, we develop deep neural networks within the reinforcement learning framework to achieve automated history matching that will reduce engineers’ efforts, human bias, automatically and intelligently explore the parameter space, and remove the need of large set of labeled training data. To that end, a fast-marching-based reservoir simulator is encapsulated as an environment for the proposed reinforcement learning. The deep neural-network-based learning agent interacts with the reservoir simulator within reinforcement learning framework to achieve the automated history matching. Reinforcement learning techniques, such as discrete Deep Q Network and continuous Deep Deterministic Policy Gradients, are used toth, used to train the learning agents. The continuous actions enable the Deep Deterministic Policy Gradients to explore more states at each iteration in a a learning episode; consequently, a better history matching is achieved using this algorithm as compared to Deep Q Network. For simplified dual-target composite reservoir models, the best history-matching performances of the discrete and continuous learning methods in terms of normalized root mean square errors are 0.0447 and 0.0038, respectively. Furthermore, our study shows that continuous action space achieved by the deep deterministic policy gradient drastically outperforms deep Q network.

42 ENGINEERING↗

The development and use of decision support framework for informing selection of select agent toxins with modelling studies to inform permissible toxin amounts

Many countries have worked diligently to establish and implement policies and processes to regulate high consequence pathogens and toxins that could have a significant public health impact if misused. In the United States, the Antiterrorism and Effective Death Penalty Act of 1996 (Public Law 104-132, 1996), as amended by the Bioterrorism Preparedness and Response Act of 2002 (Public Law 107-188, 2002) requires that the Department of Health and Human Services (HHS) [through the Centers for Disease Control and Prevention (CDC)] establish a list of bacteria, viruses, and toxins that have the potential to pose a severe threat to public health and safety. Currently, this list is reviewed and updated on a biennial basis using input from subject matter experts (SMEs). We have developed decision support framework (DSF) approaches to facilitate selection of select toxins and, where toxicity data are known, conducted modelling studies to inform selection of toxin amounts that should be excluded from select agent regulations. Exclusion limits allow laboratories to possess toxins under an established limit to support their research or teaching activities without the requirement to register with the Federal Select Agent Program. Fact sheets capturing data from a previously vetted SME workshop convened by CDC, literature review and SME input were developed to assist in evaluating toxins using the DSF approach. The output of the DSF analysis agrees with the current select toxin designations, and no other toxins evaluated in this study were recommended for inclusion on the select agent and toxin list. To inform the selection of exclusion limits, attack scenarios were developed to estimate the amount of toxin needed to impact public health. Scenarios consisted of simulated aerosol releases of a toxin in high-population-density public facilities and the introduction of a toxin into a daily consumable product supply chain. Using published inhalation and ingestion median toxic dose (TD 50 ) and median lethal dose (LD 50 ) values, where available, a range of toxin amounts was examined to estimate the number of people exposed to these amounts in these scenarios. Based on data generated by these models, we proposed toxin exclusion values corresponding to levels below those that would trigger a significant public health response (i.e., amounts estimated to expose up to ten people by inhalation or one hundred people by ingestion to LD 50 or TD 50 levels of toxin in the modeled scenarios).

60 APPLIED LIFE SCIENCES↗

RIACS

The Research Institute for Advanced Computer Science (RIACS) was established by the Universities Space Research Association (USRA) at the NASA Ames Research Center (ARC) on June 6, 1983. RIACS is privately operated by USRA, a consortium of universities that serves as a bridge between NASA and the academic community. Under a five-year co-operative agreement with NASA, research at RIACS is focused on areas that are strategically enabling to the Ames Research Center's role as NASA's Center of Excellence for Information Technology. The primary mission of RIACS is charted to carry out research and development in computer science. This work is devoted in the main to tasks that are strategically enabling with respect to NASA's bold mission in space exploration and aeronautics. There are three foci for this work: (1) Automated Reasoning. (2) Human-Centered Computing. and (3) High Performance Computing and Networking. RIACS has the additional goal of broadening the base of researcher in these areas of importance to the nation's space and aeronautics enterprises. Through its visiting scientist program, RIACS facilitates the participation of university-based researchers, including both faculty and students, in the research activities of NASA and RIACS. RIACS researchers work in close collaboration with NASA computer scientists on projects such as the Remote Agent Experiment on Deep Space One mission, and Super-Resolution Surface Modeling.

Moore, Robert C.↗

RIACS

The Research Institute for Advanced Computer Science (RIACS) was established by the Universities Space Research Association (USRA) at the NASA Ames Research Center (ARC) on June 6, 1983. RIACS is privately operated by USRA, a consortium of universities that serves as a bridge between NASA and the academic community. Under a five-year co-operative agreement with NASA, research at RIACS is focused on areas that are strategically enabling to the Ames Research Center's role as NASA's Center of Excellence for Information Technology. Research is carried out by a staff of full-time scientist,augmented by visitors, students, post doctoral candidates and visiting university faculty. The primary mission of RIACS is charted to carry out research and development in computer science. This work is devoted in the main to tasks that are strategically enabling with respect to NASA's bold mission in space exploration and aeronautics. There are three foci for this work: Automated Reasoning. Human-Centered Computing. and High Performance Computing and Networking. RIACS has the additional goal of broadening the base of researcher in these areas of importance to the nation's space and aeronautics enterprises. Through its visiting scientist program, RIACS facilitates the participation of university-based researchers, including both faculty and students, in the research activities of NASA and RIACS. RIACS researchers work in close collaboration with NASA computer scientists on projects such as the Remote Agent Experiment on Deep Space One mission, and Super-Resolution Surface Modeling.

Moore, Robert C.↗

Physical and Chemical Aspects of Fire Suppression in Extraterrestrial Environments

A fire, whether in a spacecraft or in occupied spaces on extraterrestrial bases, can lead to mission termination or loss of life. While the fire-safety record of US space missions has been excellent, the advent of longer duration missions to Mars, the moon, or aboard the International Space Station (ISS) increases the likelihood of fire events, with more limited mission termination options. The fire safety program of NASA's manned space flight program is based largely upon the principles of controlling the flammability of on-board materials and greatly eliminating sources of ignition. As a result, very little research has been conducted on fire suppression in the microgravity or reduced-gravity environment. The objectives of this study are: to obtain fundamental knowledge of physical and chemical processes of fire suppression, using gravity and oxygen concentration as independent variables to simulate various extraterrestrial environments, including spacecraft and surface bases in Mars and moon missions; to provide rigorous testing of analytical models, which include comprehensive descriptions of combustion and suppression chemistry; and to provide basic research results useful for technological advances in fire safety, including the development of new fire-extinguishing agents and approaches, in the microgravity environment associated with ISS and in the partial-gravity Martian and lunar environments.

Takahashi, F.↗

A dual-input nonlinear system analysis of autonomic modulation of heart rate

Linear analyses of fluctuations in heart rate and other hemodynamic variables have been used to elucidate cardiovascular regulatory mechanisms. The role of nonlinear contributions to fluctuations in hemodynamic variables has not been fully explored. This paper presents a nonlinear system analysis of the effect of fluctuations in instantaneous lung volume (ILV) and arterial blood pressure (ABP) on heart rate (HR) fluctuations. To successfully employ a nonlinear analysis based on the Laguerre expansion technique (LET), we introduce an efficient procedure for broadening the spectral content of the ILV and ABP inputs to the model by adding white noise. Results from computer simulations demonstrate the effectiveness of broadening the spectral band of input signals to obtain consistent and stable kernel estimates with the use of the LET. Without broadening the band of the ILV and ABP inputs, the LET did not provide stable kernel estimates. Moreover, we extend the LET to the case of multiple inputs in order to accommodate the analysis of the combined effect of ILV and ABP effect on heart rate. Analyzes of data based on the second-order Volterra-Wiener model reveal an important contribution of the second-order kernels to the description of the effect of lung volume and arterial blood pressure on heart rate. Furthermore, physiological effects of the autonomic blocking agents propranolol and atropine on changes in the first- and second-order kernels are also discussed.

NASA Discipline Regulatory Physiology↗

A2SD: Accelerating Scientific Innovation Through Autonomous Discovery Systems

The 2025 Advancing Autonomous Scientific Discovery (A2SD) workshop convened researchers from academia, national laboratories, and industry to explore the transformative role of autonomy in scientific discovery. The workshop highlighted a convergence of artificial intelligence, robotics, and computational workflows into autonomous systems capable of accelerating the scientific process. Presentations and discussions spanned autonomous experimentation, intelligent workflow orchestration, digital twins, and agent-based systems for managing complex research ecosystems. Key challenges discussed included interoperability across heterogeneous infrastructures, near real-time data management under FAIR principles, reproducibility, and the integration of human oversight. The workshop also emphasized the need for modular software interfaces, federated learning models, and education initiatives to support a next-generation scientific workforce.

Taufer, Michela [University of Tennessee, Knoxvill↗

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Deep reinforcement learning (DRL) algorithms have successfully solved many challenging problems in various power system control scenarios. However, their decision-making process is usually regarded as black-boxes. Furthermore, how DRL models interact with human intelligence remains an open problem. Thus, this paper proposes a policy extraction framework to extract a complex DRL model into an explainable policy. This framework includes three parts: 1) DRL training and data generation. We train an agent for a specific control task and generate data, which contains the control policy of the agent. 2) Policy extraction. We propose an information gain rate based weighted oblique decision tree (IGR-WODT) for DRL policy extraction. 3) Policy evaluation. We define three metrics to evaluate the performance of the proposed approach. A case study for the under-voltage load shedding problem shows that the IGR-WODT presents a performance enhancement compared with DRL, weighted oblique decision tree, and univariate decision tree. The proposed policy extraction method could provide an intuitive explanation of the neural network decision-making process to the dispatchers when making final decisions on power grid operation. Also, the resulted rule-based controller could replace the deep neural network-based controller in many field edge devices with limited computing resources, providing comparable performance.

deep reinforcement learning↗

Optimization and stabilization of Fermilab Booster using hybrid Bayesian/RL framework

PIPII project will raise Fermilab Booster intensity and ramp rate. Beam losses will limit average power and are hard to simulate. Presently, Booster uses operator-guided empirical tuning. This task is challenging due to high dimensionality, multiple objectives, critical safety constraints, and drifts. We developed a synergistic suite of Bayesian optimization (BO) and reinforcement learning (RL) tools to optimize and stabilize beam losses. First, active learning was used to build a rough model. Data was collected parasitically using two novel safety constraint types – nonlinear input space restrictions (based on optics model), and uncertainty constraints (to stop bad steps/beam aborts). We then applied online multi-objective BO with scalarized objectives and fitting to improve/rebalance losses, increasing safety margins by 25%. Using BO model as a safety veto, we tried several on/off-policy RL agents for long term stabilization; SAC had best performance. We found that adding contextual (state) information further improved performance, eventually integrating key knobs like linac phase and temperature into the parameter space. Long term testing is ongoing to enable operational use.

Kuklev, Nikita [Fermilab]↗

Applications of artificial intelligence 1993: Knowledge-based systems in aerospace and industry; Proceedings of the Meeting, Orlando, FL, Apr. 13-15, 1993

The present volume on applications of artificial intelligence with regard to knowledge-based systems in aerospace and industry discusses machine learning and clustering, expert systems and optimization techniques, monitoring and diagnosis, and automated design and expert systems. Attention is given to the integration of AI reasoning systems and hardware description languages, care-based reasoning, knowledge, retrieval, and training systems, and scheduling and planning. Topics addressed include the preprocessing of remotely sensed data for efficient analysis and classification, autonomous agents as air combat simulation adversaries, intelligent data presentation for real-time spacecraft monitoring, and an integrated reasoner for diagnosis in satellite control. Also discussed are a knowledge-based system for the design of heat exchangers, reuse of design information for model-based diagnosis, automatic compilation of expert systems, and a case-based approach to handling aircraft malfunctions.

Fayyad, Usama M.↗

Distributed fiber-optic sensing in a subscale high-temperature superconducting dipole magnet

High-temperature superconductors, such as REBa2Cu3O7−x (REBCO, RE = rare earth), are becoming pivotal for high-field magnet technology for future circular colliders and compact fusion reactors. The U.S. Magnet Development Program, in collaboration with industry, is developing REBCO magnet technology using round conductors consisting of multiple REBCO tapes. For these multi-tape cables, traditional instrumentation, such as voltage taps and resistive strain gauges, become insufficient to help measure and understand the performance-limiting factors in these model magnets. Distributed fiber-optic sensing (DFOS) is a potential solution to address this challenge. Although DFOS is well established for various applications, measuring temperature and strain in high-temperature superconducting magnets is in its infancy. Here we report the detailed implementation and test results of DFOS based on Rayleigh scattering in a subscale canted cosθ (CCT) dipole magnet using high-temperature superconducting CORC® wires. We co-wound optical fibers in each layer of the CCT magnet and compared different types of commercial fibers and mold-release agents to reduce the power attenuation in the fibers. The DFOS allowed us to measure mechanical deformation and temperature along the conductor during tests at 77 and 4.2 K. The measured strain agreed quantitively with a finite-element mechanical model of the subscale magnet. Our results indicate that DFOS can effectively identify locations of strain and temperature changes, offering unique insight into magnet performance that can advance our understanding and development of the REBCO magnet technology for high-energy physics and fusion applications.

Luo, Linqing↗

GridCoPilot for Thermal Events: An LLM-Based Platform for Power Grid Reliability Analysis

Large Language Models show promise for translating natural language into database queries, but deploying such systems in safety-critical domains requires high reliability. We present an application of GridCoPilot to thermal event analysis (heatwaves and coldwaves) that affect power grid reliability. Our approach uses a LangChain SQL Agent to translate natural language queries into auditable SQL statements, with deterministic visualization routines that parse the structured query results. We introduce structural framing as a design principle, we integrate a NERC-region-level event library with county-level meteorology and decompose the combined data into three relational tables (event metadata, county-level event details, and a county-to-NERC subregion mapping), using prompt-guided joins to direct the model toward correct multi-table queries. For two core analytical patterns (identifying worst events by region and by region-year), the system achieved 100% SQL accuracy across all 16 NERC subregions and both event types (64 queries total). These results validate the approach for target use cases, though performance on diverse natural language formulations requires further investigation. We discuss design trade-offs, failure modes including JSON output truncation, and pathways for extending this approach to other hazard domains.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Aera-Mip: Emission Pathways, Remaining Budgets, and Carbon Cycle Dynamics Compatible With 1.5 and 2 °C Global Warming Stabilization

While international climate policies now focus on limiting global warming to well below 2 °C or pursuing a 1.5 °C level of global warming, the climate modelling community has not provided an experimental design in which all Earth system models (ESMs) converge and stabilize at the same prescribed global warming levels. This gap hampers accurate estimations based on comprehensive ESMs of the carbon emission pathways and budgets needed to meet such agreed warming levels and of the associated climate impacts under temperature stabilization. Here, we apply the Adaptive Emission Reduction Approach (AERA) with ESMs to provide such simulations in which all models converge at 1.5 and 2.0 °C warming levels by adjusting their emissions over time. These emission-driven simulations provide a wide range of emission pathways and resulting atmospheric CO 2 projections for a given warming level, uncovering uncertainty ranges that were previously missing in the traditional Coupled Model Intercomparison Project (CMIP) scenarios with prescribed greenhouse gas concentration pathways. Meeting the 1.5 °C warming level requires a 40 % (full model range: 7 % to 76 %) reduction in multi-model mean CO 2 -forcing-equivalent (CO 2 -fe) emissions from 2025 to 2030, a 98 % (57 % to 127 %) reduction from 2025 to 2050, and a stabilization at 1.0 (−1.7 to 2.9) PgC yr−1 from 2100 onward after the 1.5 °C global warming level is reached. Meeting the 2.0 °C warming level requires a 47 % (8 % to 92 %) reduction in multi-model mean CO 2 -fe emissions until 2050 and a stabilization at 1.7 (−1.5 to 2.7) PgC yr−1 from 2100 onward. The on-average positive emissions under stabilized global temperatures are the result of a decreasing transient climate response to cumulative CO 2 -fe emissions over time under stabilized global warming. This evolution is consistent with a slightly negative zero emissions commitment – initially assumed to be zero – and leads to an increase in the post-2025 CO 2 -fe emission budget by a factor of 2.2 (−0.8 to 6.9) by 2150 for the 1.5 °C warming level and a factor of 1.4 (0.9 to 2.4) for the 2.0 °C warming level compared to its first estimate in 2025. The median CO 2 -only carbon budget by 2150, relative to 2020, is 800 GtCO 2 for the 1.5 °C warming level and 2250 GtCO 2 for the 2.0 °C warming level. These median values exceed the median IPCC AR6 estimates by 60 % for the 1.5 °C warming level and 67 % for 2.0 °C. Some of the differences may be explained by the choice of the mitigation scenario for non-CO 2 radiative agents. Our simulations highlight shifts in carbon uptake dynamics under stabilized temperature, such as a cessation of the carbon sinks in the North Atlantic and in tropical forests. On the other hand, the Southern Ocean remains a carbon sink centuries after temperatures stabilize. Overall, this new type of warming-level-based emission-driven simulation offers a more coherent assessment across climate models and opens up a wide range of possibilities for studying both the carbon cycle and climate impacts, such as extreme events, under climate stabilization.

Adaptive Emission Reduction Approach↗

Transactive Energy Systems: The Market-based Coordination of Distributed Energy Resources

This paper synthesizes a unifying framework that captures a large class of transactive energy systems studied in the literature. The framework consists of four key elements: agent preferences, control decisions, information structure, and solution concept. Among them, agent preference specifies the preferred outcome by each agent and models the conflict of interest in decision making. Control decision specifies the set of decisions for each agent to maximize its payoff. Information structure describes the information available to each agent before it makes a decision. Solution concept encodes the rationality assumptions of each agent in the system. These elements are important in identifying and distinguishing different transactive energy systems. In this paper, we use them to analyze a number of important ones, including the competitive equilibrium, Stackelberg game, reverse Stackelberg game, and mechanism design. Each class of these problems is formulated as a special case of the proposed transactive energy systems. Their connections and differences are discussed. Available tools and results for each category of problems are surveyed.

Li, Sen↗