Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Model-based Executive Control through Reactive Planning for Autonomous Rovers

This paper reports on the design and implementation of a real-time executive for a mobile rover that uses a model-based, declarative approach. The control system is based on the Intelligent Distributed Execution Architecture (IDEA), an approach to planning and execution that provides a unified representational and computational framework for an autonomous agent. The basic hypothesis of IDEA is that a large control system can be structured as a collection of interacting agents, each with the same fundamental structure. We show that planning and real-time response are compatible if the executive minimizes the size of the planning problem. We detail the implementation of this approach on an exploration rover (Gromit an RWI ATRV Junior at NASA Ames) presenting different IDEA controllers of the same domain and comparing them with more classical approaches. We demonstrate that the approach is scalable to complex coordination of functional modules needed for autonomous navigation and exploration.

Finzi, Alberto↗
NASA NTRS2012

Explanation Capabilities for Behavior-Based Robot Control

A recent study that evaluated issues associated with remote interaction with an autonomous vehicle within the framework of grounding found that missing contextual information led to uncertainty in the interpretation of collected data, and so introduced errors into the command logic of the vehicle. As the vehicles became more autonomous through the activation of additional capabilities, more errors were made. This is an inefficient use of the platform, since the behavior of remotely located autonomous vehicles didn't coincide with the "mental models" of human operators. One of the conclusions of the study was that there should be a way for the autonomous vehicles to describe what action they choose and why. Robotic agents with enough self-awareness to dynamically adjust the information conveyed back to the Operations Center based on a detail level component analysis of requests could provide this description capability. One way to accomplish this is to map the behavior base of the robot into a formal mathematical framework called a cost-calculus. A cost-calculus uses composition operators to build up sequences of behaviors that can then be compared to what is observed using well-known inference mechanisms.

Huntsberger, Terrance L.↗
DOE OSTI2022

Improved Particle Swarm Algorithm Using Rubik’s Cube Topology for Bilevel Building Energy Transaction

Following the rapid growth of distributed energy resources (e.g., renewables, battery), localized peer-to-peer energy transactions are receiving more attention for multiple benefits, such as reducing power loss and stabilizing the main power grid. To promote distributed renewables locally, the local trading price is usually set to be within the external energy purchasing and selling price range. Consequently, building prosumers are motivated to trade energy through a local transaction center. This local energy transaction is modeled in bilevel optimization game. A selfish upper level agent is assumed with the privilege to set the internal energy transaction price with an objective of maximizing its arbitrage profit. Meanwhile, the building prosumers at the lower level will response to this transaction price and make decisions on electricity transaction amount. Therefore, this non-cooperative leader-follower trading game is seeking for equilibrium solutions on the energy transaction amount and prices. Additionally, a uniform local transaction price structure (purchase price equals selling price) is considered here. Aiming at reducing the computational burden from classical Karush–Kuhn–Tucker (KKT) transformation and protecting the private information of each stakeholder (e.g., building), swarm intelligence-based solution approach is employed for upper level agent to generate trading price and coordinate the transactive operations. On one hand, to decrease the chance of premature convergence in global-best topology, Rubik’s Cube topology is proposed in this study based on further improvement of a two-dimensional square lattice model (i.e., one local-best topology-Von Neumann topology). Rotating operation of the cube is introduced to dynamically changing the neighborhood and enhancing information flow at the later searching state. Several groups of experiments are designed to evaluate the performance of proposed Rubik’s Cube topology-based particle swarm algorithm. The results have validated the effectiveness of proposed topology and operators comparing with global-best version PSO and Von Neumann topology-based PSO and its scalability on larger scale applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Explainable physics-based constraints on reinforcement learning for accelerator optimization

We present a reinforcement learning (RL) framework for optimizing particle accelerator experiments that builds explainable physics-based constraints on agent behavior. The goal is to increase transparency and trust by letting users verify that the agent’s decision-making process incorporates suitable physics. Our algorithm uses a learnable surrogate function for physical observables, such as energy, and uses them to fine-tune how actions are chosen. This surrogate can be represented by a neural network or by an interpretable sparse dictionary model. We test our algorithm on a range of particle accelerator optimization environments designed to emulate the Continuous Electron Beam Accelerator Facility at Jefferson Lab. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment. In addition, we find that the introduction of a physics-based surrogate enables our RL algorithms to reliably converge for difficult high-dimensional accelerator optimization environments.

explainability↗
DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗
DOE OSTI2023

Understanding racial and ethnic disparities in COVID-19 outcomes using an agent-based simulation [Poster]

Infectious disease outbreaks pose a major threat to public health, economic stability, and human security. In order to mitigate disease impacts, we need to understand drivers that contribute to its spread. Latinx, Black, and American Indian racial and ethnic groups experienced disproportionate health outcomes during the COVID-19 pandemic. Current modeling approaches often fail to capture the high level of heterogeneity present in the population needed to measure the drivers of these disparities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Optimizing Patient-Specific Medication Regimen Policies Using Wearable Sensors in Parkinson’s Disease

Effective treatment of Parkinson’s disease (PD) is a continual challenge for healthcare providers, and providers can benefit from leveraging emerging technologies to supplement traditional clinic care. We develop a data-driven reinforcement learning (RL) framework to optimize PD medication regimens through wearable sensors. We leverage a data set of n = 26 PD patients who wore wrist-mounted movement trackers for two separate six-day periods. Using these data, we first build and validate a simulation model of how individual patients’ movement symptoms respond to medication administration. We then pair this simulation model with an on-policy RL algorithm that recommends optimal medication types, timing, and dosages during the day while incorporating human-in-the-loop considerations on medication administration. The results show that the RL-prescribed medication regimens outperform physicians’ medication regimens, despite physicians having access to the same data as the RL agent. To validate our results, we assess our wearable-based RL medication regimens using n = 399 PD patients from the Parkinson’s Progression Markers Initiative data set. We show that the wearable-based RL medication regimens would lead to significant symptom improvement for these patients, even more so than training RL policies directly from this data set. In doing so, we show that RL models from even small data sets of wearable data can offer novel, generalizable clinical insights and medication strategies, which may outperform those derived from larger data sets without wearable data.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

A Simplified Approach Based on Cellular Automata for Describing Direct Reduced Iron Production in Different Reducing Conditions

A quick computation approach based on cellular automata is developed and implemented to describe the reduction of iron ore pellets by a mixture of reducing agents featured by different H 2 /CO ratios. The evolution of oxygen concentration inside the pellet is followed from the beginning to the end of contact between reducing agent and pellet. The variation of thermal state of pellets and gas mixture is computed based on their initial temperature, considering the heat involved and the convective heat exchange between pellet and gas mixture. The use of cellular automata and finite‐difference method to solve the diffusion equation point out the absence of any diffusion coefficient value, allowing to make the model fit the experimental trial, because the problem is that it is not ruled just by diffusion but also by the concentration variation of reducing agent inside the pellet due to porosity increasing during reduction. The updating of the reducing agents concentration implies a sharp decrease of oxygen concentration that the cellular automata model considers. The developed model is able to provide the in‐line control of reduction process and could be used to adjust the chemical concentration and temperature of injected reducing agents.

Metallurgy & Metallurgical Engineering↗
NASA NTRS2004

Two Formal Gas Models For Multi-Agent Sweeping and Obstacle Avoidance

The task addressed here is a dynamic search through a bounded region, while avoiding multiple large obstacles, such as buildings. In the case of limited sensors and communication, maintaining spatial coverage - especially after passing the obstacles - is a challenging problem. Here, we investigate two physics-based approaches to solving this task with multiple simulated mobile robots, one based on artificial forces and the other based on the kinetic theory of gases. The desired behavior is achieved with both methods, and a comparison is made between them. Because both approaches are physics-based, formal assurances about the multi-robot behavior are straightforward, and are included in the paper.

Kerr, Wesley↗
DOE OSTI2026

SetGo: Metadata Readiness for Scientific AI Datasets

Scientific datasets intended for AI use require both computational readiness for model training and metadata readiness for discovery, sharing, and reuse. The Readiness Engine for Data Integration (REDI) addresses computational readiness, but no corresponding tool evaluates whether a dataset’s metadata are sufficiently complete, governed, and standards-compliant for publication and agent-based consumption. Existing FAIR assessors operate only on published repository records, and no single system covers FAIR compliance, licensing, provenance, governance, reproducibility, and catalog readiness together. We present SetGo, an open-source Python toolkit that assesses and repairs metadata readiness across these six dimensions before a dataset is published or archived. Applied to four scientific corpora, SetGo surfaces deficiencies that general-purpose tools do not detect: ERA5 climate metadata scores 4% on ACDD 1.3 compliance; materials datasets fail OPTIMADE species-definition requirements; and PDB-derived proteomics data carries licensing terms incompatible with standard SPDX identifiers. Guided enrichment raises overall FAIR scores from 52–57% to 81–91%, and a single setgo publish command pushes to Hugging Face Hub, CKAN, or OpenMetadata with ML Commons Croissant 1.0 metadata sidecars. To support interactive and automated workflows, SetGo integrates with coding agents powered by large language models (LLMs) through a /setgo skill that enables natural-language execution of the full assess–enrich–publish loop, with user involvement limited to supplying missing metadata values.

Wilkinson, Sean [ORNL] (ORCID:0000000214437479)↗
NASA NTRS2008

Model Error Budgets

An error budget is a commonly used tool in design of complex aerospace systems. It represents system performance requirements in terms of allowable errors and flows these down through a hierarchical structure to lower assemblies and components. The requirements may simply be 'allocated' based upon heuristics or experience, or they may be designed through use of physics-based models. This paper presents a basis for developing an error budget for models of the system, as opposed to the system itself. The need for model error budgets arises when system models are a principle design agent as is increasingly more common for poorly testable high performance space systems.

error budgets↗
DOE OSTI2020

Integrated Dynamic Facade Control with an Agent-based Architecture for Commercial Buildings

Dynamic façades have significant technical potential to minimize heating, cooling, and lighting energy use and peak electric demand in the perimeter zone of commercial buildings, but the performance of these systems is reliant on being able to balance complex trade-offs between solar control, daylight admission, comfort, and view over the life of the installation. As the context for controllable energy-efficiency technologies grows more complex with the increased use of intermittent renewable energy resources on the grid, it has become increasingly important to look ahead towards more advanced approaches to integrated systems control in order to achieve optimum life-cycle performance at a lower cost. This study examines the feasibility of a model predictive control system for low-cost autonomous dynamic façades. A system architecture designed around lightweight, simple agents is proposed. The architecture accommodates whole building and grid level demands through its modular, hierarchical approach. Automatically-generated models for computing window heat gains, daylight illuminance, and discomfort glare are described. The open source Modelica and JModelica software tools were used to determine the optimum state of control given inputs of window heat gains and lighting loads for a 24-hour optimization horizon. Penalty functions for glare and view/ daylight quality were implemented as constraints. The control system was tested on a low-power controller (1.4 GHz single core with 2 GB of RAM) to evaluate feasibility. The target platform is a low-cost ($35/unit) embedded controller with 1.2 GHz dual-core cpu and 1 GB of RAM. Configuration and commissioning of the curtainwall unit was designed to be largely plug and play with minimal inputs required by the manufacturer through a web-based user interface. An example application was used to demonstrate optimal control of a three-zone electrochromic window for a south-facing zone. The overall approach was deemed to be promising. Further engineering is required to enable scalable, turnkey solutions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Multi-Agent Safe Policy Learning for Power Management of Networked Microgrids

This paper presents a supervised multi-agent safe policy learning (SMAS-PL) method for optimal power management of networked microgrids (MGs) in distribution systems. While unconstrained reinforcement learning (RL) algorithms are black-box decision models that could fail to satisfy grid operational constraints, our proposed method considers AC power flow equations and other operational limits. Accordingly, the training process employs the gradient information of operational constraints to ensure that the optimal control policy functions generate safe and feasible decisions. Furthermore, we have developed a distributed consensus-based optimization approach to train the agents’ policy functions while maintaining MGs’ privacy and data ownership boundaries. After training, the learned optimal policy functions can be safely used by the MGs to dispatch their local resources, without the need to solve a complex optimization problem from scratch. Lastly, numerical experiments have been devised to verify the performance of the proposed method.

, policy gradient↗
NASA NTRS2014

Evolutionary Agent-Based Simulation of the Introduction of New Technologies in Air Traffic Management

Accurate simulation of the effects of integrating new technologies into a complex system is critical to the modernization of our antiquated air traffic system, where there exist many layers of interacting procedures, controls, and automation all designed to cooperate with human operators. Additions of even simple new technologies may result in unexpected emergent behavior due to complex human/ machine interactions. One approach is to create high-fidelity human models coming from the field of human factors that can simulate a rich set of behaviors. However, such models are difficult to produce, especially to show unexpected emergent behavior coming from many human operators interacting simultaneously within a complex system. Instead of engineering complex human models, we directly model the emergent behavior by evolving goal directed agents, representing human users. Using evolution we can predict how the agent representing the human user reacts given his/her goals. In this paradigm, each autonomous agent in a system pursues individual goals, and the behavior of the system emerges from the interactions, foreseen or unforeseen, between the agents/actors. We show that this method reflects the integration of new technologies in a historical case, and apply the same methodology for a possible future technology.

Simulation↗
NASA NTRSDate not supplied

A Distributed Simulation-to-Flight Framework to Support Investigating Trust/Trustworthiness in Multi-Agent Systems

As autonomous systems continue to grow both in use and complexity, the necessity for robust and extensible simulation-to-flight methods is paramount for establishing an effective architecture for autonomous systems. A fundamental objective of the ATTRACTOR (Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability) project was to design and develop a distributed mixed-reality simulation environment to begin establishing a basis for certification of autonomous systems via research into trust and trustworthiness. In this paper, we present an autonomous systems architecture and development framework paired with a persistent distributed modeling and simulation environment for test and evaluation of autonomous systems. The Autonomous Entity Operations Network (AEON) framework enables autonomous system development with an easily extensible collection of libraries and plug-n-play nodes facilitated by the Data Distribution Service (DDS) communication protocol standard. The Baseline Environment for Autonomous Modeling (BEAM) simulation environment is a distributed mixed-reality Unity™-based environment built around the same DDS communication paradigm allowing for easy integration with AEON-based autonomous applications. They were designed under ATTRACTOR in order to measure and establish trustworthiness and trust in single- and multi-agent human-machine systems whether these machines are fixed-wing general aviation, rotary-wing Unmanned Aerial Vehicles (UAVs), ground rovers, or even spacecraft. Together AEON and BEAM enable sim-to-flight with minimal configuration changes. By using AEON and BEAM, source code that runs in simulation ports directly to hardware and has successfully flown in the lab and in the National Airspace System (NAS) at NASA LaRC many times over the lifetime of ATTRACTOR.

Benjamin N Kelley↗
NASA NTRSDate not supplied

A Distributed Simulation-to-Flight Framework to Support Investigating Trust/Trustworthiness in Multi-Agent Systems

As autonomous systems continue to grow both in use and complexity, the necessity for robust and extensible simulation-to-flight methods is paramount for establishing an effective architecture for autonomous systems. A fundamental objective of the ATTRACTOR (Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability) project was to design and develop a distributed mixed-reality simulation environment to begin establishing a basis for certification of autonomous systems via research into trust and trustworthiness. In this paper, we present an autonomous systems architecture and development framework paired with a persistent distributed modeling and simulation environment for test and evaluation of autonomous systems. The Autonomous Entity Operations Network (AEON) framework enables autonomous system development with an easily extensible collection of libraries and plug-n-play nodes facilitated by the Data Distribution Service (DDS) communication protocol standard. The Baseline Environment for Autonomous Modeling (BEAM) simulation environment is a distributed mixed-reality Unity™-based environment built around the same DDS communication paradigm allowing for easy integration with AEON-based autonomous applications. They were designed under ATTRACTOR in order to measure and establish trustworthiness and trust in single- and multi-agent human-machine systems whether these machines are fixed-wing general aviation, rotary-wing Unmanned Aerial Vehicles (UAVs), ground rovers, or even spacecraft. Together AEON and BEAM enable sim-to-flight with minimal configuration changes. By using AEON and BEAM, source code that runs in simulation ports directly to hardware and has successfully flown in the lab and in the National Airspace System (NAS) at NASA LaRC many times over the lifetime of ATTRACTOR.

Benjamin N Kelley↗
DOE OSTI2021

Unbundling Smart Meter Services Through Spatio-Temporal Decomposition Agents in DER-rich Environment

Smart meters and the advanced metering infrastructure (AMI) facilitate distribution system operators (DSOs) to gather information on energy consumption at the customer level. With the increasing penetration of building-level intermittent distributed energy resources (DERs) behind the meter, DER information is not available to DSOs. At the same time, smart meter enables users to participate in grid, with real-time information. Information for behind the meter is needed by user to coordinate building level assets for maximum benefits. The concept of unbundled smart meter (USM) needs agents to decompose smart meter measurements to provide service to DSO as well as customers. In this paper, we propose a Spatio-Temporal Decomposition Agent (STDA) for USM based on Artificial Intelligence (AI). STDA can help users optimize their energy usage, help DSO to utilize building assets for the grid operation. The energy usage strategy developed by STDA is suitable for different users, and can be customized by deep learning (DL) models according to the different energy consumption habits of each user. The power prediction performance results of various DL models and evaluation using a set of data from a Hawaii utility is presented. Furthermore, STDA integration with Home Energy management Systems (HEMS) to manage resources is presented and validated. STDA pre-processes the measurements before model training, and provides the spatio-temporal decomposed forecasting.

42 ENGINEERING↗