Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Self-healing capability is a critical factor for a resilient distribution system, which requires intelligent agents to automatically perform service restoration online, including network reconfiguration and reactive power dispatch. Here, the article proposes the imitation learning framework for training such an agent, where the agent will interact with an expert built based on the mixed-integer program to learn its optimal policy, and therefore significantly improve the training efficiency compared with exploration-dominant reinforcement learning (RL) methods. This significantly improved training efficiency makes the training problem under N-k scenarios tractable. A hybrid policy network is proposed to handle tie-line operations and reactive power dispatch simultaneously to further improve the restoration performance. The 33-bus and 119-bus systems with N-k disturbances are employed to conduct the training. The results indicate that the proposed method outperforms traditional RL algorithms such as the deep-Q network.

42 ENGINEERING↗
DOE OSTI2020

Microbial Diversity Associated with the Pollen Stores of Captive-Bred Bumble Bee Colonies

The pollen stores of bumble bees host diverse microbiota that influence overall colony fitness. Yet, the taxonomic identity of these symbiotic microbes is relatively unknown. In this descriptive study, we characterized the microbial community of pollen provisions within captive-bred bumble bee hives obtained from two commercial suppliers located in North America. Findings from 16S rRNA and ITS gene-based analyses revealed that pollen provisions from the captive-bred hives shared several microbial taxa that have been previously detected among wild populations. While diverse microbes across phyla Firmicutes, Proteobacteria, Bacteroidetes, Actinobacteria, and Ascomycota were detected in all commercial hives, significant differences were detected at finer-scale taxonomic resolution based on the supplier source. The causative agent of chalkbrood disease in honey bees, Ascosphaera apis, was detected in all hives obtained from one supplier source, although none of the hives showed symptoms of infection. The shared core microbiota across both commercial supplier sources consisted of two ubiquitous bee-associated groups, Lactobacillus and Wickerhamiella/Starmerella clade yeasts that potentially contribute to the beneficial function of the microbiome of bumble bee pollen provisions.

16S rRNA gene↗
NASA NTRS1989

Cooperating intelligent systems

Some of the issues connected to the development of a bureaucratic system are discussed. Emphasis is on a layer multiagent approach to distributed artificial intelligence (DAI). The division of labor in a bureaucracy is considered. The bureaucratic model seems to be a fertile model for further examination since it allows for the growth and change of system components and system protocols and rules. The first part of implementing the system would be the construction of a frame based reasoner and the appropriate B-agents and E-agents. The agents themselves should act as objects and the E-objects in particular should have the capability of taking on a different role. No effort was made to address the problems of automated failure recovery, problem decomposition, or implementation. Instead what has been achieved is a framework that can be developed in several distinct ways, and which provides a core set of metaphors and issues for further research.

Rochowiak, Daniel↗
NASA NTRS1997

Field Evaluation of Anti-Biofouling Compounds on Optical Instrumentation

Biofouling has been a serious question in the stability of optical measurements in the ocean, particularly in moored and drifting buoy applications. Many investigators coat optical surfaces with various compounds to reduce the amount of fouling; to our knowledge, however, there are no objective, in-situ comparative testing of these compounds to evaluate their effectiveness with respect to optical stability relative to untreated controls. We have tested a wide range of compounds at in-situ locations in Halifax Harbour and in the Adriatic Sea on passive optical sensors. Compounds tested include a variety of TBT formulations, antifungal agents, and low-friction silicone-based compounds; time-scales of up to four months were evaluated. The results of these experiments are discussed.

McLean, Scott↗
NASA NTRS2001

Learning Sequences of Actions in Collectives of Autonomous Agents

In this paper we focus on the problem of designing a collective of autonomous agents that individually learn sequences of actions such that the resultant sequence of joint actions achieves a predetermined global objective. We are particularly interested in instances of this problem where centralized control is either impossible or impractical. For single agent systems in similar domains, machine learning methods (e.g., reinforcement learners) have been successfully used. However, applying such solutions directly to multi-agent systems often proves problematic, as agents may work at cross-purposes, or have difficulty in evaluating their contribution to achievement of the global objective, or both. Accordingly, the crucial design step in multiagent systems centers on determining the private objectives of each agent so that as the agents strive for those objectives, the system reaches a good global solution. In this work we consider a version of this problem involving multiple autonomous agents in a grid world. We use concepts from collective intelligence to design goals for the agents that are 'aligned' with the global goal, and are 'learnable' in that agents can readily see how their behavior affects their utility. We show that reinforcement learning agents using those goals outperform both 'natural' extensions of single agent algorithms and global reinforcement, learning solutions based on 'team games'.

Turner, Kagan↗
DOE OSTI2024

Reinforcement Learning to Enhance Optimal Operation of Resilient Community Energy Systems

This paper presents a novel model-free multi-agent Reinforcement Learning (RL) control method to enhance the resilience of community energy systems in island mode, which coordinates multiple objectives without the necessity of identifying system models that require expert knowledge. Specifically, a community-level coordinator agent is designed to allocate renewable energy resources among different buildings, and multiple building-level agents are developed to optimize load schedules based on limited energy resources and requirements of building loads and occupants’ comfort. In a two-day evaluation, our RL approach demonstrated a similar performance against MPC without requiring system models and formulation of optimization problems as required in MPC.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗
DOE OSTI2022

Improved Particle Swarm Algorithm Using Rubik’s Cube Topology for Bilevel Building Energy Transaction

Following the rapid growth of distributed energy resources (e.g., renewables, battery), localized peer-to-peer energy transactions are receiving more attention for multiple benefits, such as reducing power loss and stabilizing the main power grid. To promote distributed renewables locally, the local trading price is usually set to be within the external energy purchasing and selling price range. Consequently, building prosumers are motivated to trade energy through a local transaction center. This local energy transaction is modeled in bilevel optimization game. A selfish upper level agent is assumed with the privilege to set the internal energy transaction price with an objective of maximizing its arbitrage profit. Meanwhile, the building prosumers at the lower level will response to this transaction price and make decisions on electricity transaction amount. Therefore, this non-cooperative leader-follower trading game is seeking for equilibrium solutions on the energy transaction amount and prices. Additionally, a uniform local transaction price structure (purchase price equals selling price) is considered here. Aiming at reducing the computational burden from classical Karush–Kuhn–Tucker (KKT) transformation and protecting the private information of each stakeholder (e.g., building), swarm intelligence-based solution approach is employed for upper level agent to generate trading price and coordinate the transactive operations. On one hand, to decrease the chance of premature convergence in global-best topology, Rubik’s Cube topology is proposed in this study based on further improvement of a two-dimensional square lattice model (i.e., one local-best topology-Von Neumann topology). Rotating operation of the cube is introduced to dynamically changing the neighborhood and enhancing information flow at the later searching state. Several groups of experiments are designed to evaluate the performance of proposed Rubik’s Cube topology-based particle swarm algorithm. The results have validated the effectiveness of proposed topology and operators comparing with global-best version PSO and Von Neumann topology-based PSO and its scalability on larger scale applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Identifying Molecular Scale Interactions of Nucleating Agents with Salt Hydrates for Thermal Energy Storage Applications

Latent heat based thermal energy storage is of interest as a method to mitigate and time-shift thermal load peaks and hence reduce energy demands for heating and cooling buildings. Notably, Glauber’s salt (GS) presents a good choice as a phase change material (PCM) for building applications because it has a melting point near room temperature (i.e., 32.4 °C), a large heat of fusion and energy density, and is low cost (~ $100/ton). However, there are several known limitations to using GS as a PCM, such as incongruent melting, high degree of supercooling, and formation of other hydration states, which renders GS unsuitable for practical use. Inorganic crystals acting as nucleating agents can avoid some of these issues with GS [1]. Understanding nucleation behavior through typical calorimetry experiments offers limited insight into the molecular-scale mechanisms. Here, we report isothermal titration calorimetry [2] to investigate the interactions between sodium sulfate salt and an organic nucleating agent - glycine. Our experiments provide qualitative and quantitative understanding of how glycine interacts with sodium sulfate across various temperatures. The findings offer insights into the nucleation onset temperature (24°C) and underscore the pivotal role of glycine's zwitterionic structure in facilitating nucleation. This preliminary work provides a foundation for subsequent exploration of nucleating agents that would be incorporated in different salt hydrate systems. 1. Purohit, B. K., and V. S. Sistla. "Inorganic salt hydrate for thermal energy storage application: A review." Energy Storage 3, no. 2 (2021): e212. 2. Velázquez-Campoy, Adrián, Hiroyasu Ohtaka, Azin Nezami, Salman Muzammil, and Ernesto Freire. "Isothermal titration calorimetry." Current protocols in cell biology 23, no. 1 (2004): 17-8.

calorimetry, nucleation & growth↗
DOE OSTI2023

Convex Decreasing Algorithms: Distributed Synthesis and Finite-Time Termination in Higher Dimension

Here we establish finite time termination algorithms for consensus algorithms based on geometric properties that yield finite-time guarantees, suited for use in high dimension and in the absence of a central authority. These pursuits motivate a new peer to peer convex hull algorithm which is utilized for one stopping algorithm. Further an alternative lightweight norm based stopping criteria is also developed. The practical utility of the algorithm is illustrated through MATLAB simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2001

Enabling Advanced Automation in Spacecraft Operations with the Spacecraft Emergency Response System

True autonomy is the Holy Grail of spacecraft mission operations. The goal of launching a satellite and letting it manage itself throughout its useful life is a worthy one. With true autonomy, the cost of mission operations would be reduced to a negligible amount. Under full autonomy, any problems (no matter the severity or type) that may arise with the spacecraft would be handled without any human intervention via some combination of smart sensors, on-board intelligence, and/or smart automated ground system. Until the day that complete autonomy is practical and affordable to deploy, incremental steps of deploying ever-increasing levels of automation (computerization of once manual tasks) on the ground and on the spacecraft are gradually decreasing the cost of mission operations. For example, NASA's Goddard Space Flight Center (NASA-GSFC) has been flying spacecraft with low cost operations for several years. NASA-GSFC's SMEX (Small Explorer) and MIDEX (Middle Explorer) missions have effectively deployed significant amounts of automation to enable the missions to fly predominately in 'light-out' mode. Under light-out operations the ground system is run without human intervention. Various tools perform many of the tasks previously performed by the human operators. One of the major issues in reducing human staff in favor of automation is the perceived increased in risk of losing data, or even losing a spacecraft, because of anomalous conditions that may occur when there is no one in the control center. When things go wrong, missions deploying advanced automation need to be sure that anomalous conditions are detected and that key personal are notified in a timely manner so that on-call team members can react to those conditions. To ensure the health and safety of its lights-out missions, NASA-GSFC's Advanced Automation and Autonomy branch (Code 588) developed the Spacecraft Emergency Response System (SERS). The SERS is a Web-based collaborative environment that enables secure distributed fault and resource management. The SERS incorporates the use of intelligent agents, threaded discussions, workflow, database connectivity, and links to a variety of communications devices (e.g., two-way paging, PDA's, and Internet phones) via commercial gateways. When the SERS detects a problem, it notifies on-call team members, who then can remotely take any necessary actions to resolve the anomalies.The SERS goes well beyond a simple '911' system that sends out an error code to everyone with a pager. Instead, SERS' software agents send detailed data (i.e., notifications) to the most appropriate team members based on the type and severity of the anomaly and the skills of the on-call team members. The SERS also allows the team members to respond to the notifications from their wireless devices. This unique capability ensures rapid response since the team members no longer have to go to a PC or the control center for every anomalous event. Most importantly, the SERS enables safe experimentation with various techniques for increasing levels of automation, leading to robust autonomy. For the MIDEX missions at NASA GSFC, the SERS is used to provide 'human-in-the-loop' automation. During lights-out operations, as greater control is given to the MIDEX automated systems, the SERS can be configured to page remote personnel and keep them informed regarding actions taking place in the control center. Remote off-duty operators can even be given the option of enabling or inhibiting a specific automated response in near real time via their two-way pagers. The SERS facilitates insertion of new technology to increase automation, while maintaining the safety and security of mission resources. This paper will focus on SERS' overall functionality and how SERS has been designed to handle the monitoring and emergency response for missions with varying levels of automation. The paper will also convey some of the key lessons learned from SERS' deployment across of variety of missions, highlighting this incremental approach to achieving 'robust autonomy'.

Breed, Julie↗
NASA NTRS2010

BGen: A UML Behavior Network Generator Tool

BGen software was designed for autogeneration of code based on a graphical representation of a behavior network used for controlling automatic vehicles. A common format used for describing a behavior network, such as that used in the JPL-developed behavior-based control system, CARACaS ["Control Architecture for Robotic Agent Command and Sensing" (NPO-43635), NASA Tech Briefs, Vol. 32, No. 10 (October 2008), page 40] includes a graph with sensory inputs flowing through the behaviors in order to generate the signals for the actuators that drive and steer the vehicle. A computer program to translate Unified Modeling Language (UML) Freeform Implementation Diagrams into a legacy C implementation of Behavior Network has been developed in order to simplify the development of C-code for behavior-based control systems. UML is a popular standard developed by the Object Management Group (OMG) to model software architectures graphically. The C implementation of a Behavior Network is functioning as a decision tree.

Huntsberger, Terry↗
DOE OSTI2023

DSS-SimPy-RL (Open-DSS and SimPy based Cyber-Physical RL environment) [SWR-23-29]

Recently, numerous data-driven approaches to control an electric grid using machine learning techniques have been investigated. With the advancement of reinforcement learning (RL) based techniques, gradually the conventional optimization based solvers are being replaced with RL approach where there is uncertainty in the environment such as renewable generation or cyber system emulation. However, to train an agent efficiently, it requires numerous interactions with an environment to learn the best policies. There are numerous RL environments for the power systems based on some well-known simulators, similarly there are environment for communication domains. While majority of the cyber emulators are based in an UNIX environment, the power simulators are based in the Windows-based operating system, the generation of cyber-physical mixed domain RL environment has been challenging. Existing co-simulation methods are efficient but resource and time intensive to generate large scale data set for training RL agents. Hence, this software focuses on development and validation of a mixed domain RL environment using Open DSS for the physical side and leverages a discrete event simulator python package, SimPy, for cyber-side emulation which is Operating Systems agnostic. Further utilizing this software co-simulation and training RL agents for re-routing based resilient control for network reconfiguration and volt-var control in power distribution feeder are performed.

Sahu, Abhijeet↗
DOE OSTI2020

Rational design, synthesis, and evaluation of uncharged, “smart” bis-oxime antidotes of organophosphate-inhibited human acetylcholinesterase

Organophosphate (OP) intoxications from nerve agent and OP pesticide exposures are managed with pyridinium aldoxime–based therapies whose success rates are currently limited. The pyridinium cation hampers uptake of OPs into the central nervous system (CNS). Furthermore, it frequently binds to aromatic residues of OP-inhibited acetylcholinesterase (AChE) in orientations that are nonproductive for AChE reactivation, and the structural diversity of OPs impedes efficient reactivation. Improvements of OP antidotes need to include much better access of AChE reactivators to the CNS and optimized orientation of the antidotes' nucleophile within the AChE active-center gorge. On the basis of X-ray structures of a CNS-penetrating reactivator, monoxime RS194B, reversibly bound to native and venomous agent X (VX)–inhibited human AChE, here we created seven uncharged acetamido bis-oximes as candidate antidotes. Both oxime groups in these bis-oximes were attached to the same central, saturated heterocyclic core. Diverse protonation of the heterocyclic amines and oxime groups of the bis-oximes resulted in equilibration among up to 16 distinct ionization forms, including uncharged forms capable of diffusing into the CNS and multiple zwitterionic forms optimal for reactivation reactions. Conformationally diverse zwitterions that could act as structural antidote variants significantly improved in vitro reactivation of diverse OP-human AChE conjugates. Oxime group reorientation of one of the bis-oximes, forcing it to point into the active center for reactivation, was confirmed by X-ray structural analysis. Finally, our findings provide detailed structure-activity properties of several CNS-directed, uncharged aliphatic bis-oximes holding promise for use as protonation-dependent, conformationally adaptive, “smart” accelerated antidotes against OP toxicity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

A Physics-Informed Reinforcement Learning Framework for Economic-Thermal Co-Optimization of Crypto Mining Data Centers: Preprint

The rapid expansion of cryptocurrency mining has created a new class of high-density data centers characterized by extreme thermal flux and high sensitivity to volatile economic markets. Traditional thermal management strategies, typically reliant on rule-based control, maintain static setpoints that fail to account for fluctuating electricity prices and cryptocurrency values - factors critical to mining profitability. To address this, we present a physics-informed reinforcement learning (PIRL) framework for economic-thermal co-optimization in crypto mining data centers. This framework consists of a proximal policy optimization (PPO) agent, a virtual testbed powered by high-fidelity physics-based models, and an interactive frontend dashboard. The PPO agent is trained using the virtual testbed and strict hardware safety limits. This physics-informed approach allows the agent to learn a stochastic policy that dynamically balances mining revenue against operational costs by co-optimizing HVAC cooling setpoints and IT computational hashrate. The simulation results demonstrate that the integrated framework achieved an 8.62% increase in net operational profit compared to traditional baseline strategies while strictly adhering to safety-critical temperature constraints (coolant supply temperature < 32 degrees C). This work provides a scalable template for the deployment of reinforcement learning in mission critical facilities where economic volatility and physical safety must be managed simultaneously.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)

Hybrid-RL-MPC4CLR was developed as a hybrid controller for active distribution grid critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. The RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while the MPC models grid operations incorporating the RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. The developers formulated the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The software is developed using various software packages in Python. The MPC's optimal power flow (OPF) model is implemented using the Pyomo package, the RL simulation environment is implemented using the MPC simulation with various scenarios of renewable energy and load demand profiles and power outage beginning times, based on the OpenAI Gym framework. The RL agent training is performed using the RLlib Ray package. The RL algorithm is trained offline using historical forecasts of renewable generation and load demand profiles. Simulation analysis and performance tests are conducted using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery.

Eseye, Abinet Tesfaye↗
NASA NTRS2011

PMA-PhyloChip DNA Microarray to Elucidate Viable Microbial Community Structure

Since the Viking missions in the mid-1970s, traditional culture-based methods have been used for microbial enumeration by various NASA programs. Viable microbes are of particular concern for spacecraft cleanliness, for forward contamination of extraterrestrial bodies (proliferation of microbes), and for crew health/safety (viable pathogenic microbes). However, a "true" estimation of viable microbial population and differentiation from their dead cells using the most sensitive molecular methods is a challenge, because of the stability of DNA from dead cells. The goal of this research is to evaluate a rapid and sensitive microbial detection concept that will selectively estimate viable microbes. Nucleic acid amplification approaches such as the polymerase chain reaction (PCR) have shown promise for reducing time to detection for a wide range of applications. The proposed method is based on the use of a fluorescent DNA intercalating agent, propidium monoazide (PMA), which can only penetrate the membrane of dead cells. The PMA-quenched reaction mixtures can be screened, where only the DNA from live cells will be available for subsequent PCR reaction and microarray detection, and be identified as part of the viable microbial community. An additional advantage of the proposed rapid method is that it will detect viable microbes and differentiate from dead cells in only a few hours, as opposed to less comprehensive culture-based assays, which take days to complete. This novel combination approach is called the PMA-Microarray method. DNA intercalating agents such as PMA have previously been used to selectively distinguish between viable and dead bacterial cells. Once in the cell, the dye intercalates with the DNA and, upon photolysis under visible light, produces stable DNA adducts. DNA cross-linked in this way is unavailable for PCR. Environmental samples suspected of containing a mixture of live and dead microbial cells/spores will be treated with PMA, and then incubated in the dark. Thereafter, the sample is exposed to visible light for five minutes, so that the DNA from dead cells will be cross-linked. Following this PMA treatment step, the sample is concentrated by centrifugation and washed (to remove excessive PMA) before DNA is extracted. The 16S rRNA gene fragments will be amplified by PCR to screen the total microbial community using PhyloChip DNA microarray analysis. This approach will detect only the viable microbial community since the PMA intercalated DNA from dead cells would be unavailable for PCR amplification. The total detection time including PCR reaction for low biomass samples will be a few hours. Numerous markets may use this technology. The food industry uses spore detection to validate new alternative food processing technologies, sterility, and quality. Pharmaceutical and medical equipment companies also detect spores as a marker for sterility. This system can be used for validating sterilization processes, water treatment systems, and in various public health and homeland security applications.

Venkateswaran, Kasthuri J.↗
NASA NTRS1998

Integration of the Remote Agent for the NASA Deep Space One Autonomy Experiment

This paper describes the integration of the Remote Agent (RA), a spacecraft autonomy system which is scheduled to control the Deep Space 1 spacecraft during a flight experiment in 1999. The RA is a reusable, model-based autonomy system that is quite different from software typically used to control an aerospace system. We describe the integration challenges we faced, how we addressed them, and the lessons learned. We focus on those aspects of integrating the RA that were either easier or more difficult than integrating a more traditional large software application because the RA is a model-based autonomous system. A number of characteristics of the RA made integration process easier. One example is the model-based nature of RA. Since the RA is model-based, most of its behavior is not hard coded into procedural program code. Instead, engineers specify high level models of the spacecraft's components from which the Remote Agent automatically derives correct system-wide behavior on the fly. This high level, modular, and declarative software description allowed some interfaces between RA components and between RA and the flight software to be automatically generated and tested for completeness against the Remote Agent's models. In addition, the Remote Agent's model-based diagnosis system automatically diagnoses when the RA models are not consistent with the behavior of the spacecraft. In flight, this feature is used to diagnose failures in the spacecraft hardware. During integration, it proved valuable in finding problems in the spacecraft simulator or flight software. In addition, when modifications are made to the spacecraft hardware or flight software, the RA models are easily changed because they only capture a description of the spacecraft. one does not have to maintain procedural code that implements the correct behavior for every expected situation. On the other hand, several features of the RA made it more difficult to integrate than typical flight software. For example, the definition of correct behavior is more difficult to specify for a system that is expected to reason about and flexibly react to its environment than for a traditional flight software system. Consequently, whenever a change is made to the RA it is more time consuming to determine if the resulting behavior is correct. We conclude the paper with a discussion of future work on the Remote Agent as well as recommendations to ease integration of similar autonomy projects.

Dorais, Gregory A.↗