Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Discovering mechanisms for materials microstructure optimization via reinforcement learning of a generative model

Abstract The design of materials structure for optimizing functional properties and potentially, the discovery of novel behaviors is a keystone problem in materials science. In many cases microstructural models underpinning materials functionality are available and well understood. However, optimization of average properties via microstructural engineering often leads to combinatorically intractable problems. Here, we explore the use of the reinforcement learning (RL) for microstructure optimization targeting the discovery of the physical mechanisms behind enhanced functionalities. We illustrate that RL can provide insights into the mechanisms driving properties of interest in a 2D discrete Landau ferroelectrics simulator. Intriguingly, we find that non-trivial phenomena emerge if the rewards are assigned to favor physically impossible tasks, which we illustrate through rewarding RL agents to rotate polarization vectors to energetically unfavorable positions. We further find that strategies to induce polarization curl can be non-intuitive, based on analysis of learned agent policies. This study suggests that RL is a promising machine learning method for material design optimization tasks, and for better understanding the dynamics of microstructural simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Emergency Voltage Regulation in Power Systems via Ripple-Type Control

With increasing penetrations of volatile renewable generation and cyber-physical disruptions, ensuring the safe operation of bulk power systems has become unprecedentedly challenging. Because communication and computational costs restrict centralized system dispatch to being called upon every few minutes, and because purely local schemes are shown to be insufficient, distributed controls have been advocated for handling unanticipated system conditions in real time. The applicability of distributed control schemes, however, is fundamentally limited by their need for widespread communication and model cognizance. In this context, we put forth a hybrid, low-communication, saturation-driven protocol for the coordination of control agents that are distributed over a physical system and are allowed to communicate with peers over a "hotline" communication network. Under this protocol, when agents observe a constraint violation based on local measurements, they respond locally until their control resources saturate, in which case they send a beacon for assistance to peer agents. The scheme ensures that minor violations are efficiently mitigated via fast local controls, whereas severe violations can be handled by collaboration among a relatively small set of agents. We evaluate the performance of this scheme via numerical tests on the IEEE 14-bus test feeder, where agents act upon noisy measurements under diverse scenarios of load variations and severe low-/high-voltage events.

collaboration↗
DOE OSTI2021

Emergency Voltage Regulation in Power Systems via Ripple-Type Control: Preprint

With increasing penetration of volatile renewable generation and cyber-physical disruptions, ensuring safe operation of bulk power systems has become unprecedentedly challenging. Since communication and computational costs restrict centralized system dispatch to be called upon every few minutes; and purely local schemes are shown to be insufficient, distributed controls have been advocated for handling unanticipated system conditions in real time. However, the applicability of distributed control schemes is fundamentally limited by their need for widespread communication and model cognizance. In this con-text, we put forth a hybrid low-communication saturation-driven protocol for the coordination of control agents that are distributed over a physical system and are allowed to communicate with peers over a ‘hotline’ communication network. Under this protocol, when agents observe a constraint violation based on local measurements, they respond locally until their control resources saturate, in which case they send a beacon for assistance to peer agents. The scheme ensures that minor violations are efficiently mitigated via fast local controls, while severe violations may be handled by a collaboration of a relatively small set of agents. We evaluate the performance of this scheme via extensive numerical tests on the IEEE 14-bus test feeder, where agents act upon noisy measurements under diverse scenarios of random load variations and severe low/high voltage events.

distributed control↗
DOE OSTI2022

Antioxidant-induced transformations of a metal-acid hydrocracking catalyst in the deconstruction of polyethylene waste

Plastics are ideal for numerous applications, as driven by the development of complex formulations containing various additives to improve performance and processability. Unfortunately, chemical valorization strategies (e.g., catalytic deconstruction) often can be challenged by the presence of small-molecule additives, and quantification of the impact of these molecular constituents on upcycling processes remains elusive. Here, this dearth of information restricts catalyst design efforts to combine the robustness and performance necessary to improve plastics circularity. In this communication, we describe a systematic study of common plastics additives—phenolic antioxidants, and we quantify the relationship between additive content and deconstruction yields of high-density polyethylene (HDPE) over a platinum on tungstated zirconium (Pt/WO 3 /ZrO 2 ) hydrocracking catalyst. In the simplest case of a base (antioxidant- and slip agent-containing) HDPE resin versus a pure (additive-stripped) HDPE polymer, a two-fold decrease in the yield of gas and liquid products is noted for base HDPE resin. Furthermore, both antioxidant chemistry and concentration strongly impact conversion and individual product selectivities. Using infrared spectroscopy, we determine that antioxidants change the effective ratio of metal to acid sites (i.e., metal-acid balance) through reactions of phenols and/or other functional groups (e.g., acids, esters) with catalyst active sites. Overall, this work demonstrates the impact of one common set of additives on plastics deconstruction, and the analysis herein provides a blueprint for quantitatively assessing the effects of additives on plastics deconstruction processes and for evaluating the development of more robust catalytic strategies or more tolerable additive formulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Sorption agent, method of making a sorption agent and barrier system

An anionic sorption agent, method for forming the anionic sorption agent and a barrier system are disclosed. The anionic sorption agent including a modified pseudo or glycol-boehmite base comprising a structure having cationic metal ion sites. The method for forming the anionic sorption agent includes providing a pseudo or glycol-boehmite base and contacting the pseudo or glycol-boehmite base a modifying composition comprising a metallic ion to form the modified pseudo or glycol-boehmite base comprising a structure having cationic metal ion sites. The barrier system includes the anionic sorption agent comprising a first barrier component comprising a modified pseudo or glycol-boehmite base comprising a structure having cationic metal ion sites and a second barrier component comprising a cationic sorption agent.

Matteo, Edward N.↗
DOE OSTI2023

Two-Stage Deep Reinforcement Learning for Distribution System Voltage Regulation and Peak Load Management

The growing integration of distributed solar photovoltaic (PV) in distribution systems could result in adverse effects during grid operation. This paper develops a two-agent soft actor critic-based deep reinforcement learning (SAC-DRL) solution to simultaneously control PV inverters and battery energy storage systems for voltage regulation and peak demand reduction. The novel two-stage framework, featured with two different control agents, is applied for daytime and nighttime operations to enhance control performance. Comparison results with other control methods on a real feeder in Western Colorado demonstrate that the proposed method can provide advanced voltage regulation with modest active power curtailment and reduce peak load demand from feeder's head.

deep reinforcement learning↗
DOE OSTI2023

Distributed Power Allocation for 6-GHz Unlicensed Spectrum Sharing via Multi-agent Deep Reinforcement Learning

We consider the problem of power allocation over the 6 GHz Unlicensed National Information Infrastructure (UNII)- 5 spectrum. We propose a novel deep Reinforcement Learning (DRL)-based distributed power allocation scheme which utilizes the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. In particular, we model the base stations (BSs) as DRL agents that simultaneously determine the transmit powers to their scheduled user equipment (UE) in a synchronized manner. The power decision of each BS is based on its own observation of the radio environment, which consists of several local interference measurements and a limited amount of information obtained from other BSs. One advantage of the proposed scheme is that it addresses the single-agent non-stationarity problem of RL in the multi-agent scenario by incorporating the actions and observations of other BSs into each BS’s own critic which helps it to gain a more accurate perception of the overall radio environment. A centralized-training-distributed execution framework is used to train the policies where the critics are trained over the joint actions and observations of all BSs while the actor of each BS only takes the local observation as input in order to produce the transmit power. Simulation shows that the proposed power allocation scheme can achieve better throughput performance than several state-of-the-art approaches.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Soft Actor Critic Based Volt-VAR Co-optimization in Active Distribution Grids

Modern distribution networks are undergoing several technical challenges, such as voltage fluctuations, because of high penetration of distributed energy resources (DERs). This paper proposes a deep reinforcement learning (DRL)-based Volt VAR co-optimization technique for reducing voltage fluctuations as well as power loss under high penetration of DERs. In addition, the proposed approach minimizes the operational cost of the grid. A stochastic policy optimization based soft actor critic (SAC) agent is proposed to configure the optimal set-points of the reactive power outputs of the inverters. The performance of the proposed model is verified on the modified IEEE 34- and 123-bus systems and compared with a base case scenario with no reactive supply by inverters, and a local droop control approach. The results demonstrate that the proposed framework outperforms the conventional droop control method in improving the voltage profile, minimizing the network power loss, and reducing grid operational cost.

—Distribution grids, deep reinforcement learning, ↗
DOE OSTI2022

A P-61 Black Widow Inspired Palladium Biladiene Complex for Efficient Sensitization of Singlet Oxygen Using Visible Light

Photodynamic therapy (PDT) is a promising treatment option that ablates cancerous cells and tumors via photoinduced sensitization of singlet oxygen. Over the last few decades, much work has been devoted to the development of new photochemotherapeutic agents for PDT. A wide variety of macrocyclic tetrapyrrole based photosensitizers have been designed, synthesized and characterized as PDT agents. Many of these complexes have a variety of issues that pose a barrier to their use in humans, including biocompatibility, inherent toxicity, and synthetic hurdles. We have developed a non-traditional, non-cyclic, and non-aromatic tetrapyrrole ligand scaffold, called the biladiene (DMBil1), as an alternative to these traditional photosensitizer complexes. Upon insertion of a heavy atom such as Pd 2+ center, Pd[DMBil1] generates singlet oxygen in substantial yields (Φ Δ = 0.54, λ exc = 500 nm) when irradiated with visible light. To extend the absorption profile for Pd[DMBil1] deeper into the phototherapeutic window, the tetrapyrrole was conjugated with alkynyl phenyl groups at the 2- and 18-positions (Pd[DMBil2-PE]) resulting in a significant redshift while also increasing singlet oxygen generation (Φ Δ = 0.59, 600 nm). To further modify the dialkynyl-biladiene scaffold, we conjugated a 1,8-diethynylanthracene with to the Pd[DMBil1] tetrapyrrole in order to further extend the compound’s π-conjugation in a cyclic loop that spans the entire tetrapyrrole unit. This new compound (Pd[DMBil2-P61]) is structurally reminiscent of the P61 Black Widow aircraft and absorbs light into the phototherapeutic window (600–900 nm). In addition to detailing the solid-state structure and steady-state spectroscopic properties for this new biladiene, photochemical sensitization studies demonstrated that Pd[DMBil2-P61] can sensitize the formation of 1 O 2 with quantum yields of Φ Δ = 0.84 upon irradiation with light λ = 600 nm. These results distinguish the Pd[DMBil2-P61] platform as the most efficient biladiene-based singlet oxygen photosensitizer developed to date. When taken together, the improved absorption in the phototherapeutic window and high singlet oxygen sensitization efficiency of Pd[DMBil2-P61] mark this compound as a promising candidate for future study as an agent of photodynamic cancer therapy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2000

A Cybernetic Approach to the Modeling of Agent Communities

In an earlier paper [1] examples of agent technology in a NASA context were presented. Both groundbased and space-based applications were addressed. This paper continues the discussion of one aspect of the Goddard Space Flight Center's continuing efforts to develop a community of agents that can support both ground-based and space-based systems autonomy. The paper focuses on an approach to agent-community modeling based on the theory of viable systems developed by Stafford Beer. It gives the status of an initial attempt to capture some of the agent-community behaviors in a viable system context. This paper is expository in nature and focuses on a discussion of the modeling of some of the underlying concepts and infrastructure that will serve as the basis of more detailed investigative work into the behavior of agent communities. The paper is organized as follows. First, a general introduction to agent community requirements is presented. Secondly, a brief introduction to the cybernetic concept of a viable system is given. This concept forms the foundation of the modeling approach. Then the concept of an agent community is modeled in the cybernetic context.

Truszkowski, Walt↗
NASA NTRS2004

An Architecture for Controlling Multiple Robots

The Control Architecture for Multirobot Outpost (CAMPOUT) is a distributed-control architecture for coordinating the activities of multiple robots. In the CAMPOUT, multiple-agent activities and sensor-based controls are derived as group compositions and involve coordination of more basic controllers denoted, for present purposes, as behaviors. The CAMPOUT provides basic mechanistic concepts for representation and execution of distributed group activities. One considers a network of nodes that comprise behaviors (self-contained controllers) augmented with hyper-links, which are used to exchange information between the nodes to achieve coordinated activities. Group behavior is guided by a scripted plan, which encodes a conditional sequence of single-agent activities. Thus, higher-level functionality is composed by coordination of more basic behaviors under the downward task decomposition of a multi-agent planner

Aghazarian, Hrand↗
DOE OSTI2023

Data exchange and processing synchronization in distributed systems

Systems, methods, techniques and apparatuses of asynchronous communication is distributed systems are disclosed. One exemplary embodiment is a method determining, with a plurality of agent nodes structured to communicate asynchronously in a distributed system, a first set of iterations including an iteration determined by each of the plurality of agent nodes; determining, with a first agent node of the plurality of agent nodes, a local vector clock; receiving, with the first agent node, a first iteration of the first set of iterations and a remote vector clock determined based on the first iteration; updating, with the first agent node, the local vector clock based on the received remote vector clock; and determining a first iteration of a second set of iterations based on the first set of iterations after determining all iterations of the first set of iterations have been received based on the local vector clock.

Cintuglu, Mehmet H.↗
DOE OSTI2023

Reinforcement Learning for Load-balanced Parallel Particle Tracing

We explore an online reinforcement learning (RL) paradigm to dynamically optimize parallel particle tracing performance in distributed-memory systems. Our method combines three novel components: (1) a work donation algorithm, (2) a high-order workload estimation model, and (3) a communication cost model. First, we design an RL-based work donation algorithm. Our algorithm monitors workloads of processes and creates RL agents to donate data blocks and particles from high-workload processes to low-workload processes to minimize program execution time. The agents learn the donation strategy on the fly based on reward and cost functions designed to consider processes' workload changes and data transfer costs of donation actions. Second, we propose a workload estimation model, helping RL agents estimate the workload distribution of processes in future computations. Third, we design a communication cost model that considers both block and particle data exchange costs, helping RL agents make effective decisions with minimized communication costs. We demonstrate that our algorithm adapts to different flow behaviors in large-scale fluid dynamics, ocean, and weather simulation data. Our algorithm improves parallel particle tracing performance in terms of parallel efficiency, load balance, and costs of I/O and communication for evaluations with up to 16,384 processors.

Distributed and parallel particle tracing↗
NASA NTRS1990

Constraint-based evaluation of sequential procedures

Constraining the operation of an agent requires knowledge of the restrictions to physical and temporal capabilities of that agent, as well as an inherent understanding of the desires being processed by that agent. Usually a set of constraints are available that must be adhered to in order to foster safe operations. In the worst case, violation of a constraint may be cause to terminate operation. If the agent is carrying out a plan, then a method for predicting the agent's desires, and therefore possible constraint violations, is required. The conceptualization of constraint-based reasoning used herein assumes that a system knows how to select a constraint for application as well as how to apply that constraint once it is selected. The application of constraint-based reasoning for evaluating certain kinds of plans known as sequential procedures is discussed. By decomposing these plans, it is possible to apply context dependent constraints in production system fashion without incorporating knowledge of the original planning process.

Barry, Matthew R.↗
NASA NTRS2011

Microencapsulation of Self Healing Agents for Corrosion Control Coatings

Corrosion, the environmentally induced degradation of materials, is a very costly problem that has a major impact on the global economy. Results from a 2-year breakthrough study released in 2002 by the U.S. Federal Highway Administration (FHWA) showed that the total annual estimated direct cost associated with metallic corrosion in nearly every U.S. industry sector was a staggering $276 billion, approximately 3.1% of the nation's Gross Domestic Product (GOP). Corrosion protective coatings are widely used to protect metallic structures from the detrimental effects of corrosion but their effectiveness can be seriously compromised by mechanical damage, such as a scratch, that exposes the metallic substrate. The incorporation of a self healing mechanism into a corrosion control coating would have the potential to significantly increase its effectiveness and useful lifetime. This paper describes work performed to incorporate a number of microcapsule-based self healing systems into corrosion control coatings. The work includes the preparation and evaluation of self-healing systems based on curable epoxy, acrylate, and siloxane resins, as well as, microencapsulated systems based on passive, solvent born, healing agent delivery. The synthesis and optimization of microcapsule-based self healing systems for thin coating (less than 100 micron) will be presented.

Jolley, S. T.↗
NASA NTRS2012

Implementing Artificial Intelligence Behaviors in a Virtual World

In this paper, we will present a look at the current state of the art in human-computer interface technologies, including intelligent interactive agents, natural speech interaction and gestural based interfaces. We describe our use of these technologies to implement a cost effective, immersive experience on a public region in Second Life. We provision our Artificial Agents as a German Shepherd Dog avatar with an external rules engine controlling the behavior and movement. To interact with the avatar, we implemented a natural language and gesture system allowing the human avatars to use speech and physical gestures rather than interacting via a keyboard and mouse. The result is a system that allows multiple humans to interact naturally with AI avatars by playing games such as fetch with a flying disk and even practicing obedience exercises using voice and gesture, a natural seeming day in the park.

Krisler, Brian↗
NASA NTRS2004

Comparison of Carbon Dioxide and Helium as Fire Extinguishing Agents for Spacecraft

The effects of radiation heat transfer in microgravity compared to convection heat transfer in earth gravity for opposed-flow (downward) over thermally-thick fuel using low density foam fuel were investigated. Microgravity experiments on flame spread over thermally-thick fuels were conducted using foam fuels to obtain low density and thermal conductivity, and thus large flame spread rate compared to dense fuels such as PMMA. And thereby valid microgravity results were obtained even in 2.2 second drop-tower experiments not to mention for the longer duration tests in Zero Gravity Facility. Contrast to the conventional understanding, it was found that steady flame spread can occur over thick fuels in quiescent microgravity environments, especially when radiatively-active diluent gases such as CO2 were employed. This is proposed to result from radiative heat transfer from the flame to the fuel surface, which could lead to steady spread even when the amount of the heat transfer via conduction from the flame to the fuel bed is negligible. Radiative effects are more significant at microgravity conditions because the flame is thicker and thus the volume of radiating combustion products is larger as well. These results suggested that helium may be a better inert or extinguishment agent on both a mass and a mole bases at microgravity even though CO2 is much better on a mole bases at earth gravity, and these are relevant to studies of fire safety in manned spacecraft, particularly the International Space Station that uses CO2 fire extinguishers. CO2 may not be as effective as an extinguishing agent at g as it is at earth gravity in some conditions because of the differences in spread mechanisms between the two cases. In particular, the difference between conduction-dominated heat transport to the fuel bed at earth gravity and radiation-dominated heat transport at g indicates that radiatively-inert diluent such as helium could be preferable in g applications. Helium may be a superior fire suppression agent at g on several bases. First, helium is more effective than CO2 on a mole basis (thus pressure times storage volume basis) at g, meaning that the size and weight of storage bottles would be smaller for the same fire-fighting capability. Second; helium is much more effective on a mass basis (by about 11 times) at g. Third; helium has no physiological activity, unlike CO2 that affects human respiration. Fourth, as compared to N2 or CO2, is not very soluble in water and thus has fewer tendencies to cause bloodstream bubble formation following rapid spacecraft cabin depressurization.

Gokoglu, Suleyman↗