Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Risk-averse optimization for resilience enhancement of complex engineering systems under uncertainties

With the growth of complexity and extent, large scale interconnected network systems, e.g., transportation networks or infrastructure networks, become more vulnerable to external disturbances. Hence, managing potential disruptive events during the design, operating, and recovery phase of an engineered system and therefore improving the system’s resilience is an important yet challenging task. Here, to ensure system resilience after the occurrence of failure events, this study proposes a mixed-integer linear programming (MILP) based restoration framework using heterogeneous dispatchable agents. The scenario-based stochastic optimization (SO) technique is adopted to deal with the inherent uncertainties imposed on the recovery process from nature. Moreover, different from conventional SO using deterministic equivalent formulations, the CVaR risk measure is implemented for this study because of the temporal sparsity of the decision making in applications such as the recovery from extreme events. The resulting restoration framework involves a large-scale MILP problem and thus an adequate decomposition technique i.e. modified Lagrangian dual decomposition, is also employed to achieve tractable computational complexity. Case study results based on the IEEE 37-bus test feeder demonstrate the benefits of using the proposed framework for resilience improvement as well as the advantages of adopting SO formulations.

42 ENGINEERING↗
DOE OSTI2020

dGen (Distributed Generation Market Demand) Model Data: Alpha Release

Open sourced data needed to run the basic alpha release version of the dGen model. Includes a pre-generated agent file of 100,000 agents in pickle file format along with the base schema and table data in parquet format that are needed to create a postgreSQL database for the model to interact with.

14 SOLAR ENERGY↗
DOE OSTI2023

Electrolytic Coagulant Generation for Treating Flowback and Produced Water for Reuse

This research investigated a novel electrochemical process for producing a ferric iron coagulant for use in treating flowback and produced water (FPW) from hydraulic fracturing and oil production operations. The electrolytic coagulant generation (ECG) system uses an electrochemical cell to produce acid and base from oilfield brine solutions. The acid is used to dissolve scrap iron to provide a Fe 3+ coagulating agent, and the base is used to neutralize the treated water. The costs for generating the ferric iron coagulant were determined as a function of current density and feed water salinity. The process was shown to be effective for removing colloidal bentonite particles from brine solutions. Here, the process has several advantages over conventional electrocoagulation using iron anodes, including: the ability to treat anoxic waters, elimination of electrode fouling, lower cost for the coagulant, and the ability to deliver Fe 3+ doses greater than 1 mM, since it is not limited by the amount of dissolved oxygen required to oxidize ferrous to ferric iron.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Explainable physics-based constraints on reinforcement learning for accelerator optimization

We present a reinforcement learning (RL) framework for optimizing particle accelerator experiments that builds explainable physics-based constraints on agent behavior. The goal is to increase transparency and trust by letting users verify that the agent’s decision-making process incorporates suitable physics. Our algorithm uses a learnable surrogate function for physical observables, such as energy, and uses them to fine-tune how actions are chosen. This surrogate can be represented by a neural network or by an interpretable sparse dictionary model. We test our algorithm on a range of particle accelerator optimization environments designed to emulate the Continuous Electron Beam Accelerator Facility at Jefferson Lab. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment. In addition, we find that the introduction of a physics-based surrogate enables our RL algorithms to reliably converge for difficult high-dimensional accelerator optimization environments.

explainability↗
DOE OSTI2025

Model-based Hierarchical Reinforcement Learning for Improved Physical Security Design: A Prototype

Prior work in FY24 developed an adversarial AI agent aid in path analysis of physical protection systems. This agent, trained using a model-based reinforcement learning algorithm, was able to successfully learn the most vulnerable path in facilities. It was able to extend the current state of practice for physical protection design by exhibiting dynamic behavior based on current environmental conditions. Whereas PathTrace largely performs a static, graph-based analysis, the AI agent was able to make decisions based on relative position in the facility, current conditions (was the adversarial agnet discovered?), and proximity to secondary targets. The agent demonstrated some novel capabilities, but had limitations that need to be resolved before it can be used for production purposes. For example, the adversarial agent generalizes poorly and takes a relatively long time to train. Nonetheless, there is still considerable promise for developing the adversarial agent further in order to explore even richer, more dynamic behaviors (e.g., adversary motivations, environmental debris, and more). This work considers a complementary idea; development of a planning agent. The planning agent is envisioned as an auto-complete-like tool that can help accelerate security system design by human experts. The agent would respect existing barriers and sensors placed by a human expert while offering cost-effective suggestions (i.e., implicitly balancing effectiveness with cost) to improve the design. The goal is for this agent to be part of an expert’s toolbox, not to totally upend the current state-of-practice, or to displace human experts. The ultimate goal would be concurrent training of both the adversarial and planning agent together, to learn entirely through self-play. This would represent an entirely new way of performing system deign. We selected a hierarchical, model-based reinforcement learning algorithm to serve as the planning agent. This is an extension of concepts used in the prior FY24 adversarial agent work. There, we had a single agent acting an environment. Here, we have two different sub-agents (policies), working together, to form a complete agent. There is a manager policy, which can select abstract goals on slower time scales, and a worker, which performs primitive actions to reach goals selected by the manager. It is worth noting that this class of algorithm is challenging to work with. From our understanding, our work is one of the first successful uses of model-based reinforcement learning (MBRL) in nuclear energy1 , and likely the first hierarchical model-based reinforcement learning application in nuclear energy. Further, this work is one of the first known attempts to apply AI to perform a design tasks in nuclear energy. Consequently, there were significant implementation challenges and the bulk of the work was focused on successful implementation and algorithm design. The results presented here are very low technology readiness level as a consequence of the lack of related literature, but still represent a significant step forward in the pursuit of applied AI for design.

42 ENGINEERING↗
DOE OSTI2020

High-Temperature Stability of Aqueous Foams as Potential Waterless Hydrofracking Fluid for Geothermal Reservoir Stimulation

Hydraulic fracturing is the process of fracturing rock formations with high-pressure water-based fluids. In Enhanced Geothermal Systems (EGS) hydraulic fracturing is carried out by injecting high-pressure fluids into the Hot Dry Rocks (HDR) under carefully controlled conditions. The fluid used for fracturing is an important component for EGS, not only concerning the technical approch but also environmental impact. Recent research has been carried out to develop waterless fracturing technologies for EGS, including foam-based hydrofracking, where foams are mixtures of gas and liquid fluids. Foam fracturing fluids have potential benefits over water-based fluids because of less water consumption, less damage in water sensitive formations, and less liquid to recover and handle after fracturing process. One challenge for implementing foam fracturing in EGS is to achieve stable foams at high temperatures, as the foam stability tends to decay with increase in temperature.This research is focused on investigating the stability of foams at high temperatures. Preliminary results show that 1) N2 foams are more stable compared to CO2 foams, 2) foams containing only surfactants were not stable at high temperatures, and 3) addition of different stabilizing agents increased the thermal stability of all the foams. It was observed that different stabilizing agents interacted differently with anionic, nonionic, and cationic surfactants. It was further shown that the crosslinking agent with a base solution of pH 6 and graphene oxide dispersion was the most effective stabilizing agent for anionic surfactant AOS, while bentonite clay effectively enhanced thermal stability for another anionic surfactant SDS. SiO2 nanoparticles showed limited improvement in thermal stability at high temperatures compared to other stabilizing agents.

Thakore, Virensinh↗
DOE OSTI2022

Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters

Over the last few decades, the deployment of distributed solar photovoltaic (PV) systems has increased consistently. High PV penetration could cause adverse effects on the grid, such as voltage violations. This paper proposes a new distributed soft actor-critic based multi-agent deep reinforcement learning (SAC-MADRL) control solution to minimize the PV real power curtailment while keeping the grid voltage in an acceptable range. New reward functions have been designed to coordinate different agents during the learning process, yielding improved convergence. Comparison results with other control methods on a real feeder in western Colorado U.S. with 80% penetration of PVs demonstrate that the proposed method has better capability of effectively regulating voltage while minimizing the PV real power curtailment.

distribution system↗
DOE OSTI2023

A Physics-Based Data-Driven Approach for Modeling of Environmental Degradation in Elastomers

Abstract Elastomers are now commonly used in a number of industries, including aerospace, structure, transportation, shipbuilding, and automotive, due to their excellent workability, formability, and flexibility. During their activity, elastomers are subjected to harsh environmental conditions, which decreases their resilience. False predictions made early in their lives can have major financial and environmental implications. Elastomers’ performance and properties, such as strength, durability, and density, are influenced by chemical changes in these materials, known as degradation, which occurs over time. This process can alter the morphology of a polymer matrix as well as cause chain scission and cross-linking, resulting in different behaviors than that of the unaged material. To demonstrate the effect of thermaloxidative aging on the mechanical behavior of elastomers, several experimental and theoretical models have been proposed. In view of the large volume of experimental data available on micro-structural evolution in the course of aging, we propose a physics-based data-driven approach to overcome the shortcomings of both phenomenological and micro-mechanical models. This work presents a novel thermodynamically consistent, multiagent machine-learned model for predicting the constitutive behavior of cross-linked elastomers during environmental aging, such as thermo-oxidative and hydrolytic aging for various states of deformation. Single mechanism degradation changes the polymer matrix over time where it is causing chain scission, reduction of cross-links, and morphology change. To capture the idealized Mullins effect and permanent set due to the effect of single aging mechanisms on nonlinear mechanical responses of elastomers, we propose a data-driven model for simulating inelastic elements in a polymer matrix. By using a sequential order reduction, we were able to reduce the 3D stress-strain tensor mapping problem to a small number of super-constrained 1D mapping problems. To systematically classify such mapping problems into a few categories, an assembly of multiple replicated conditional neural network learning agents (L-agents) is used based on our recent work. Each category is represented by a different type of agent. The effect of deformation history, aging time, and aging temperature is captured by this model. The model is validated using a broad collection of data, ranging from our experimental results to data from the literature. In addition, thermodynamic consistency and frame independence are investigated. The most significant achievements of this model are its precision, simplicity, and prediction of inelasticity under various states of deformation. The model’s accuracy and simplicity make it a good option for commercial and industrial applications. Conveniently, due to the model modular nature, it can be expanded in the future to include viscoelasticity and non-isotropic formation for better precision.

Ghaderi, Aref↗
DOE OSTI2025

Reinforcement Learning-Based Oscillation Dampening: Scaling Up Single-Agent Reinforcement Learning Algorithms to a 100-Autonomous-Vehicle Highway Field Operational Test

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. Here, we explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

"Reinforcement Learning Based Oscillation Dampening: Scaling up Single-Agent RL algorithms to a 100 AV highway field operational test"

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. We explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

Jang, Kathy↗
DOE OSTI2021

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several subnetworks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each subnetwork is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33-and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Active distribution grids can experience voltage fluctuations and violations due to the high penetration of variable distributed energy resources (DERs). These problems might occur because of the uncertain and variable generation natures of these resources, especially solar photovoltaic resources, during panel shadowing scenarios. Volt-VAR control (VVC) is an efficient method that controls the reactive power set-points of the inverters to regulate the voltage of distribution grids. Although several VVC approaches have been proposed recently, the performance of these approaches degrades significantly if behind-the-meter solar generation data are unobservable/missing. Therefore, it is necessary to impute missing/unobservable PV data accurately to be utilized in VVC approaches. Further, this paper proposes a model-free, data-driven, centrally trained, and decentrally executed multi-agent deep reinforcement learning-based VVC architecture to regulate the voltage of distribution networks. A generative adversarial network (GAN) is incorporated to impute the unobservable PV data accurately, which improves the performance of the proposed control architecture. The proposed multi-agent-soft-actor–critic algorithm (MASAC)-based VVC technique utilizes the actual PV dataset as well as the imputed dataset from the GAN framework to learn the optimal coordinated control policy for controlling the optimal reactive power set-points of PV inverters. The effectiveness of the proposed approach is analyzed on a modified IEEE 34-bus test case with added PV inverters. The results are compared and analyzed with a base case model with no VVC and VVC with a local droop control approach, genetic algorithm optimization, and a centralized soft actor–critic-based approach. Moreover, the performance of the proposed approach is compared with that of a multi-agent VVC framework without using the PV generation data and load information as the system state. The results illustrate that the proposed method with more state input improves the voltage profile and reduces the power loss of the network across various loading and PV generation scenarios.

14 SOLAR ENERGY↗
DOE OSTI2020

Homopiperazine-based catalysts for neutralization of organophosphorus-based compounds

Novel compositions of matter based on homopiperazine precursor materials and forming a homopiperazine-based ligand are disclosed, along with suitable techniques and materials for the synthesis and utilization thereof. In particular various synthetic schemes and techniques for applying the disclosed compositions of matter as a decontaminating agent. The decontaminating agents include homopiperazine-based ligand-metal complexes that are particularly effective at neutralizing toxicity of nerve agents, pesticides, and other toxic organophosphorus-based compounds. In preferred approaches, the homopiperazine-based ligand-metal complexes act as catalysts to facilitate substitution of a leaving group of the organophosphorus-based compound with a functional group that does not permit the organophosphorus-based compound to inactivate acetylcholinesterase upon introduction of the organophosphorus-based compound to a living organism such as insects and mammals. Advantageously, the catalytic homopiperazine-based ligand-metal complexes are formed using inexpensive, readily-available precursor materials, and may be utilized to neutralize toxins without relying on damaging caustic reactants or environmentally unfriendly organic solvents.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

A Reactive Inkjet Printing Process for Fabricating Biodegradable Conductive Zinc Structures

A novel reactive inkjet printing (RIJ) process for fabricating environmentally benign, conductive zinc structures on biodegradable substrates is reported. A precursor consisting of oxidized zinc microparticles is formulated into a viscous ink for high‐resolution, high‐repeatability screen printing. A single RIJ process is performed to selectively dispense an acid‐based oxide etching agent onto the screen‐printed zinc ink to significantly improve its electrical conductivity, up to 4 × 10 4 S m −1 . The RIJ process enables precise control of the amount of deposited etching agent, allowing control of conductivity and avoiding damage caused by excessive acid exposure. The RIJ process deposits the etching agent selectively on the screen‐printed, water‐soluble zinc patterns, thus preserving the print fidelity by avoiding ink smearing that can occur when using a blanket treatment such as submersion. A printed biodegradable antenna with a minimum feature size of 500 μm for use with a batteryless radio frequency identification (RFID) chip is fabricated as demonstration. Due to the higher ink conductivity and improved pattern fidelity, the antenna fabricated by the RIJ process shows a ≈2.2× increase in reading distance compared with a similar antenna that is screen printed and activated using a drop‐casting process.

Sui, Yongkun↗
DOE OSTI2021

Microstructural refinement of cement paste internally cured by polyacrylamide composite hydrogel particles containing silica fume and nanosilica

Supplementary cementitious materials were incorporated into hydrogel-based internal curing agents to improve the hydration, microstructure, and ultimately strength of internally cured high-performance cement paste. Polyacrylamide composite hydrogel particles containing amorphous silica – either silica fume or nanosilica – and two different polymer network crosslink densities were synthesized and incorporated into cement paste. The presence of silica and low crosslink density increased the absorption capacity of the particles in pore solution. Micrographs of internally cured paste indicated a significant improvement in hydrogel-related void-filling ability and an increase in void size for low crosslink density particles containing silica. Compressive strength and electrical resistivity increased at later ages for paste samples containing particles with higher silica dosage. The relationship between extent of hydration, void size, and void-filling activity was found to strongly influence the paste's long-term strength and is thus an important structure-property relationship to consider when selecting hydrogels for internal curing purposes.

36 MATERIALS SCIENCE↗
DOE OSTI2021

New insights into fuel blending effects: Intermolecular chemical kinetic interactions affecting autoignition times and intermediate-temperature heat release

Fuel blending effects on chemically-dominated fuel properties, such as gasoline anti-knock quality, are influenced by fundamental chemical kinetic interactions between the blending agent and the base fuel. Historically, quantification of such interactions has focused on changes to the radical pool, including $\dot{O}$H and HO 2 , while intermolecular interactions pertaining to carbonated, non-fuel-specific intermediates are typically overlooked. In this regard, this work aims to derive new insight into intermolecular chemical kinetic interactions that are intrinsic to fuel blending effects via a case study on blends of 0–30% ethanol (by volume) into FGF-LLNL (a multi-component gasoline surrogate for FACE-F research gasoline) using a rapid compression machine at a diluted/stoichiometric fuel loading, compressed pressure of 40 bar and low- to intermediate-temperature regimes that are representative of boosted SI engine operation. Ethanol blending effects on the intermediate temperature heat release (ITHR) of FGF-LLNL are characterized using experimental measurements, where ethanol is found to promote the extent of ITHR and suppress the transition from ITHR to main ignition. Chemical kinetic modeling is undertaken using a recently updated gasoline surrogate model. Sensitivity analyses on ITHR characteristics further corroborate the ethanol blending effects, and highlight the significant dependence of ITHR on both fuel-specific and non-fuel-specific reactions. An approach allowing comprehensive characterization of the complex intermolecular chemical kinetic interactions between constitutes in a fuel blend is then proposed. Application of the approach to FGF-LLNL/E0–E30 reveals that ethanol perturbs the heat release and autoignition characteristics of FGF-LLNL not only by directly changing the $\dot{O}$H and HO 2 radical pools via fuel-specific reactions, but also through intermolecular interactions where participating intermediates can be produced and consumed by various sub-chemistries. Disabling the intermolecular interactions in carbonated species between ethanol and FGF-LLNL sub-chemistries leads to somewhat slower ITHR evolution and lower ignition reactivity. The role of the individual intermolecular interaction is also characterized using the proposed approach. Lastly, implications of intermolecular interactions for future studies that aim to improve model performance are highlighted, where it is found that further investigations on the core C0-C4 chemistry are needed for developing highly accurate chemistry models for complex fuel blends.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗