Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Agents”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

ATTRACTOR: Toward Trustworthy and Trusted Autonomous Systems

The question of what it means and what it takes for an autonomous system to consider another autonomous system justifiably trustworthy must be addressed by all who seek to integrate intelligent machine agents into real-world operations. A satisfactory answer to this question is an essential component in accepting autonomous machine decision-making in safety-critical and time-critical environments, such as aviation. Historically, simulation platforms for test and evaluation of complex systems have proven to be effective in assessing performance and contributing to decisions on the fitness of systems to operate in current general and commercial aviation airspace. Moreover, simulations have informed the definition of safety-critical constraints. However, as machine systems progressively take on responsibilities for decision-making traditionally supplied by humans, simulations require enhancement. Mixed reality simulation that integrates real-world platforms and data or high-fidelity simulation data in a sim-to-flight paradigm provides insight into agent interaction and the rationale behind autonomous agent decision-making as well as the capacity for seamless integrated implementation, testing, and operation of systems. Strong simulation capabilities are especially important in the presence of algorithms that hold great promise in decision-making yet increase the uncertainty in the system. Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) is a subproject of NASA’s Convergent Aeronautics Solutions (CAS) Project. ATTRACTOR’s objective is to build a basis for understanding trust and trustworthiness in multi-agent autonomous teams, and thus to inform future certification of safety-critical and time-critical autonomous systems in aviation. Because the concepts of trust and trustworthiness must be addressed in a context, ATTRACTOR has chosen Search and Rescue (SAR) in dynamic and unstructured environments, with emphasis on search, as its design reference mission (DRM). During dynamic planning and execution of trajectory-based operations, autonomous agents determine their trajectories given an assigned mission or missions and call for assistance from an appropriate teammate when needed. This experience along with the attendant human-machine and machine-machine interactions, serve as a platform for developing approaches to identifying and measuring trustworthiness and increasing trust. In this paper, we give an overview of some of ATTRACTOR’s research and development activities, findings, and ongoing work.

ATTRACTOR↗
NASA NTRSDate not supplied

Zero-Trust Architecture for Autonomous Edge Computing

We are at the apex of an aviation revolution where autonomy will play a central role in enabling complex, multi-agent systems to communicate, interact, and collaborate on a myriad of applications spanning autonomous swarms to wild-fire management. Autonomy is not an absolute but rather a spectrum ranging from a system requiring significant human intervention to one requiring little to none [1]. For example, the extreme, in the case of an autonomous aircraft, is one that operates independently in the airspace interacting with all other elements (air traffic controllers, other pilots) as if it were a human pilot. Critical to this vision is an architecture that enables autonomous agents to interact with minimal latency. Edge computing is an emerging architecture where compute and storage is pushed to the ‘edge’ of the network in order to minimize the round-trip time from agent to resource thereby mitigating the latency associated with cloud-only based approaches. Additionally, services can generate massive amounts of data (e.g., video feeds), which may require analysis in near real-time. Moving this data to the cloud for further processing may not be feasible due to latency, bandwidth, and cost. Privacy, security, and reliability can also be improved by edge computing architectures. However, this geo-distributed and dynamic* architecture complicates the establishment of unambiguous network security boundaries and can lead to vulnerabilities including man in the middle attacks, replay attacks, physical security breaches of edge nodes, signal interception, etc. This motivates the need for zero-trust architectures [2–4] which de-emphasize the notion of static network perimeters and, as the name implies, do not instill any innate trust in any particular agent. It is required that all agents must be authorized and approved in every transaction. In this paper, we present a zero-trust architecture suitable for edge-computing applications that demand significant low-latency, security, privacy, and reliability.

zero trust↗
DOE OSTI2020

High-Temperature Stability of Aqueous Foams as Potential Waterless Hydrofracking Fluid for Geothermal Reservoir Stimulation

Hydraulic fracturing is the process of fracturing rock formations with high-pressure water-based fluids. In Enhanced Geothermal Systems (EGS) hydraulic fracturing is carried out by injecting high-pressure fluids into the Hot Dry Rocks (HDR) under carefully controlled conditions. The fluid used for fracturing is an important component for EGS, not only concerning the technical approch but also environmental impact. Recent research has been carried out to develop waterless fracturing technologies for EGS, including foam-based hydrofracking, where foams are mixtures of gas and liquid fluids. Foam fracturing fluids have potential benefits over water-based fluids because of less water consumption, less damage in water sensitive formations, and less liquid to recover and handle after fracturing process. One challenge for implementing foam fracturing in EGS is to achieve stable foams at high temperatures, as the foam stability tends to decay with increase in temperature.This research is focused on investigating the stability of foams at high temperatures. Preliminary results show that 1) N2 foams are more stable compared to CO2 foams, 2) foams containing only surfactants were not stable at high temperatures, and 3) addition of different stabilizing agents increased the thermal stability of all the foams. It was observed that different stabilizing agents interacted differently with anionic, nonionic, and cationic surfactants. It was further shown that the crosslinking agent with a base solution of pH 6 and graphene oxide dispersion was the most effective stabilizing agent for anionic surfactant AOS, while bentonite clay effectively enhanced thermal stability for another anionic surfactant SDS. SiO2 nanoparticles showed limited improvement in thermal stability at high temperatures compared to other stabilizing agents.

Thakore, Virensinh↗
DOE OSTI2021

Emergency Voltage Regulation in Power Systems via Ripple-Type Control: Preprint

With increasing penetration of volatile renewable generation and cyber-physical disruptions, ensuring safe operation of bulk power systems has become unprecedentedly challenging. Since communication and computational costs restrict centralized system dispatch to be called upon every few minutes; and purely local schemes are shown to be insufficient, distributed controls have been advocated for handling unanticipated system conditions in real time. However, the applicability of distributed control schemes is fundamentally limited by their need for widespread communication and model cognizance. In this con-text, we put forth a hybrid low-communication saturation-driven protocol for the coordination of control agents that are distributed over a physical system and are allowed to communicate with peers over a ‘hotline’ communication network. Under this protocol, when agents observe a constraint violation based on local measurements, they respond locally until their control resources saturate, in which case they send a beacon for assistance to peer agents. The scheme ensures that minor violations are efficiently mitigated via fast local controls, while severe violations may be handled by a collaboration of a relatively small set of agents. We evaluate the performance of this scheme via extensive numerical tests on the IEEE 14-bus test feeder, where agents act upon noisy measurements under diverse scenarios of random load variations and severe low/high voltage events.

distributed control↗
DOE OSTI2021

Ripple-Type Control for Enhancing Resilience of Networked Physical Systems: Preprint

Distributed control agents have been advocated as an effective means for improving the resiliency of our physical in-frastructures under unexpected events. While purely local control has been shown to be insufficient, centralized optimal resource allocation approaches can be slow. In this context, we put forth a hybrid low-communication saturation-driven protocol for the coordination of control agents that are distributed over a physicalsystem and are allowed to communicate with peers over a 'hot-line' communication network. According to this protocol, agents act upon on local readings unless their control resources have been depleted, in which case they send a beacon for assistance to peer agents. Our ripple-type scheme triggers communication locally only for the agents with saturated resources, and is proved to converge. Moreover, under a monotonicity assumption on the underlying physical law coupling control outputs to inputs, the devised control is proved to converge to a configuration satisfying safe operational constraints. The assumption is shown to hold for voltage control in electric power systems and pressure control in water distribution networks. Numerical tests on both networks corroborate the efficacy of the novel scheme.

distributed control↗
DOE OSTI2021

Autonomous semantic data discovery for distributed networked systems

Systems, methods, techniques and apparatuses for managing distributed applications of networked intelligent agents are disclosed. The agents are operably to autonomously discover semantic profiles and associated data of other agents in a networked system participating in a given application. The agents need not be in direct communication with or known to all the other agents in the networked system.

Brissette, Alexander↗
DOE OSTI2021

The Effect of Antagonistic Behavior in Reinforcement Learning

The significant achievements of deep reinforcement learning (RL) have motivated researchers to also investigate its shortcomings. Such work has shown that typical methods in deep RL tend to produce brittle policies that overfit to the training environment. In this paper, we introduce the notion of purely antagonistic behavior in value-based agents, where the objective is not to maximize reward but to minimize the victim’s value over time. This notion is motivated by the scenario in which an antagonistic human architect, without access to the environment’s reward function, wants to build an RL agent that can impede another well-trained RL victim agent. First, we formalize a notion of antagonistic behavior in RL. Then, we provide experiments that show how a purely antagonistic agent performs compared to a well-trained victim that learns directly from the game’s rewards. Our results suggest that if one’s goal is to find vulnerabilities in well-trained agents, direct access to the environment’s rewards is not necessary, and antagonistic behavior can be measured independently from environment wins and losses.

Fujimoto, Ted C.↗
DOE OSTI2022

Oil and water resistant low adhesion surface treatment and coatings

A novel omniphobic surface coating is disclosed that provides both high oil-repellency and high-water repellency to the coated surface. The coating may contain perfluoropolyether (PFPE) or a similar fluorinated or perfluorinated polymer, as well as a hardener and a catalyst. One or more surfactants or viscosifiers may also be added. Further, the coating may contain one or more functional additives, including, but not limited to, coloring agents, anti-corrosive agents, anti-fouling agents, water-repellent agents, and/or oil-repellent agents. Methods for formulating the novel omniphobic surface coating are also described. Such methods include preparing a first part of the coating, which may contain, at minimum, a polymer such as PFPE or a similar fluorinated or perfluorinated polymer, and a suitable solvent; preparing a second part of the coating, which is a water-free system that may contain, at minimum, a hardener and a catalyst; and combining the first part and the second part together.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Methods of recovering an elemental rare earth metal, and methods of forming a rare earth metal

A method of recovering an elemental rare earth metal comprises placing a rare earth-containing material comprising a rare earth metal in a reaction solution comprising a reducing agent and a non-aqueous solvent comprising an ionic liquid or a eutectic mixture, reducing the rare earth metal with the reducing agent to form a metallic rare earth metal and cations of the reducing agent, transferring the cations of the reducing agent from the reaction solution to an electrochemical cell through an ion exchange membrane, and reducing the cations of the reducing agent in the electrochemical cell. Related methods of forming an elemental rare earth metal, and related systems are disclosed.

Case, Mary E.↗
DOE OSTI2023

Carbon fiber-nanoparticle composites with electromechanical properties

A carbon fiber composite material comprising: (i) a carbon fiber having an outer surface, a thickness of at least 1 micron, and an aspect ratio of at least 1000; (ii) a sizing agent coated on the outer surface of the carbon fiber, wherein the sizing agent has a thickness of up to 200 nm; and (iii) nanoparticles having a size in at least one dimension of up to 100 nm embedded within the sizing agent, wherein the nanoparticles have a metal carbide, metal oxide, metal nitride, and/or metal boride composition. A method for producing the fiber composite material comprises: (a) continuously feeding and coating a continuous carbon fiber with a liquid containing a solvent, sizing agent, and nanoparticles in a continuous feed-through process to result in said sizing agent and nanoparticles coating the surface of the continuous carbon fiber; and (b) removing the solvent from the coated fiber.

Bowland, Christopher C.↗
DOE OSTI2020

Amphiphilic Block Copolymers for Flocculation and Hydrophobization of Legacy Waste Suspensions in Flotation Driven Dewatering Operations - 20162

Flotation has been effectively demonstrated as a low footprint, high efficiency separation process when utilising simple anionic surface modifying agents such as sodium dodecyl sulphate (SDS). These agents increase hydrophobicity to remove suspended cationic surface charged particulates such as Mg(OH){sub 2} from waste suspensions similar to those at British nuclear fuel management sites- where corrosion processes have degraded fuel cladding. The technology is advantageous as it could effectively be retrofitted into existing waste management facilities at nuclear waste management sites such Sellafield and Hanford as the simplicity (no moving parts) and size of this technology coupled with low cost of construction could be rapidly deployed to aid in risk and hazard reduction decommissioning operations with minimum impact to secondary waste generation. Previously, sodium dodecyl sulphate has been deployed as a collector agent to increase the hydrophobicity of Mg(OH){sub 2} suspended particles, achieving a maximum particulate extraction percentage of 93%. To achieve further particulate extraction, flotation performance has been determined to be a function of the hydrodynamic limitations extracting fine material (>20 μm). Fine particulates have been shown in the minerals industry to have an inability to overcome slipstreams created from rising bubbles in flotation cells, likely due to the low mass and thus momentum of the colloidal material, preventing interaction between the hydrophobized particles and air-water interfaces (bubbles). To combat this mass transfer resistance poly(acrylic acid)-block-poly(n-butyl acrylate) amphiphilic diblock copolymers were synthesised via reversible addition-fragmentation chain-transfer (RAFT) polymerisation. The anticipated mode of action of these polymers involves the hydrophilic poly(acrylic acid) block polymer acting as a flocculation agent to increase the mass of the aggregates, and the hydrophobic poly(n-butyl acrylate) modifies the hydrophobicity of the resulting flocs. The concentration of this polymer (termed a macro-collector) was varied and the change in particle size distribution of the particulate suspension was analysed using static light scattering. The flotation performance was then characterised by several methods including percentage particulate recovery, volume reduction factor and residual bulk particulate concentration. This was then benchmarked against the traditional surfactant collector agent SDS. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

A non-cooperative meta-modeling game for automated third-party calibrating, validating and falsifying constitutive laws with parallelized adversarial attacks

The evaluation of constitutive models, especially for high-risk and high-regret engineering applications, requires efficient and rigorous third-party calibration, validation and falsification. While there are numerous efforts to develop paradigms and standard procedures to validate models, difficulties may arise due to the sequential, manual, and often biased nature of the commonly adopted calibration and validation processes, thus slowing down data collections, hampering the progress towards discovering new physics, increasing expenses and possibly leading to misinterpretations of the credibility and application ranges of proposed models. This work attempts to introduce concepts from game theory and machine learning techniques to overcome many of these existing difficulties. Here, we introduce an automated meta-modeling game where two competing AI agents systematically generate experimental data to calibrate a given constitutive model and to explore its weakness such that the experiment design and model robustness can be improved through competitions. The two agents automatically search for the Nash equilibrium of the meta-modeling game in an adversarial reinforcement learning framework without human intervention. In particular, a protagonist agent seeks to find the more effective ways to generate data for model calibrations, while an adversary agent tries to find the most devastating test scenarios that expose the weaknesses of the constitutive model calibrated by the protagonist. By capturing all possible design options of the laboratory experiments into a single decision tree, we recast the design of experiments as a game of combinatorial moves that can be resolved through deep reinforcement learning by the two competing players. Our adversarial framework emulates idealized scientific collaborations and competitions among researchers to achieve a better understanding of the application range of the learned material laws and prevent misinterpretations caused by conventional AI-based third-party validation. Numerical examples are given to demonstrate the wide applicability of the proposed meta-modeling game with adversarial attacks on both human-crafted constitutive models and machine learning models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Nuclear microreactor transient and load-following control with deep reinforcement learning

The economic feasibility of nuclear microreactors will depend on minimizing operating costs through advancements in autonomous control, especially when these microreactors are operating alongside other types of energy systems (e.g., renewable energy). This study explores the application of deep reinforcement learning (RL) for real-time drum control in microreactors, exploring performance in regard to load-following scenarios. By leveraging a point kinetics model with thermal and xenon feedback, we first establish a baseline using a single-output RL agent, then compare it against a traditional proportional–integral–derivative (PID) controller. This study demonstrates that RL controllers, including both single- and multi-agent RL (MARL) frameworks, can achieve similar or even superior load-following performance as traditional PID control across a range of load-following scenarios. In short transients, the RL agent was able to reduce the tracking error rate in comparison to PID by one half to one third. Over extended 300-minute load-following scenarios in which xenon feedback becomes a dominant factor, PID maintained better accuracy, but RL still remained within a 1% error margin despite being trained only on short-duration scenarios. This highlights RL’s strong ability to generalize and extrapolate to longer, more complex transients, affording substantial reductions in training costs and reduced overfitting. Furthermore, when control was extended to multiple drums, MARL enabled independent drum control as well as maintained reactor symmetry constraints without sacrificing performance---an objective that standard single-agent RL could not learn. We also found that, as increasing levels of Gaussian noise were added to the power measurements, the RL controllers were able to maintain lower error rates than PID, and to do so with at least 10% and upwards of 150% less control effort. These findings illustrate RL's potential for autonomous nuclear reactor control, laying the groundwork for future integration into high-fidelity simulations and experimental validation efforts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Blockchain Enabled Intelligence of Federated Systems (BELIEFS): An attack-tolerant trustable distributed intelligence paradigm

In this article, a Blockchain Enabled Intelligence of Federated Systems (BELIEFS) is proposed to conduct cooperative control for the multi-regional large-scale power system with a multi-agents system (MAS). By establishing a two levels blockchain, each regional AI agent can simultaneously manage intra-regional controllers and cooperate with other AI agents. Under the consensus mechanism, the agents, which respectively conducted distributed deep reinforcement learning (DDRL) algorithm in multi-regions, can have the tolerant capability of malicious attacks in their training process. The demonstration of the proposed approach is within a multi-regional large-scale interconnected power system. Under the mode of "centralized dispatching and hierarchical management", this article aims to definite a mathematical model to deal with the control problem of the power systems. With the comparison experiments, the effectiveness and efficiency of our proposed method in the training process are verified. In addition, malicious attacks are set on the main chain and shard chains to verify the attack-tolerant capability. We expect that such approach and results can suggest a new paradigm of attack-tolerant trustable distributed AI deployment.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Stability study of aqueous foams under high-temperature and high-pressure conditions relevant to Enhanced Geothermal Systems (EGS)

In Enhanced Geothermal System (EGS) hydraulic fracturing is carried out by injecting cold water into deep Hot Dry Rocks (HDR) under carefully controlled conditions to create new or reopen existing fractures. Water-based fracturing fluids demonstrate some challenges including immense quantity of water usage, water sensitivity of the formations, water blocking, and lack of proppant carrying capacity and transportation. Thus, an alternative is to use foam-based fracturing fluid which offers potential advantage over conventional water-based fracturing fluid such as minimum water usage, reduced wellbore damage, high proppant carrying capacity, and less environmental damage. However, foams are complex mixture of gaseous phase and liquid phase which are thermodynamically unstable at downhole conditions, and their stability can decrease over time due to liquid drainage, bubble coarsening, and coalescence. This paper shows laboratory experiments executed to study foam stability at high temperature (200°C) and high pressure (6.9MPa) conditions which simulates the geothermal environment. Foam stability was characterized by half-life of foam, which is defined as the time taken by the foam to decreases by 50% of its original height due to drainage. In this paper, two types of gaseous phases, nitrogen (N 2 ) and carbon dioxide (CO 2 ) were investigated. Also, based on successful practice of foam-based fracturing fluid in oil and gas industries, four surfactants, including Alpha olefin sulfonate (AOS), Sodium dodecyl sulfonate (SDS), Tergitol™ (NP – 40), and Cetyltrimethylammonium chloride (CTAC) at optimum concentration of 1wt.% were tested for best stability performance. In addition, different stabilizing agents including guar gum, bentonite clay, crosslinker, silicon dioxide nanoparticles (SiO 2 ), graphene oxide (GO) were also studied. Experimental results showed that N 2 foams were more stable than CO 2 foams. It was observed that foam half-life decreased with the increase in temperature. Among all the surfactants, AOS foams showed the most promising thermal stability at high temperatures. Moreover, with the addition of stabilizing agents, foam's half-life was enhanced. Stabilizing agents such as crosslinker and GO dispersion showed the most stable foams with half-life recorded at 20min and 17min, respectively, at 200°C and 6.9MPa. Finally, pressure also showed a positive effect on foam stability; with increased pressure, foam half-life was increased. Based on the experimental data, analytical models for the effect of temperature and pressure were developed, considering foam degradation is a first-order kinetic reaction that linearly depends on the foam drainage mechanism. The effect of temperature on foam half-life was studied as an exponential decay model. In this model, foam half-life is a function of drainage rate constant (D A ) and activation energy (E a ) of the foam system. The effect of pressure on foam half-life was found to obey a power-law model where an increase in pressure showed an increase in foam half-life. Furthermore, a linear relation was studied for the effect of pressure on foam activation energy and drainage rate. Then the combined effects of temperature and pressure were studied, which yielded an analytical model to predict the foam stabilities in terms of half-life for different foam compositions. In conclusion, this research indicates that with an appropriate selection of surfactants and stabilizing agents, it is possible to obtain stable foams, which could replace conventional water fracturing fluid under EGS conditions.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Microscopic Origins of the Nonlinear Behavior of Particle-Filled Rubber Probed with Dynamic Strain XPCS

The underlying microscopic response of filler networks in reinforced rubber to dynamic strain is not well understood due to the experimental difficulty of directly measuring filler network behavior in samples undergoing dynamic strain. This difficulty can be overcome with in situ X-ray photon correlation spectroscopy (XPCS) measurements. The contrast between the silica filler and the rubber matrix for X-ray scattering allows us to isolate the filler network behavior from the overall response of the rubber. This in situ XPCS technique probes the microscopic breakdown and reforming of the filler network structure, which are responsible for the nonlinear dependence of modulus on strain, known in the rubber science community as the Payne effect. These microscopic changes in the filler network structure have consequences for the macroscopic material performance, especially for the fuel efficiency of tire tread compounds. In this work, we elucidate the behavior with in situ dynamic strain XPCS experiments on industrially relevant, vulcanized rubbers filled (13 vol %) with novel air-milled silica of ultrahigh-surface area (UHSA) (250 m 2 /g). The addition of a silane coupling agent to rubber containing this silica causes an unexpected and counterintuitive increase in the Payne effect and decrease in energy dissipation. For this rubber, we observe a nearly two-fold enhancement of the storage modulus and virtually equivalent loss tangent compared to a rubber containing a coupling agent and conventional silica. Interpretation of our in situ XPCS results simultaneously with interpretation of traditional dynamic mechanical analysis (DMA) strain sweep experiments reveals that the debonding or yielding of bridged bound rubber layers is key to understanding the behavior of rubber formulations containing the silane coupling agent and high-surface area silica. These results demonstrate that the combination of XPCS and DMA is a powerful method for unraveling the microscale filler response to strain which dictates the dynamic mechanical properties of reinforced soft matter composites. With this combination of techniques, we have elucidated the great promise of UHSA silica when used in concert with a silane coupling agent in filled rubber. Such composites simultaneously exhibit large moduli and low hysteresis under dynamic strain.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Traffic Signal Optimization by Integrating Reinforcement Learning and Digital Twins

Machine learning (ML) methods, especially reinforcement learning (RL), have been widely considered for traffic signal optimization in intelligent transportation systems. Most of these ML methods are centralized, lacking in scalability and adaptability in large traffic networks. Further, it is challenging to train such ML models due to the lack of training platforms and/or the cost of deploying and training in a real traffic networks. This paper presents an approach for the integration of decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin (DT) to optimize traffic signals for the reduction of traffic congestion and network-wide fuel consumption related to stopping. Specifically, the DGMARL agents learn traffic state patterns and make decisions regarding traffic signal control with assistance from a Digital Twin module, which simulates and replicates the traffic behaviors of a real traffic network. The proposed approach was evaluated using PTV-Vissim [1], a microscopic traffic simulation platform. PTV-Vissim is also the simulation engine of the DT, enabling emulation and optimization of the traffic signals on the MLK Smart Corridor in Chattanooga, Tennessee. Compared to an actuated signal control baseline approach, experiment results show that Eco_PI, a developed performance measure capturing the impact of stops on fuel consumption, was reduced by 44.27% in a 24-hour and an average of 29.88% in a PM-peak-hour scenario.

Multi-Agent Reinforcement Learning, Digital Twin, ↗
DOE OSTI2026

Signal Whisperers: Enhancing Wireless Reception Using DRL-Guided Reflector Arrays

This paper presents a multi-agent reinforcement learning (MARL) approach for controlling adjustable metallic reflector arrays to enhance wireless signal reception in non-line-of-sight (NLOS) scenarios. Unlike conventional reconfigurable intelligent surfaces (RIS) that require complex channel estimation, our system employs a centralized training with decentralized execution (CTDE) paradigm where individual agents corresponding to reflector segments autonomously optimize reflector element orientation in three-dimensional space using spatial intelligence based on user location information. Through extensive ray-tracing simulations with dynamic user mobility, the proposed multi-agent beam-focusing framework demonstrates substantial performance improvements over single-agent reinforcement learning baselines, while maintaining rapid adaptation to user movement within one simulation step. Comprehensive evaluation across varying user densities and reflector configurations validates system scalability and robustness. The results demonstrate the potential of learning-based approaches for adaptive wireless propagation control.

deep reinforcement learning↗