Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Optimal control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Extending the operational boundaries of RMP-ELM suppression with optimized 3D field control

The use of 3D magnetic fields is one of the promising approaches to control edge localized modes (ELMs), and ITER has plans to utilize a flexible 3D coil set for ELM suppression using 3D fields. This study focuses on optimizing the 3D field spectrum to expand the operational window for n = 1 resonant magnetic perturbation (RMP) ELM suppression in KSTAR. The optimized n = 1 RMP effectively suppresses ELMs throughout the entire H-mode discharge, including the first ELM crash, while avoiding the onset of disruptive locked modes in low-density L-mode plasmas. The predicted suppression window aligns well with experimental data, highlighting the challenges and solutions of using n = 1 RMP at low densities. Moreover, the optimization successfully achieved n = 1 RMP ELM suppression for the first time in ITER-relevant q 95 and shaping conditions, including cases with q95 as low as 3.6, as well as other q 95 and shape configurations. This highlights the importance and utility of 3D coil optimization while emphasizing the potential of long-wavelength low-n RMP, which will be valuable for ex-vessel coils designed to avoid complications of nuclear degradation.

3D magnetic field control↗

Ice storage model-predictive control in an office building with PV: scenario, error and sensitivity analysis

Thermal energy storage (TES) can enable more building-sited renewable electricity generation and lower utility bill costs for buildings owners and occupants, especially when there are high demand and variable time-of-use (TOU) charges. A model predictive control (MPC) strategy can offer additional savings over a schedule-based control with added complexity and reliance on forecasts. Here, this study examines savings for medium office buildings with chiller plants in three locations with building-installed solar photovoltaics (PV) to understand the impact of MPC. Control setpoints are fixed by a schedule-based control or optimized by nonlinear MPC. These control setpoints are actuated within EnergyPlus building models to simulate the utility cost of the chiller plant. NLP solutions can be unstable or unrealistic, but our results show that by regularizing the NLP, the solutions can be reasonably followed by the building model. MPC models make simplifications that lead to errors once the controller is participating in and changing the operation of the building. These errors average 9 % across the cases, showing that the most important parts of the system are represented. The no-thermal load costs are computed to show that the optimization can in some cases achieve both the minimum TOU and minimum monthly demand costs by demand management while reducing TOU energy costs by energy arbitrage. The MPC saves 35–66 % in the annual chiller plant operating costs, which is an additional savings above the schedule by 1–33 %. PV and TES are complementary and mostly independent, but a load with PV often results in better performance for the schedule. Our case study and sensitivity analysis show the importance of modeling and optimization for complex rates, but also the circumstances wherein a simpler strategy achieves the same performance with less potential for error.

14 SOLAR ENERGY↗

Endogenous Interface Pricing for Consistent Transmission–Distribution Co-Optimization With Discrete Distribution Controls

This paper proposes an endogenous interface pricing model for day-ahead transmission–distribution co-optimization that co-determines the interface locational marginal price (LMP) and the transmission–distribution exchange, ensuring price–dispatch consistency while optimally scheduling discrete distribution controls. The formulation couples a DC optimal power flow (OPF) with a branch-flow AC OPF that schedules distributed energy resources (DERs), tap-changer settings, capacitor banks (CBs), and multi-period energy storage systems (ESSs) under feeder voltage and current limits, and is solved as a mixed-integer second-order cone program (MISOCP). In a T14–D33 system, coordinated device scheduling recovers about 90% of the distribution-to-transmission export achievable in a reference case that ignores distribution network (DN) limits, while satisfying a 1.05 p.u. voltage upper bound. In a T39–D34/D37/D123 system, a sequential decoupled benchmark produces interface LMP distortions up to 12.5% and a 7.28% mismatch in net export energy, whereas the proposed model removes these distortions and the associated settlement mismatches. Second-order cone (SOC) relaxation gaps remain below $10^{-3}$ in all cases.

Noh, Seung-Gil↗

Analysis of Control Behavior in Eco-Driving Speed Optimization Using Pontryagin’s Minimum Principle

The energy efficiency of autonomous vehicles can be improved by selecting an optimized speed profile. Energy savings can be maximized by performing control optimization with knowledge of the powertrain characteristics and future driving conditions. Previous studies have shown that Pontryagin’s minimum principle (PMP) performs well in vehicle speed optimization problems. Building on the methods proposed in previous studies, the contribution of this study is to derive meaningful observations from the concepts and results of PMP to enhance the understanding of the control problem. In particular, the switching behavior of the control mode is analyzed with supportive variables, such as ξ and mv, which dictates the changes in the control modes. Additionally, the existence of the singular control is analyzed, which helps in understanding the cruise driving in the control problem. Finally, we obtain several solutions that satisfy various boundary conditions along with a map of the reachable states, and discuss the impact of cruise driving. This is helpful for designing practical control concepts for real-world applications based on this map. Previous studies have contributed significantly to this control problem; however, this study provides a better understanding of the issue and offers guidance and inspiration for future real-world applications based on these meaningful observations.

33 ADVANCED PROPULSION SYSTEMS↗

CUDO: closed-form universal dwell-time optimization for computer-controlled optical surfacing

Precision optical figuring demands fast and accurate dwell time optimization to reach nanometer- and sub-nanometer-level accuracy in next-generation optical systems. We introduce CUDO (closed-form universal dwell-time optimization), the first, to the best of our knowledge, unified closed-form analytical framework that supports both function-form and matrix-form dwell time models in computer-controlled optical surfacing (CCOS). In contrast to traditional methods, which rely on iterative optimization and hyperparameter tuning, our framework derives direct analytical solutions with no adjustable parameters. This approach unifies the solution principles of existing methods within a single mathematical model, delivering three key advantages: (1) accuracy on par with, or superior to, iterative solvers, (2) substantial reduction in computation time, and (3) numerical robustness. Comparative studies with prior art confirm that closed-form solutions achieve equivalent residual error while removing runtime bottlenecks. By simplifying the implementation and enabling real-time, scalable deployment, CUDO establishes a practical foundation for future deterministic fabrication of large-aperture and high-performance optics.

36 MATERIALS SCIENCE↗

Virtual to Physical: Reinforcement Learning to Optimize SNS Particle Accelerator Controls

Complex accelerators must have control systems that can handle dynamic nonlinear environments. This makes traditional control methods unsuitable as they can struggle to adapt to these uncertainties. This provides an ideal environment for reinforcement learning algorithms as they are adaptable and generalizable. We present a reinforcement learning pipeline that can effectively handle the dynamics of a complex accelerator. We test and prove our pipelines capabilities on multiple environments including the Spallation Neutron Source (SNS) and the Beam Test Facility (BTF) at Oakridge National Lab (ORNL). Due to the limited time available to train an online algorithm like reinforcement learning on a real accelerator, we utilize a virtual twin accelerator (VIRAC) developed by ORNL to pretrain the policy and show its ability to converge in the virtual environment. We then test the adaptability of the pretrained RL model by applying it on the real accelerator and comparing the results. Utilizing our Scientific Optimization and Controls Toolkit (SOCT) and open-source standards such as Gymnasium we create and solve for a MEBT orbit correction problem in the SNS and an emittance maximization problem in the BTF. We show how Twin Delayed Deep Deterministic Policy Gradient (TD3) can solve this optimization environment in the virtual accelerator and transfer this policy onto the real accelerator for inference and model retraining. We show how reinforcement learning can be utilized as a control system for complex accelerators and provide a model pipeline for how an implementation performs and can be adapted to new accelerator control problems.

Kasparian, Armen [Thomas Jefferson National Accele↗

Qudit Gate Decomposition Dependence for Lattice Gauge Theories

In this work, we investigate the effect of decomposition basis on primitive qudit gates on superconducting radio-frequency cavity-based quantum computers with applications to lattice gauge theory. Three approaches are tested: SNAP & Displacement gates, ECD & single-qubit rotations $R(\theta,\phi)$, and optimal pulse control. For all three decompositions, implementing the necessary sequence of rotations concurrently rather then sequentially can reduce the primitive gate run time. The number of blocks required for the faster ECD &$R_p(\theta)$ is found to scale $\mathcal{O}(d^2)$, while slower SNAP & Displacement set scales at worst $\mathcal{O}(d)$. For qudits with $d<10$, the resulting gate times for the decompositions is similar, but strongly-dependent on experimental design choices. Optimal control can outperforms both decompositions for small $d$ by a factor of 2-12 at the cost of higher classical resources. Lastly, we find that SNAP & Displacement are slightly more robust to a simplified noise model.

Kürkçüoglu, Doga Murat↗

Cost-optimal selection of pH control for mineral scaling prevention in high recovery reverse osmosis desalination

Conference proceedings for Foundations of Computer Aided Process Design (FOCAPD) 2024 that details a cost optimization model for pH control to prevent mineral scaling in high recovery reverse osmosis. This work applies a framework for incorporating detailed water chemistry predictions such as precipitation, pH change, and mineral scaling tendencies with process-scale optimization. It demonstrates that the acid choice for pH control can have a significant impact on pretreatment costs.

Amusat, Oluwamayowa↗

Development of Automated Atom Probe Tomography capability to study the influence of applied voltage and laser power on the final apparent composition of the analyzed specimen

This study presents the development and implementation of an autonomous Bayesian optimization (BO) framework for controlling and optimizing experimental parameters in Atom Probe Tomography (APT). Using commercial silicon needle samples as a benchmark system, we demonstrate that BO can efficiently navigate the complex parameter space of voltage and laser power to achieve target charge state ratios (specifically Si + /(Si + +Si 2+ )) with minimal experimental evaluations. Our implementation integrates Gaussian Process modeling with the CAMECA atom probe control framework, enabling autonomous adjustment of experimental conditions in real-time. Results show that the algorithm successfully converges to target ratios under different scenarios: maintaining a reference ratio, increasing the ratio (favoring Si 1+ ), and decreasing the ratio (favoring Si 2+ ). The system adapts to specimen evolution during analysis, compensating for changes in apex geometry while maintaining optimization targets. This work establishes a proof of concept for AI-driven optimization in APT, addressing the traditional challenges of manual parameter tuning and paving the way for applications to more complex materials where compositional accuracy is critical.

36 MATERIALS SCIENCE↗

Diagnostics: Chapter 8 of the special issue: on the path to tokamak burning plasma operation

This chapter presents the activity conducted by the ITPA topical group (TG) on Diagnostics over about the last 15 years. Following a general introduction of the ITER Diagnostics led by their measurement roles, the document is organized in several subchapters detailing the design support, research and development activity conducted by each of the specialist working groups (WGs) of the TG. Please note that the magnetic diagnostics were supported at the TG without a specific WG. Their status is included in the general introduction. In the following some highlights of the subchapter’s contents are provided. Recent advances in ITER first wall (FW) diagnostics for the measurements of plasma-metallic wall interaction in support of the ITER research plan are reported. An InfraRed imaging Video Bolometer for ITER has been developed and tested on several tokamaks to measure the radiated power loss. A laser-induced breakdown spectroscopy (LIBS) technique which utilizes a pulsed laser beam to ablate locally by forming a crater, will measure local tritium inventory in the FW material. Real-time Residual Gas Analyzers will measure the neutral gas composition in a divertor port and an equatorial port during plasma operation. Due to the full metallic FW environment, the plasma-wall interaction in ITER will face several challenges such as the compromised radiated power and divertor heat flux measurements by reflection. Ray tracing and analysis codes have been developed to eliminate and correct the effects of reflection in the measurements. The characteristics of the reflecting surfaces depending on the roughness and angle of the incidence have been measured by dedicated experiments, and the results were applied to the reflection elimination. For the measurement of the metallic impurity radiation induced by eroded metallic atoms, a vacuum ultraviolet spectrometer has been developed and tested. An extensive thermonuclear diagnostic suite will be required to support the operation of ITER and the planned experimental program for future burning plasma experiments. Due to the harsh environmental conditions, the implementation of diagnostic systems in ITER is a major challenge. These conditions include high levels of neutron and gamma fluxes, neutron heating, particle bombardment. Therefore, the selection and design of diagnostic systems must take into account a number of phenomena previously unseen in diagnostic design. For this reason, the measurement of neutrons and confined or lost fast ions, with particular emphasis on alpha particles, is critical to ITER. The diagnostics associated with these measurements will be important for future plasma-burning experiments at ITER. The high neutron emission and very large plasma size in ITER make neutron diagnostics the main diagnostic method used to measure plasma parameters such as fusion power, fusion power density, ion temperature, energy of fast ions and their spatial distributions in the plasma core. Active spectroscopy techniques are methods where a neutral particle beam is injected into the plasma and information on plasma parameters is extracted from the measurement of line emission resulting from the beam-plasma interaction, either by plasma ions or by beam atoms. Spatial localization is achieved by crossing the beamline and multiple observation lines. The ITER plasma will be a high temperature, moderately dense, fully ionized collisional plasma. The plasma facing surfaces are principally metallic being fashioned from beryllium or tungsten but many other elements, arising from either structural or from operational needs, may enter this plasma. The energy range of the emitted photons range from meV (infra-red) to multi keV (x-rays) and originate from all areas of the plasma volume. The primary role of passive emission diagnostics is to identify what is in the plasma from spectral signatures. Extracting quantitative information from these measurements such as impurity content, ion temperature, rotation, degree of detachment and radiated power depends on calibrated instruments, a physics model of the atomic and molecular processes and plasma transport and an analysis workflow that takes into account environmental effects such as reflections. The particular needs for ITER have prompted a multi-machine, many-year effort to address all these aspects and this chapter reviews the work on diagnostic design, experiments and new analysis techniques. An overview of the laser diagnostics to be implemented on ITER is also provided in this paper. This includes descriptions of the Thomson scattering in the core, edge and divertor regions, polarimetry and interferometry diagnostics used for measuring plasma density and also measurements of helium density in the divertor using Laser Induced Flourescence. Techniques which can allow improvements on current measurements are also addressed in particular expanding poloidal polarimetry measurements to measure field fluctuations and proposed use of dispersion interferometery which has a number of advantages over existing methods. This paper identifies particular areas where further research and testing on existing tokamaks is useful even at this advanced stage to inform the design of diagnostics for ITER. Outstanding areas of concern for the implementation of laser diagnostics, in particular with a view to reliable operation are identified. An overview of the latest developments of microwave diagnostic systems and techniques is given. The primary focus is the contributions for ITER—the next step burning plasma experiment—which is supplemented by describing recent progress of techniques applicable for fusion experiments beyond ITER. The contributions are intentionally kept concise, and are being supplemented by a rich list of references for further studies. Radiation induced effects are receiving continuous and well-deserved attention of the ITER diagnostic community and they are in many cases one of the primary design drivers of the ITER diagnostic systems. The paper summarizes recent progress in this area focusing primarily on the ITER diagnostics but in some cases provides also outlook for the possible solutions for even more demanding radiation environment of fusion reactors beyond ITER. Despite advancements in the area of modeling and simulation of various radiation induced effects, experimental testing in a nuclear environment as close as possible to the target one is still seen as unavoidable for proper qualification of particular diagnostic functional elements. Recent advancement within three diagnostic areas: optical diagnostics, magnetics and bolometers is covered. Encouraging results on qualification of silica glass vacuum window assemblies are presented. In the area of magnetic sensors, progress of irradiation tests performed on ITER in-vessel LTCC inductive sensors is presented with outlook for novel technological approaches to inductive sensors utilizing thick printing and photolithography technologies being highlighted. Summary of advancements in the area of steady state magnetic field sensors based on Hall effect is given. New results of neutron irradiation test of the ITER borosilicate glass inserts for vacuum electrical feedthroughs are summarized finding negligible swelling at target level of neutron fluence. Off-line irradiation tests of fiber optic current sensors for plasma current measurement demonstrated that both for gamma doses up to 5 MGy and a total neutron fluence up to 10 15 cm −2 , radiation induced changes are still compatible with required measurement accuracy on ITER. The ITER bolometers are given as an example how considering radiation effects may influence the diagnostic design. Finally, outlook for future main R&D directions is outlined. All optical and laser-based diagnostics in ITER will be using mirrors to guide plasma radiation toward detectors, cameras and sensors. In the hostile plasma, radiation and particle environment the optical characteristics of diagnostic mirrors will degrade directly affecting the entire performance of involved diagnostic systems. An assessment of factors affecting mirror performance is provided. Among the prime adverse factors are deposition of plasma impurities, sputtering of mirror surface and steam ingress in the vicinity of mirrors. Within the International Tokamak Physics Activity with active support by ITER central team and domestic agencies, the structured research and development (R&D) program on mitigation of risks for diagnostic mirrors is underway. Within this program the mirror material development, the passive mitigation of mirror degradation by using diagnostic ducts and shutters along with an active mirror recovery program comprising the in-situ mirror cleaning and calibration is underway. Recent developments in diagnostic mirror R&D are described in this Chapter along with an example of their implementation of R&D solutions in ITER Infrared Thermography diagnostic. An assessment of still open engineering and physics questions, considerations on mirror risks during an early phase of ITER operation are given along with an overview of diagnostic mirror evolution in the late ITER operation stage toward the demonstration fusion power plant. Several crucial areas of diagnostic R&D outlined in ITER Research Plan are addressed. The basic control groups in a fusion reactor can be broken-down in five categories: (1) plasma position, magnetic configuration, and plasma current control, (2) profile control and confinement optimization, (3) MHD control and suppression, (4) edge dissipation control, radiation and plasma exhaust control and (5) break-down optimization. These categories are coupled via the physics (a control action in one domain will affect the other domains) and via shared actuators (e.g. ECRH for impurity accumulation avoidance, current density distribution control and MHD suppression). Consequently, a supervisory control system should determine the priority of the various control tasks, their couplings, and the interfaces with the safety and interlock system. For the systematic development of the various controllers taking the complexity of the plasma and the control system into account, a model-based approach is required. A short historical overview is given of the developments in systems and control theory and control engineering with special emphasis on those developments that are most relevant for Nuclear Fusion research and operation. An overview is given of the state of the field of fusion plasma control for the control categories. It will be shown how synthetic diagnostics are being developed in ITER and how they are used in diagnostic design and design validation and how they can be in model-based controller synthesis using relatively simple models. In modern control methods, multiple diagnostics are used to constrain relatively simple models. The constrained models provide an estimate for the state. This opens the route to state controllers, such as model predictive control. A major challenge in nuclear fusion research is the coherent combination of data from heterogeneous diagnostics and modeling codes for machine control and safety as well as physics studies. Measured data from different diagnostics often provide information about the same subset of physical parameters. Additionally, information provided by some diagnostics might be needed for the analysis of other diagnostics. A joint analysis of complementary and redundant data allows, e.g. to improve the reliability of parameter estimation, to increase the spatial and temporal resolution of profiles, to obtain synergistic effects, to consider diagnostics interdependencies and to find and resolve data inconsistencies. Physics-based modeling and parameter relationships provide additional information improving the treatment of ill-posed inversion problems. A coherent combination of all kind of available information within a probabilistic framework allows for improved data analysis results. The concept of integrated data analysis (IDA) in the framework of Bayesian probability theory is outlined and contrasted with conventional data analysis. Components of the probabilistic approach are summarized and specific ingredients beneficial for data analysis at fusion devices are discussed.

ITER↗

Deep reinforcement learning control for co-optimizing energy consumption, thermal comfort, and indoor air quality in an office building

With the recent demand for decarbonization and energy efficiency, advanced HVAC control using Deep Reinforcement Learning (DRL) becomes a promising solution. Due to its flexible structures, DRL has been successful in energy reduction for many HVAC systems. However, only a few researches applied DRL agents to manage the entire central HVAC system and control multiple components in both the water loop and the air loop, owing to its complex system structures. Moreover, those researches have not extended their applications by incorporating the indoor air quality, especially both CO2 and PM2.5concentrations, on top of energy saving and thermal comfort, as achieving those objectives simultaneously can cause multiple control conflicts. What's more, DRL agents are usually trained on the simulation environment before deployment, so another challenge is to develop an accurate but relatively simple simulator. Therefore, we propose a DRL algorithm for a central HVAC system to co-optimize energy consumption, thermal comfort, indoor CO2 level, and indoor PM2.5 level in an office building. To train the controller, we also developed a hybrid simulator that decoupled the complex system into multiple simulation models, which are calibrated separately using laboratory test data. The hybrid simulator combined the dynamics of the HVAC system, the building envelope, as well as moisture, CO2, and particulate matter transfer. Three control algorithms (rule-based, MPC, and DRL) are developed, and their performances are evaluated on the hybrid simulator environment with a realistic scenario (i.e., with stochastic noises). The test results showed that, the DRL controller can save 21.4 % of energy compared to a rule-based controller, and has improved thermal comfort, reduced indoor CO2 concentration. The MPC controller showed an 18.6 % energy saving compared to the DRL controller, mainly due to savings from comfort and indoor air quality boundary violations caused by unmeasured disturbances, and it also highlights computational challenges in real-time control due to non-linear optimization. Finally, we provide the practical considerations for designing and implementing the DRL and MPC controllers based on their respective pros and cons.

Guo, Fangzhou↗

A comprehensive review of dwell time optimization methods in computer-controlled optical surfacing

Dwell time plays a vital role in determining the accuracy and convergence of the computer-controlled optical surfacing process. However, optimizing dwell time presents a challenge due to its ill-posed nature, resulting in non-unique solutions. To address this issue, several well-known methods have emerged, including the iterative, Bayesian, Fourier transform, and matrix-form methods. Despite their independent development, these methods share common objectives, such as minimizing residual errors, ensuring dwell time's positivity and smoothness, minimizing total processing time, and enabling flexible dwell positions. This paper aims to comprehensively review the existing dwell time optimization methods, explore their interrelationships, provide insights for their effective implementations, evaluate their performances, and ultimately propose a unified dwell time optimization methodology.

36 MATERIALS SCIENCE↗

Advanced Transmission Technologies – GETs and HPCs Session 2: Advanced Power Flow Control and Transmission Topology Optimization

The INL TADA GETs Cohort Session 2, held on November 7, 2025, conducted in collaboration with ScottMadden, focused on two core Advanced Transmission Technologies (ATTs): Advanced Power Flow Control (APFC) and Transmission Topology Optimization (TTO). These technologies are pivotal in enhancing grid flexibility, reliability, and cybersecurity resilience. APFC, particularly through modular FACTS devices like Modular Static Synchronous Series Compensators (M-SSSCs), enables dynamic voltage injection to reroute power flows. The session highlighted the deployment benefits of APFC, such as rapid installation, minimal civil works, and re-deployability. Regulatory drivers like FERC Order 2023 mandate the inclusion of Grid-Enhancing Technologies (GETs) in interconnection studies. Case studies from Central Hudson, CAISO, and National Grid (UK) demonstrated APFC’s effectiveness in congestion relief and cost savings. The session also addressed cybersecurity concerns, including firmware vulnerabilities, SCADA integration risks, and supply chain dependencies. Participants engaged in interactive exercises to rank cybersecurity and supply chain risks, emphasizing the need for robust digital assurance strategies. TTO involves software-based reconfiguration of transmission networks to optimize power flow without new infrastructure. The session showcased its operational value, with examples from SPP, PJM, and MISO showing significant congestion cost reductions. Cybersecurity vulnerabilities were discussed, particularly in API security and software supply chains, referencing incidents like SolarWinds and attacks on Danish utilities. Digital assurance exercises explored worst-case scenarios, attack paths, and mitigation responsibilities between vendors and utilities. Reliability challenges such as algorithm stability, vendor dependency, and operator trust were also examined. Cross-cutting themes emphasized the importance of digital assurance tools, including Software Bills of Materials (SBOMs) and hardware-in-loop testing. Human performance, training, and operational confidence were identified as critical enablers of technology adoption. The session concluded with a preview of Session 3, which will focus on High Performance Conductors (HPCs) and risk-based cybersecurity tools. Session 2 of 3.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Reinforcement Learning Control for Buildings Co-Optimizing Energy, Comfort, and Indoor Air Quality: An Annual Assessment

Efficient control of Heating, Ventilation, and Air Conditioning (HVAC) systems is crucial for optimizing energy use and maintaining indoor comfort in buildings. Traditional control methods, such as PID control, cannot handle energy use trade-offs among multiple components in the building energy system at a supervisory level. Reinforcement learning (RL) presents a promising solution, offering adaptive and data-driven control strategies that optimize performance over time. However, RL also faces several challenges, including the conflicts encountered in co-optimizing energy savings, occupant comfort, and indoor air quality, and the requirement for extensive interactions with the environment in training. We proposed a flexible simulation platform that integrates a hybrid model for RL training and designed an RL agent to control the entire central HVAC system, focusing on co-optimizing energy consumption, thermal comfort, and indoor air quality ($\text{CO}_{2}$ and PM2.5 concentrations). Finally, we evaluated the RL agent's performance over an annual cycle. Our findings indicate that the RL agent can effectively manage the HVAC system with 14.7 % energy savings annually and balance multiple objectives, which demonstrates significant potential for improving HVAC system control and sustainability in buildings.

Guo, Fangzhou↗

Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Active distribution grids can experience voltage fluctuations and violations due to the high penetration of variable distributed energy resources (DERs). These problems might occur because of the uncertain and variable generation natures of these resources, especially solar photovoltaic resources, during panel shadowing scenarios. Volt-VAR control (VVC) is an efficient method that controls the reactive power set-points of the inverters to regulate the voltage of distribution grids. Although several VVC approaches have been proposed recently, the performance of these approaches degrades significantly if behind-the-meter solar generation data are unobservable/missing. Therefore, it is necessary to impute missing/unobservable PV data accurately to be utilized in VVC approaches. Further, this paper proposes a model-free, data-driven, centrally trained, and decentrally executed multi-agent deep reinforcement learning-based VVC architecture to regulate the voltage of distribution networks. A generative adversarial network (GAN) is incorporated to impute the unobservable PV data accurately, which improves the performance of the proposed control architecture. The proposed multi-agent-soft-actor–critic algorithm (MASAC)-based VVC technique utilizes the actual PV dataset as well as the imputed dataset from the GAN framework to learn the optimal coordinated control policy for controlling the optimal reactive power set-points of PV inverters. The effectiveness of the proposed approach is analyzed on a modified IEEE 34-bus test case with added PV inverters. The results are compared and analyzed with a base case model with no VVC and VVC with a local droop control approach, genetic algorithm optimization, and a centralized soft actor–critic-based approach. Moreover, the performance of the proposed approach is compared with that of a multi-agent VVC framework without using the PV generation data and load information as the system state. The results illustrate that the proposed method with more state input improves the voltage profile and reduces the power loss of the network across various loading and PV generation scenarios.

14 SOLAR ENERGY↗