Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “agent based simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

An Edge-Cloud Integrated Solution for Buildings Demand Response Using Reinforcement Learning

Buildings, as major energy consumers, can provide great untapped demand response (DR) resources for grid services. However, their participation remains low in real-life. One major impediment for popularizing DR in buildings is the lack of cost-effective automation systems that can be widely adopted. Existing optimization-based smart building control algorithms suffer from high costs on both building-specific modeling and on-demand computing resources. To tackle these issues, this paper proposes a cost-effective edge-cloud integrated solution using reinforcement learning (RL). Beside RL’s ability to solve sequential optimal decision-making problems, its adaptability to easy-to-obtain building models and the off-line learning feature are likely to reduce the controller’s implementation cost. Using a surrogate building model learned automatically from building operation data, an RL agent learns an optimal control policy on cloud infrastructure, and the policy is then distributed to edge devices for execution. Simulation results demonstrate the control efficacy and the learning efficiency in buildings of different sizes. A preliminary cost analysis on a 4-zone commercial building shows the annual cost for optimal policy training is only 2.25% of the DR incentive received. Results of this study show a possible approach with higher return on investment for buildings to participate in DR programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Towards Immobilized Proton-Coupled Electron Transfer Agents for Electrochemical Carbon Capture from Air and Seawater

Electrochemical CO 2 separation has drawn attention as a promising strategy for using renewable energy to mitigate climate change. Redox-active compounds that undergo proton-coupled electron transfer (PCET) are an impetus for pH-swing-driven CO 2 capture at low energetic costs. However, multiple barriers hinder this technology from maturing, including sensitivity to oxygen and the slow kinetics of CO 2 capture. Here, we use vapor phase chemistry to construct a textile electrode comprising an immobilized PCET agent, poly(1-aminoanthraquinone) (PAAQ), and incorporate it into redox flow cells. This design contrasts with others that use dissolved PCET agents by confining proton-storage to the surface of an electrode kept separate from an aqueous, CO 2 -capturing phase. This system facilitates carbon capture from gaseous sources (a 1% CO 2 feed and air), as well as seawater, with the latter at an energetic cost of 202 kJ/mol CO2 , and we find that quinone moieties embedded within the electrode are more stable to oxygen than dissolved counterparts. Simulations using a 1D reaction-transport model show that moderate energetic costs should be possible for air capture of CO 2 with higher loadings of polymer-bound PCET moieties. The remarkable stability of this system sets the stage for producing textile-based electrodes that facilitate pH-swing-driven carbon capture in practical situations.

Ali, Fawaz↗
DOE OSTI2026

Optimization and stabilization of Fermilab Booster using hybrid Bayesian/RL framework

PIPII project will raise Fermilab Booster intensity and ramp rate. Beam losses will limit average power and are hard to simulate. Presently, Booster uses operator-guided empirical tuning. This task is challenging due to high dimensionality, multiple objectives, critical safety constraints, and drifts. We developed a synergistic suite of Bayesian optimization (BO) and reinforcement learning (RL) tools to optimize and stabilize beam losses. First, active learning was used to build a rough model. Data was collected parasitically using two novel safety constraint types – nonlinear input space restrictions (based on optics model), and uncertainty constraints (to stop bad steps/beam aborts). We then applied online multi-objective BO with scalarized objectives and fitting to improve/rebalance losses, increasing safety margins by 25%. Using BO model as a safety veto, we tried several on/off-policy RL agents for long term stabilization; SAC had best performance. We found that adding contextual (state) information further improved performance, eventually integrating key knobs like linac phase and temperature into the parameter space. Long term testing is ongoing to enable operational use.

Kuklev, Nikita [Fermilab]↗
NASA NTRS2017

Distributed Fast Motion Planning for Spacecraft Swarms in Cluttered Environments using Spherical Expansions and Sequence of Convex Optimization Problems

This paper presents a novel guidance algorithm for spacecraft swarms in an environment cluttered with many obstacles like a debris field or the asteroid belt. The objective of this algorithm is to reconfigure the swarm to a desired formation in a distributed manner while minimizing fuel and avoiding collisions among themselves and with the obstacles. The agents first use a spherical-expansion-based sampling algorithm to cooperatively explore the workspace and find paths to the desired terminal positions. Using a distributed assignment algorithm, the agents converge on an optimal assignment of the target locations in the desired formation. Then each agent generates a locally optimal trajectory from its current location to its terminal position by solving a sequence of convex optimization problems. As the agent moves along this trajectory, it receives the position of other agents and updates its trajectory to avoid collisions with other agents and the obstacles. Thus the swarm achieves the desired formation in a distributed manner while avoiding collisions. Moreover, this algorithm is computationally efficient, therefore it can be implemented onboard resource-constrained spacecraft. Simulations results show that the proposed distributed algorithm can be used by a spacecraft swarm to reconfigure a desired formation around an asteroid in a collision-free manner.

Bandyopadhyay, Saptarshi↗
DOE OSTI2022

Stochastic simulation of occupant-driven energy use in a bottom-up residential building stock model

The residential buildings sector is one of the largest electricity consumers worldwide and contributes disproportionally to peak electricity demand in many regions. Strongly driven by occupant activities, household energy consumption is stochastic and heterogeneous in nature. However, most residential energy models applied by industry use homogeneous, deterministic activity schedules, which work well for predictions of annual energy consumption, but can result in unrealistic hourly or sub-hourly electric load profiles, with exaggerated or muted peaks. The increasing proportion of variable renewable energy generators means that representing the heterogeneity and stochasticity of occupant behavior is now crucial for reliable planning at both bulk-power and distribution-system scales. This work presents a novel and open-source occupancy simulation approach that can simulate a diverse set of individual occupant and household event schedules for all major electricity, fuel, and hot water end uses. To accomplish this, we evaluated three alternative occupant activity simulation approaches before selecting a hybrid combining time-inhomogeneous Markov chains and probability-sampling of event durations and magnitudes. Further, we integrated the stochastic occupancy simulation with an open-source bottom-up physics-simulation building stock model and published a set of 550,000 diverse household end-use activity schedules representing a national housing stock. The simulator was verified against time-use survey data, and simulation results were validated against measured end-use electricity data for accuracy and reliability. While we use data for the United States, our application demonstrates how similar approaches could be applied using the time-use survey data collected in many countries around the world.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

A Self-Organizing Nano Grid (SONG) for Energy Access Clusters

In this paper, a bottom-up dc Self-Organizing Nano Grid (SONG) for rural electrification at ultra-low-cost through a flexible and scalable topology is presented. The proposed nano grid requires no communication and uses dc-bus signaling to convey excess energy in the system through the bus voltage, allowing independent agents to conduct energy transactions based on their needs in a decentralized manner. Moreover, this article proposes a concept that uses a novel multi-source, multi-segment droop control strategy with demand coordination, allowing for dynamic self-balancing of the system under fluctuating operating conditions. The unique use of a surplus photovoltaic (PV) power droop mode precisely estimates energy availability to minimize load chattering that helps in stabilizing the dc-bus. In addition, this architecture uses standard dc/dc converters for a low-cost, modular, Plug and Play ( PnP ) solution with highly automated operation, enabling local assembly, operation, and maintenance by non-technical personnel. The proposed SONG concept was validated in simulation and with a 700W, laboratory-scale, custom made nano grid system. The simulation and experimental test cases show the feasibility of the SONG system by demonstrating the proposed multi-source, multi-segment droop control strategy without load chattering under varying operating conditions.

dc-dc converter↗
DOE OSTI2023

Distributed Optimal Power Management for Battery Energy Storage Systems: A Novel Accelerated Tracking ADMM Approach

Optimal power management (OPM) is critical for large-scale battery energy storage systems. Today’s methods often require formidable computational effort due to the design based on centralized numerical optimization. Thus, this paper investigates computationally distributed OPM where the agents based on the cells communicate over a network to cooperatively solve the OPM problem. We propose an accelerated tracking alternating direction method of multipliers (ADMM) algorithm to solve the distributed OPM. The proposed algorithm embeds dynamic average consensus and Nesterov’s acceleration technique in the ADMM algorithm. Not only is the proposed algorithm fully distributed without a need for fusion or aggregating nodes, but it also accelerates convergence. The paper formulates the OPM in a model predictive control framework where it seeks to regulate the charging/discharging power of each battery cell to minimize the total power losses and promote balanced use of the constituent cells while complying with the safety constraints. The paper provides ample simulation results to demonstrate the effectiveness and advantages of the proposed distributed OPM in terms of computation and convergence.

Farakhor, Amir↗
NASA NTRS1996

A dual-input nonlinear system analysis of autonomic modulation of heart rate

Linear analyses of fluctuations in heart rate and other hemodynamic variables have been used to elucidate cardiovascular regulatory mechanisms. The role of nonlinear contributions to fluctuations in hemodynamic variables has not been fully explored. This paper presents a nonlinear system analysis of the effect of fluctuations in instantaneous lung volume (ILV) and arterial blood pressure (ABP) on heart rate (HR) fluctuations. To successfully employ a nonlinear analysis based on the Laguerre expansion technique (LET), we introduce an efficient procedure for broadening the spectral content of the ILV and ABP inputs to the model by adding white noise. Results from computer simulations demonstrate the effectiveness of broadening the spectral band of input signals to obtain consistent and stable kernel estimates with the use of the LET. Without broadening the band of the ILV and ABP inputs, the LET did not provide stable kernel estimates. Moreover, we extend the LET to the case of multiple inputs in order to accommodate the analysis of the combined effect of ILV and ABP effect on heart rate. Analyzes of data based on the second-order Volterra-Wiener model reveal an important contribution of the second-order kernels to the description of the effect of lung volume and arterial blood pressure on heart rate. Furthermore, physiological effects of the autonomic blocking agents propranolol and atropine on changes in the first- and second-order kernels are also discussed.

NASA Discipline Regulatory Physiology↗
DOE OSTI2026

Surrogates for Valve-Controlled Pipe Flow: Accelerating Nuclear Reactor Design

Neural surrogate models are developed to replace expensive steady-state RANS CFD simulations for valve-controlled pipe flow in nuclear reactor design. Using parametric CFD data generated with MOOSE Pronghorn across a range of valve geometry and flow conditions, three approaches are compared: a POD-based reduced-order model, a structured UNet on a cylindrical grid, and unstructured models (DeepONet and BiStride MeshGraphNet) on nondimensionalized point clouds. POD achieves the highest accuracy (99%) with fast inference but requires storing all solution snapshots, while the DeepONet and BSMS-GNN both achieve ~89% accuracy at sub-second inference, with the BSMS-GNN offering superior geometric generalizability. These surrogates enable rapid ranking of candidate valve designs and can warm-start CFD solvers to accelerate convergence, supporting agentic design iteration on the Prometheus platform.

42 - ENGINEERING↗
DOE OSTI2023

Transactive Energy System Deployment Over Insecure Communication Links

Here, in this paper, the privacy and security issues associated with the transactive energy system (TES) deployment over insecure communication links are addressed. In particular, it is ensured that 1) individual agents’ bidding information is kept private throughout hierarchical market-based interactions; and 2) any extraneous data injection attack can be quickly and easily detected. An implementation framework is proposed to enable the cryptography-based enhancement of privacy and security for the deployment of any general hierarchical systems including TESs. Under the proposed framework, a unified cryptography-based approach is developed to achieve both privacy and security simultaneously. Specifically, privacy preservation is realized by an enhanced Paillier encryption scheme, where a block design is proposed to significantly improve computational efficiency. Attack detection is further achieved by an enhanced Paillier digital signature scheme, where a stamp-concatenation mechanism is proposed to enable detection of data replace and reorder attacks. Simulation results verify the effectiveness of the proposed cyber-resilient design for transactive energy systems. Note to Practitioners—This paper is motivated by addressing the issues of cyber resiliency for practically deploying transactive energy system (TES) but it is also applicable to the problem of enhancing the privacy and security for any general hierarchical control systems. TES is an emerging control approach that engages energy suppliers and customers through market operations and uses the price to optimally allocate energy resources. While it has been shown to be promising for power system applications, the underlying market-based interactions raise significant concerns of privacy (data leakage) and security (data tampering). However, existing TES works only focus on the coordination mechanism instead of privacy and security issues. This paper proposes a new cryptography-based TES design for practical deployment. Specifically, to protect privacy, individual supply and demand amounts to be exchanged are all encrypted in a particular way such that the original amounts cannot be inferred from the encrypted amounts, while the desired computation for setting the market clearing price can be carried out over the encrypted amounts, thus generating an encrypted result which, when decrypted, matches that of the same computation over the original amounts. To achieve security, for each exchanged data, its sender generates a particular digital signature which is exchanged together with the data. This enables the receiver to automatically detect the integrity by checking whether a mathematical relationship holds for the pair of data and signature. In our future research, we will investigate more challenging scenarios where some suppliers and customers themselves could be corrupted and purposely submit distorted amounts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Efficient Reinforcement Learning for Real-Time Hardware-Based Energy System Experiments: Preprint

In the context of urgent climate challenges and the pressing need for rapid technology development, Reinforcement Learning (RL) stands as a compelling data-driven method for controlling real-world physical systems. However, RL implementation often entails time-consuming and computationally intensive data collection and training processes, rendering them inefficient for real-time applications that lack non-real-time models. To address these limitations, real-time emulation techniques have emerged as valuable tools for the lab-scale rapid prototyping of intricate energy systems. While emulated systems offer a bridge between simulation and reality, they too face constraints, hindering comprehensive characterization, testing, and development. In this research, we construct a surrogate model using limited data from simulated systems, enabling an efficient and effective training process for a Double Deep Q-Network (DDQN) agent for future deployment. Our approach is illustrated through a hydropower application, demonstrating the practical impact of our approach on climate-related technology development.

deep Q-learning↗
DOE OSTI2024

SEAS Communication Engine: An Extensible, Flexible Wrapper for Co-Simulation Agents

When modeling and analyzing the power grid and other large scale systems, researchers often express scenarios as optimization problems and feed them into advanced software solvers. In order to allow multiple solvers to communicate with each other and share data from different domains, the National Renewable Energy Laboratory (NREL) and associated Department of Energy (DOE) labs have developed a software framework called the Hierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS). HELICS allows cosimulation via a collection of client libraries for different languages that can be called from the appropriate optimization software. However, these client libraries do not provide a higher level of abstraction beyond reading and writing data off of the shared HELICS bus. In this paper, we describe a new software library called the SEAS Communication Engine that exposes a higher-level API for running cosimulation problems. The SEAS Engine provides a class-based abstraction on top of the Python HELICS client, in order to allow users to implement their domain-specific cosimulations without needing to interact with core HELICS primitives. This will make adoption of HELICS and cosimulation in general easier, by exposing a simpler API. In the second part of the paper, we validate our library on a collection of different simulation examples, including the canonical IEEE 13 Bus Feeder. Lastly, we demonstrate using the SEAS Engine to directly call domain-specific code written in the Julia programming language. Our hope is that this will serve as a template for easily calling software in different programming languages via the SEAS Engine, thereby avoiding code duplication and complexity.

co-simulation↗
NASA NTRS2003

A Real-Time Rover Executive based On Model-Based Reactive Planning

This paper reports on the experimental verification of the ability of IDEA (Intelligent Distributed Execution Architecture) effectively operate at multiple levels of abstraction in an autonomous control system. The basic hypothesis of IDEA is that a large control system can be structured as a collection of interacting control agents, each organized around the same fundamental structure. Two IDEA agents, a system-level agent and a mission-level agent, are designed and implemented to autonomously control the K9 rover in real-time. The system is evaluated in the scenario where the rover must acquire images from a specified set of locations. The IDEA agents are responsible for enabling the rover to achieve its goals while monitoring the execution and safety of the rover and recovering from dangerous states when necessary. Experiments carried out both in simulation and on the physical rover, produced highly promising results.

Bias, M. Bernardine↗
NASA NTRS1992

Human-like agents with posture planning ability

Human body models are geometric structures which may be ultimately controlled by kinematically manipulating their joints, but for animation, it is desirable to control them in terms of task-level goals. We address a fundamental problem in achieving task-level postural goals: controlling massively redundant degrees of freedom. We reduce the degrees of freedom by introducing significant control points and vectors, e.g., pelvis forward vector, palm up vector, and torso up vector, etc. This reduced set of parameters are used to enumerate primitive motions and motion dependencies among them, and thus to select from a small set of alternative postures (e.g., bend vs. squat to lower shoulder height). A plan for a given goal is found by incrementally constructing a goal/constraint set based on the given goal, motion dependencies, collision avoidance requirements, and discovered failures. Global postures satisfying a given goal/constraint set are determined with the help of incremental mental simulation which uses a robust inverse kinematics algorithm. The contributions of the present work are: (1) There is no need to specify beforehand the final goal configuration, which is unrealistic for the human body, and (2) the degrees of freedom problem becomes easier by representing body configurations in terms of 'lumped' control parameters, that is, control points and vectors.

Jung, Moon R.↗
NASA NTRS2001

Architectures and Evaluation for Adjustable Control Autonomy for Space-Based Life Support Systems

In the past five years, a number of automation applications for control of crew life support systems have been developed and evaluated in the Adjustable Autonomy Testbed at NASA's Johnson Space Center. This paper surveys progress on an adjustable autonomous control architecture for situations where software and human operators work together to manage anomalies and other system problems. When problems occur, the level of control autonomy can be adjusted, so that operators and software agents can work together on diagnosis and recovery. In 1997 adjustable autonomy software was developed to manage gas transfer and storage in a closed life support test. Four crewmembers lived and worked in a chamber for 91 days, with both air and water recycling. CO2 was converted to O2 by gas processing systems and wheat crops. With the automation software, significantly fewer hours were spent monitoring operations. System-level validation testing of the software by interactive hybrid simulation revealed problems both in software requirements and implementation. Since that time, we have been developing multi-agent approaches for automation software and human operators, to cooperatively control systems and manage problems. Each new capability has been tested and demonstrated in realistic dynamic anomaly scenarios, using the hybrid simulation tool.

Malin, Jane T.↗
NASA NTRS2013

Dynamics and Control of a Disordered System in Space

In this paper, we present some ideas regarding the modeling, dynamics and control aspects of granular spacecraft. Granular spacecraft are complex multibody systems composed of a spatially disordered distribution of a large number of elements, for instance a cloud of N grains in orbit, with N greater than 10(exp 3). These grains can be large (Cubesat-size) or small (mm-size), and can be active, i.e., a fully equipped vehicle capable sensing their own position and attitude, and enabled with propulsion means, or entirely passive. The ultimate objective would be to study the behavior of the single grains and of large ensembles of grains in orbit and to identify ways to guide and control the shape of a cloud composed of these grains so that it can perform a useful function in space, for instance, as an element of an optical imaging system for astrophysical applications. This concept, in which the aperture does not need to be continuous and monolithic, would increase the aperture size several times compared to large NASA observatories such as ATLAST, allowing for a true Terrestrial Planet Imager that would be able to resolve exo-planet details and do meaningful spectroscopy on distant world. In the paper, we address the modeling and autonomous operation of a distributed assembly (the cloud) of large numbers of highly miniaturized space-borne elements (the grains). A multi-scale, multi-physics model is proposed of the dynamics of the cloud in orbit, as well as a control law for cloud shape maintenance, and preliminary simulation studies yield an estimate of the computational effort, indicating a scale factor of approximately N(exp 1.4) as a function of the number of grains. A granular spacecraft can be defined as a collection of a large number of space-borne elements (in the 1000s) designed and controlled such that a desirable collective behavior emerges, either from the interactions among neighboring grains, and/or between the grains and the environment. In this paper, each grain is considered to be a highly miniaturized spacecraft which has limited size and mass, hence it has limited actuation, limited propulsive capability, limited power, limited sensing, limited communication, limited computational resources, limited range of motion, limited lifetime, and may be expendable. The modeling and dynamics of clouds of vehicles is more challenging than with conventional vehicles because we are faced with a probabilistic vehicle composed of a large number of physically disconnected vehicles. First, different scales of motion occur simultaneously in a cloud: translations and rotations of the cloud as a whole (macro-dynamics), relative rotation and translation of one cloud member with respect to another (meso-dynamics), and individual cloud member dynamics (micro-dynamics). Second, the control design needs to be tolerant of the system complexity, of the system architecture (centralized vs. decentralized large scale system control) as well as robust to un-modeled dynamics and noise sources. Figure 1, top left, shows the kinematic parameters of a 1000 element cloud in orbit. The motion of the system is described with respect to a local vertical-local horizontal (LV-LH) orbiting reference frame (x,y,z)=F(sub ORF) of origin O(sub ORF) which rotates with mean motion omega and orbital semi-major axis R(sub 0). The orbital geometry at the initial time is defined in terms of its six orbital elements, and the orbital dynamics equation for point O(sub ORF) is propagated forward in time under the influence of the gravitational field of the primary and other external perturbations, described below. The origin of this frame coincides with the initial position of the center of mass of the system, and the coordinate axes are z along the local vertical, x toward the flight direction, and y in the orbit normal direction. The assumptions we used to model the dynamics are as follows: 1) The inertial frame is fixed at Earth's center. 2) The orbiting Frame ORF follows Keplerian orbit. 3) the cloud system dynamics is referred to ORF. 4) the attitude of each grain uses the principal body frame as body fixed frame. 5) the atmosphere is assumed to be rigidly rotating with the Earth. Regarding the grains forming the cloud: 1) each grain is modeled as a rigid body; 2) a simple attitude estimator provides attitude estimates, 3) a simple guidance logic commands the position and attitude of each grain, 4) a simple local feedback controller based on PD control of local states is used to stabilize the attitude of the vehicle. Regarding the cloud: 1) the cloud as a whole is modeled as an equivalent rigid body in orbit, and 2) an associated graph establishes agent connectivity and enables coupling between modes of motion at the micro and macro scales; 3) a simple guidance and estimation logic is modeled to estimate and command the attitude of this equivalent rigid body; 4) a cloud shape maintenance controller is based on the dynamics of a stable virtual truss in the orbiting frame. Regarding the environmental perturbations acting on the cloud: 1) a non-spherical gravity field including JO (Earth's spherical field) zonal component, J2 (Earth's oblateness) and J3 zonal components is implemented; 2) atmospheric drag is modeled with an exponential model; 3) solar pressure is modeled assuming the Sun is inertially fixed; and 4) the Earth's magnetic field is model using an equivalent dipole model. The equations of motion are written in a referential system with respect to the origin of the orbiting frame and the state is propagated forward in time using an incremental predictor-corrector scheme. A representative cloud with varying number of grains is simulated to identify the limitations in computation time as the number of grains grows. We derive a control law to track a desired surface in the ORF (equivalently to maintain a reference cloud shape) by defining an error from a desired surface shape, and designing a control law that is exponentially stable and reduces the tracking error to zero. Figure 1 (top right) shows a comparison of various requirements for simulation of single spacecraft vs. granular spacecraft, indicating the high degree of complexity that needs to be taken into consideration. The ORF components of control force required by one of the grains is, for this particular case, in the micro-Newton range. However, no attempt has been made yet to reconfigure (or re-orient) the cloud configuration internally, for which forces in the milli-Newton level are expected, depending on the time required to do the reconfiguration. Figure 1, bottom, shows the computation time as a function of the number of grains, indicating an order N(exp 1.43) scaling on a 8 Gb, 1067 MHz RAM MacOSX computer with a 3.06 GHz Intel Core 2 Duo processor. With this metric, the same simulation for a system of N=1000 grains would take 5.4 hours, and 146 hours (i.e., 6 days) for a system with N=10,000 grains. Therefore, efficient ways to simulate this complex system, where not only the time scales of natural system dynamics, but also the sampling times of the Guidance, Navigation, and Control are included, remain to be explored. Additional details on the cloud modeling, dynamics, and control will be described in the paper.

simulation↗
DOE OSTI2021

Distributed and communication-efficient solutions to linear equations with special sparse structure

In this paper we report two distributed and communication-efficient algorithms based on the multi-agent system are proposed to solve a system of linear equations with the Laplacian sparse system matrix. One algorithm is based on the gradient descent method in optimization. In this algorithm, the agents only share partial information instead of all of their collective state vectors to save significant communication. The other algorithm is obtained by approximating Newton’s method for a faster convergence rate. Although it requires twice as much communication as the first one, it is still communication-efficient given the low dimension of the information shared among agents. The convergence at a linear rate is proved for both algorithms, and a comprehensive comparison of their convergence rate, communication burden, and computation costs is also performed. The proposed algorithms can be applied to various systems to solve those problems that can be modeled as a system of linear equations with a Laplacian sparse system matrix. Simulation results with the electric power system illustrate their effectiveness.

42 ENGINEERING↗
DOE OSTI2020

Evaluating the Adaptability of Reinforcement Learning Based HVAC Control for Residential Houses

Intelligent Heating, Ventilation, and Air Conditioning (HVAC) control using deep reinforcement learning (DRL) has recently gained a lot of attention due to its ability to optimally control the complex behavior of the HVAC system. However, more exploration is needed on understanding the adaptability challenges that the DRL agent could face during the deployment phase. Using online learning for such applications is not realistic due to the long learning period and likely poor comfort control during the learning process. Alternatively, DRL can be pre-trained using a building model prior to deployment. However, developing an accurate building model for every house and deploying a pre-trained DRL model for HVAC control would not be cost-effective. In this study, we focus on evaluating the ability of DRL-based HVAC control to provide cost savings when pre-trained on one building model and deployed on different house models with varying user comforts. We observed around 30% of cost reduction by pre-trained model over baseline when validated in a simulation environment and achieved up to 21% cost reduction when deployed in the real house. This finding provides experimental evidence that the pre-trained DRL has the potential to adapt to different house environments and comfort settings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗