Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “resource optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

Federated Architecture for Secure and Transactive Distributed Energy Resource Management Solutions (FAST-DERMS)

This document provides system-level specifications for a federated architecture for secure and transactive distributed energy resource management solutions (FAST-DERMS), presents a solution, and describes operational concepts for the proposed solution. FAST-DERMS enables the provision of reliable, resilient, and secure transmission and distribution (T&D) grid services through the scalable aggregation and near-real-time management of utility-scale and small-scale distributed energy resources (DERs). We first present the principles and objectives of FAST-DERMS. Then, after discussing important system concepts, we present the specifications for FAST-DERMS and a solution that employs a distributed and federated control methodology in which the DERs connected to a single point of common coupling with the rest of the system, such as individual substations, are optimized coordinately to provide system-level grid services. FAST-DERMS aims to aggregate and coordinate the operations of DERs to support T&D grid operations. The key optimization and control component of this FAST-DERMS reference implementation is a flexible resource scheduler (FRS) that aggregates the DERs within a substation service area. These FRSs operate at the substation level and perform constrained economic dispatch of DERs, either directly or through a transactive market or aggregator, as shown in Figure ES-1. An FRS Coordinator at the distribution system operator (DSO) level aggregates distribution substations operated by FRSs and interfaces with the transmission system operator (TSO) to provide transmission services. FAST-DERMS also allows for the integration of the FRS Coordinator with an existing distribution utility management system that could be employed by the DSO to enhance distribution grid operations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimizing I/O Performance of HPC Applications with Autotuning

Parallel Input output is an essential component of modern high-performance computing (HPC). Obtaining good I/O performance for a broad range of applications on diverse HPC platforms is a major challenge, in part, because of complex inter dependencies between I/O middleware and hardware. The parallel file system and I/O middleware layers all offer optimization parameters that can, in theory, result in better I/O performance. Unfortunately, the right combination of parameters is highly dependent on the application, HPC platform, problem size, and concurrency. Scientific application developers do not have the time or expertise to take on the substantial burden of identifying good parameters for each problem configuration. They resort to using system defaults, a choice that frequently results in poor I/O performance. We expect this problem to be compounded on exascale-class machines, which will likely have a deeper software stack with hierarchically arranged hardware resources. We present as a solution to this problem an autotuning system for optimizing I/O performance, I/O performance modeling, I/O tuning, and I/O patterns. We demonstrate the value of this framework across several HPC platforms and applications at scale.

Behzad, Babak↗

Learning with Adaptive Conservativeness for Distributionally Robust Optimization: Incentive Design for Voltage Regulation

Information asymmetry between the Distribution System Operator (DSO) and Distributed Energy Resource Aggregators (DERAs) obstructs designing effective incentives for voltage regulation. To capture this effect, we employ a Stackelberg game-theoretic framework, where the DSO seeks to overcome the information asymmetry and refine its incentive strategies by learning from DERA behavior over multiple iterations. We introduce a model-based online learning algorithm for the DSO, aimed at inferring the relationship between incentives and DERA responses. Given the uncertain nature of these responses, we also propose a distributionally robust incentive design model to control the probability of voltage regulation failure and then reformulate it into a convex problem. This model allows the DSO to periodically revise distribution assumptions on uncertain parameters in the decision model of the DERA. Finally, we present a gradient-based method that permits the DSO to adaptively modify its conservativeness level, measured by the size of a Wasserstein metric-based ambiguity set, according to historical voltage regulation performance. The effectiveness of our proposed method is demonstrated through numerical experiments.

adaptation models↗

REopt Lite User Manual

REopt Lite evaluates the economic viability of grid-connected solar photovoltaics, wind, combined heat and power, and electric and thermal storage at commercial and small industrial sites. It allows building owners to identify the system sizes and dispatch strategies that minimize the site’s life cycle cost of energy. REopt Lite also estimates the amount of time on-site generation and storage can sustain the site's critical load during a grid outage and allows the user the choice of optimizing for energy resilience. It is primarily used to inform project development decisions and to support research on the factors that drive project feasibility for market development and policy analysis. It is available through a web interface, application programing interface, and open-source code. This user manual provides an overview of the model, including its capabilities and typical applications; inputs and outputs; economic calculations; technology descriptions; and model parameters, variables, and equations. The model is highly flexible and is continually evolving to meet the needs of each analysis. Therefore, this report is not an exhaustive description of all capabilities, but rather a summary of the core components of the model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimal Droop Setting for Congestion Reduction in a 100% Grid-Forming Inverter-based Power System

he high penetration of inverter-based resources (IBRs) introduces new challenges to power systems due to the complex inverter control. However, IBRs can be configured to maximize their benefits to improve system resilience and reliability. This paper proposes a steady-state optimization model that aims to mitigate transmission congestion in a 100% grid- forming (GFM) IBR-based power system. This goal is achieved by determining the optimal droop settings for the GFM IBRs under different congestion conditions due to renewable energy and load variations. The numerical solution is rigorously verified by a high-fidelity model of the IEEE 39-bus test system with detailed GFM IBR control in the time-domain electromagnetic transient (EMT) simulation tool PSCAD. The numerical solution and simulation results show a significant congestion reduction while meeting all other operating requirements. It is also observed that the numerical solving time is substantially less compared to the EMT simulation time.

Nguyen, Quan H.↗

Determining the Adsorption Energetics of 2,3-Butanediol on RuO 2 (110): Coupling First-Principles Calculations With Global Optimizers

As climate change continues to pose a threat to the Earth due to the disrupted carbon cycles and fossil fuel resources remain finite, new sources of sustainable hydrocarbons must be explored. 2,3-butanediol is a potential source to produce butene because of its sustainability as a biomass-derived sugar. Butene is an attractive product because it can be used as a precursor to jet fuel, categorizing this work in the alcohol-to-jet pathway. While studies have explored the conversion of 2,3-butanediol to butene, little is understood about the fundamental reaction itself. We quantify the energetics for three pathways that were reported in the literature in the absence of a catalyst. One of these pathways forms a 1,3-butadiene intermediate, which is a highly exothermic process and thus is unlikely to occur since 2,3-butanediol likely gets thermodynamically trapped at this intermediate. We further determined the corresponding energetics of 2,3-butanediol adsorption on an ensemble of predetermined binding sites when it interacts with a defect-free stoichiometric RuO 2 (110) surface. Within this ensemble of adsorption sites, the most favorable site has 2,3-butanediol covering a Ru 5–coordinated cation. This approach is compared to that obtained using the global optimization algorithm as implemented in the Northwest Potential Energy Surface Search Engine. When using such a global optimization algorithm, we determined a more favorable ground-state structure that was missed during the manual adsorption site testing, with an adsorption energy of -2.61 eV as compared to -2.34 eV when using the ensemble-based approach. We hypothesize that the dehydration reaction requires a stronger chemical bond, which could necessitate the formation of oxygen vacancies. As such, this study has taken the first step toward the utilization of a global optimization algorithm for the rational design of Ru-based catalysts toward the formation of butene from sustainable resources.

09 BIOMASS FUELS↗

Intelligent Partitioning based Fully Parallel AC Security-Constrained Optimal Power Flow

Today’s power grid is becoming more diverse and integrated with high-level distributed energy resources and smart control technologies that is creating a new set of grid management challenges in terms of large-scale, nonlinear, and non-convex problem modeling, complex and time-consuming computation, as well as difficult uncertainty handling. This project focused on solving a challenging multi-period security-constrained generation scheduling problem, which is of great importance for maximizing the social welfare of real-time dispatch, day-ahead market, as well as weekly planning of power systems. Our developed software explored parallel optimization algorithms for complex and realistic power system models, and develop fast, efficient, and robust grid optimization solutions on the high-performance computing platform that will enable increased grid economics, flexibility, resilience, as well as energy security in the United States.

24 POWER TRANSMISSION AND DISTRIBUTION↗

BI-LEVEL OPTIMIZATION FOR ELECTRICITY TRANSACTION IN SMART COMMUNITY WITH MODULAR PUMP HYDRO STORAGE

Grid integration of the increasing distributed energy resources could be challenging in terms of new infrastructure investment, power grid stability, etc. To resolve more renewables locally and reduce the need for extensive electricity transmission, a community energy transaction market is assumed with market operator as the leader whose responsibility is to generate local energy prices and clear the energy transaction payment among the prosumers (followers). The leader and multi-followers have competitive objectives of revenue maximization and operational cost minimization. This non-cooperative leader-follower (Stackelberg) game is formulated using a bi-level optimization framework, where a novel modular pump hydro storage technology (GLIDES system) is set as an upper level market operator, and the lower level prosumers are nearby commercial buildings. The best responses of the lower level model could be derived by necessary optimality conditions, and thus the bi-level model could be transformed into single level optimization model via replacing the lower level model by its Karush-Kuhn-Tucker (KKT) necessary conditions. Several experiments have been designed to compare the local energy transaction behavior and profit distribution with the different demand response levels and different local price structures. The experimental results indicate that the lower level prosumers could benefit the most when local buying and selling prices are equal, while maximum revenue potential for the upper level agent could be reached with non-equal trading prices.

Chen, Yang↗

A Hierarchical VLSM-Based Demand Response Strategy for Coordinative Voltage Control between Transmission and Distribution Systems

This paper presents a two-stage hierarchical voltage-load sensitivity matrix (VLSM)-based demand response (DR) algorithm for regulating the voltage at distribution feeders while fulfilling the transmission system DR requests. This allows distributed energy resources be aggregated to provide transmission-level services without causing operational issues in the distribution grids. At the beginning of each operation interval, a transmission system controller will issue DR commands to distribution system controllers. In the first stage, the distribution system controller uses a VLSM-based dispatch algorithm to dispatch the available DR resources on the distribution feeder with an objective to minimize the total voltage deviations at the lowest cost. Then, power flow studies are conducted assuming that the DR commands have been executed. If voltage violations are detected, a second-stage VLSM-based DR dispatch will be performed to remove those violations. After that, the upper and lower DR limits calculated for the next operation interval are sent back to the transmission system controller so the transmission optimization algorithm can use them as operational constraints to make subsequent decisions. The DR resources include smart photovoltaic (PV) inverters that can curtail real power and provide reactive power support, controllable loads and capacitor banks. In conclusion, the IEEE 123-bus test system with 5-minute PV and load data is used to evaluate the performance of the algorithm. In conclusion, simulation results show that the proposed algorithm can fulfill the transmission-level DR requests while maintaining the voltage in the distribution system within limits.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Utilization of Renewable Energy to Meet New National Challenges in Energy and Climate Change

The project aims to design a microgrid system to promote utilization of renewable energy resources such as wind and solar to address the national challenges in energy and climate change. Different optimization techniques and simulation software are used to study the performance of the renewable energy system under study. A series of research works performed under the grant Department of Energy (DOE) is presented. This grant opportunity affords Howard faculty, students, graduates, undergraduates, K-12, postdocs and visiting scholars to benefit state of the art research work. The research work has led to improve or advance understanding of new hardware technologies, software development and engineering optimization methods necessary and sufficient for handling probabilistic models and real-time computation and functions necessary for development of microgrid system. Consistent with State of Project Objective Howard University has partitioned the task into the following integrated activities: 1. Stochastic Model for RER and Load • Development of modeling Renewable Energy Resources (RER) and load which is used to perform distribution power flow study which leads to publication in refereed journals and conferences. The work was also published at the IEEE conference. 2. Stochastic optimization for voltage/Var • The development of voltage VAr optimization based on a review of existing knowledge in optimization led to the use of stochastic program and evolution of programming optimization method for V/VAr optimization. Papers were presented at the North America Power Systems Conference and the IEEE PES general meeting. 3. Modeling RER and Storage • Extending the concept of optimization method an RER with storage, such as the development of microgrid V/VAr and storage is performed. Several papers were published at the North America Power Systems Conference and the IEEE PES general meeting. 4. Power Game • Development of power game experiment using Labvolt to allow for hands on understanding of design and development of microgrid functions is performed. Publication were done by students at the end of their summer program. 5. Designing Microgrid Testbed • Example microgrid test bed is developed. In addition, function of the test bed are developed. The papers were presented at the North America Power Systems Conference and the IEEE general meeting. 6. Outreach Program • From the outreach program, topics from the project have been included in the revision of courses at Howard University, new book called Energy Processing and Smartgrid has being developed. • Hosted masters students from University of Denver to complete their projects with us. • Hosted high school students for early exposure for careers in STEM • Representations made in IEEE conferences to share the lessons learned in the use of micro grid to expose students to STEM education and research.

54 ENVIRONMENTAL SCIENCES↗

Reductive Analysis with Compiler-Guided Large Language Models for Input-Centric Code Optimizations

Input-centric program optimization aims to optimize code by considering the relations between program inputs and program behaviors. Despite its promise, a long-standing barrier for its adoption is the difficulty of automatically identifying critical features of complex inputs. This paper introduces a novel technique, reductive analysis through compiler-guided Large Language Models (LLMs), to solve the problem through a synergy between compilers and LLMs. It uses a reductive approach to overcome the scalability and other limitations of LLMs in program code analysis. The solution, for the first time, automates the identification of critical input features without heavy instrumentation or profiling, cutting the time needed for input identification by 44× (or 450× for local LLMs), reduced from 9.6 hours to 13 minutes (with remote LLMs) or 77 seconds (with local LLMs) on average, making input characterization possible to be integrated into the workflow of program compilations. Optimizations on those identified input features show similar or even better results than those identified by previous profiling-based methods, leading to optimizations that yield 92.6% accuracy in selecting the appropriate adaptive OpenMP parallelization decisions, and 20-30% performance improvement of serverless computing while reducing resource usage by 50-60%.

Input-Centric Optimization↗

Comprehensive Land Use and Environmental Stewardship

Comprehensive Land Use and Environmental Stewardship (CLUES) Report serves as a summary document of the land use and environmental stewardship activities occurring on the Idaho National Laboratory (INL) Site (Desert) and the Research and Education Campus (REC) within Idaho Falls. Land and facility use planning and decisions at the INL Site are guided by a comprehensive planning process in accordance with the United States (U.S.) Department of Energy (DOE) Order 430.1C, “Real Property Asset Management,” which states "Establish a data-driven, risk-informed, performance-based approach to the life-cycle management of real property assets that aligns the real property portfolio with DOE mission needs; acquire, manage, positively account for, and dispose of real property assets in a safe, secure, cost-effective, and sustainable manner; and ensure the real property portfolio is appropriately sized, aligned, and in the proper condition to support efficient mission execution." Land use planning, like Campus Master Planning and 5-Year Facility Planning, provides a means for better, more sustainable use of the INL Site in a coordinated effort to ensure current and future mission needs are met, including acquisition, recapitalization, maintenance, disposition, real property utilization and long-term stewardship. This document and all functions of INL are guided by the DOE Vision for INL and the INL Mission and builds on the baseline established in the FY 2015 CLUES Report. However, it delivers a revised structure with focus on new resource management zones, which provide organization of key management considerations and access restrictions to optimize land use and environmental stewardship. A new set of thematic Guiding Principles presented in this report provide the intent and sustainable management direction to protect the INL Site natural environment. They demonstrate that the mission and vision of DOE and INL can be realized with inclusion of first of a kind technology, private sector development, and globally recognized testing and demonstration. This CLUES Report encourages comprehensive management decision-making with additional resource discussions for sustainability, the built environment, and the ecological landscape at the INL Site. Enhanced resource considerations and trends emphasize the importance of air, land use, environmental, subsurface, and cultural resources. The INL Site supports exceptional and interdependent resources. A diversity of bat species are accommodated by culturally significant caves distributed around the Site. Big game species like elk, deer, and moose seek vegetation communities which are protected by long-term stewardship and monitoring programs.

99 GENERAL AND MISCELLANEOUS↗

Y-12 Groundwater Protection Program Monitoring Optimization Plan For Groundwater Monitoring Wells At The U.S. Department Of Energy Y-12 National Security Complex, Oak Ridge, Tennessee

This document is the monitoring optimization plan for groundwater monitoring wells associated with the U.S. Department of Energy (DOE) Y-12 National Security Complex (Y-12) in Oak Ridge, Tennessee (Figure A.1). The plan describes the technical approach that will be implemented under the Y-12 Groundwater Protection Program (GWPP) to focus available resources on the monitoring wells at Y-12 that provide the most useful hydrologic and groundwater quality monitoring data. The technical approach is based on the GWPP status designation for each well (Section 2.0). Under this approach, wells granted 'active' status are used by the GWPP for hydrologic monitoring and/or groundwater quality sampling (Section 3.0), whereas wells granted 'inactive' status are not used for either purpose. The status designation also defines the frequency at which the GWPP will inspect applicable wells, the scope of these well inspections, and extent of any maintenance actions initiated by the GWPP (Section 3.0). Details regarding the ancillary activities associated with implementation of this plan (e.g., well inspection) are deferred to the referenced GWPP plans and procedures (Section 4.0). This plan applies to groundwater wells associated with Y-12 and related waste management areas and facilities located within three hydrogeologic regimes (Figure A.1): the Bear Creek Hydrogeologic Regime (Bear Creek Regime), the Upper East Fork Poplar Creek Hydrogeologic Regime (East Fork Regime), and the Chestnut Ridge Hydrogeologic Regime (Chestnut Ridge Regime). The Bear Creek Regime encompasses a section of Bear Creek Valley (BCV) immediately west of Y-12. The East Fork Regime encompasses most of the Y-12 process, operations, and support facilities in BCV and, for the purposes of this plan, includes a section of Union Valley east of the DOE Oak Ridge Reservation (ORR) boundary along Scarboro Road. The Chestnut Ridge Regime encompasses a section of Chestnut Ridge directly south of Y-12 that is bound on the west by a surface drainage feature (Dunaway Branch) and on the east by Scarboro Road. For this plan, the Chestnut Ridge Regime includes an area known as the South Campus Facility that is located west of Scarboro Road and south of Bethel Valley Road. The GWPP maintains an extensive database of construction details and related information for the monitoring wells in each hydrogeologic regime (including wells that have been destroyed or intentionally plugged and abandoned); the most recent hardcopy version of the database was issued in February 2003 (BWXT Y-12, L.L.C. [BWXT] 2003). As specified in the Y-12 GWPP Management Plan (Babcock & Wilcox Technical Services Y-12, LLC [B&W Y-12] 2009a), this plan will be reviewed and updated every three years.

54 ENVIRONMENTAL SCIENCES↗

Voltage Optimization

Voltage optimization refers to a volt-var optimization technique which was originally designed to minimize energy consumption and improve end-use efficiency on the distribution system. This reduces source voltage at a substation, which lowers generation demand, reduces system losses, and improves system stability. This has been the conventional implementation of voltage optimization: focusing on regulation of voltages throughout the distribution system via coordinated adjustments of load tap changers, line-voltage regulators, switched shunt compensation, and the targeted placement of new shunt compensation. These methods have received increased focus following a growth in distributed-energy resources (DERs) interconnecting at the distribution level, given their developing capability to participate in voltage regulation. This is paired with the intermittency challenges they pose, load imbalances, and voltage fluctuations. Voltage optimization has long been an area of extensive research within the distribution system, with various strategies and philosophies demonstrating a range of effects. The conventional focus on the distribution system owes to the typical operation of transmission systems, which assumes firm local resources and flows strictly from generation resources to loads. The strategies employed in distribution have also been optimized for this typically radial configuration and operation of the distribution system.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Distributed MPC for Efficient Coordination of Storage and Renewable Energy Sources Across Control Areas

In electric power systems, multiple entities are responsible for ensuring an economic and reliable way of delivering power from producers to consumers. With the increase of variable renewable generation it is becoming increasingly important to take advantage of the individual entities' (and their areas') capabilities for balancing variability. Hence, in this paper, we employ and extend the approximate Newton directions method to optimally coordinate control areas leveraging storage available in one area to balance variable resources in another area with only minimal information exchange among the areas. The problem to be decomposed is a model predictive control problem including generation constraints, energy storage constraints, and AC power flow constraints. Singularity issues encountered when formulating the respective Newton-Raphson steps due to intertemporal constraints are addressed and extensions to the original decomposition method are proposed to improve the convergence rate and required communication of the method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimal Power Flow Pursuit

Past works that focused on addressing power-quality and reliability concerns related to renewable energy resources (RESs) operating with business-as-usual practices have looked at the design of Volt/VAr and Volt/Watt strategies to regulate real or reactive powers based on local voltage measurements, so that terminal voltages are within acceptable levels. These control strategies have the potential of operating at the same time scale of distribution-system dynamics, and can therefore mitigate disturbances precipitated fast time-varying loads and ambient conditions; however, they do not necessarily guarantee system-level optimality, and stability claims are mainly based on empirical evidences. On a different time scale, centralized and distributed optimal power flow (OPF) algorithms have been proposed to compute optimal steady-state inverter setpoints, so that power losses and voltage deviations are minimized and economic benefits to end-users providing ancillary services are maximized. However, traditional OPF schemes may offer decision making capabilities that do not match the dynamics of distribution systems. Particularly, during the time required to collect data from all the nodes of the network (e.g., loads), solve the OPF, and subsequently dispatch setpoints, the underlying load, ambient, and network conditions may have already changed; in this case, the DER output powers would be consistently regulated around outdated setpoints, leading to suboptimal system operation and violation of relevant electrical limits. The present work focuses on the synthesis of distributed RES-inverter controllers that leverage the opportunities for fast feedback offered by power-electronics interfaced RESs. The overarching objective is to bridge the temporal gap between long-term system optimization and real-time control, to enable seamless RES integration in large scale with stability and efficiency guarantees, while congruently pursuing system-level optimization objectives. The design of the control framework is based on suitable linear approximations of the AC power-flow equations as well as Lagrangian regularization methods. The proposed controllers enable an update of the power outputs at a time scale that is compatible with the underlying dynamics of loads and ambient conditions, and continuously drive the system operation towards OPF-based solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

On the Rapid Calculation of Binding Affinities for Antigen and Antibody Design and Affinity Maturation Simulations

The accurate and efficient calculation of protein-protein binding affinities is an essential component in antibody and antigen design and optimization, and in computer modeling of antibody affinity maturation. Such calculations remain challenging despite advances in computer hardware and algorithms, primarily because proteins are flexible molecules, and thus, require explicit or implicit incorporation of multiple conformational states into the computational procedure. The astronomical size of the amino acid sequence space further compounds the challenge by requiring predictions to be computed within a short time so that many sequence variants can be tested. In this study, we compare three classes of methods for antibody/antigen (Ab/Ag) binding affinity calculations: (i) a method that relies on the physical separation of the Ab/Ag complex in equilibrium molecular dynamics (MD) simulations, (ii) a collection of 18 scoring functions that act on an ensemble of structures created using homology modeling software, and (iii) methods based on the molecular mechanics-generalized Born surface area (MM-GBSA) energy decomposition, in which the individual contributions of the energy terms are scaled to optimize agreement with the experiment. When applied to a set of 49 antibody mutations in two Ab/HIV gp120 complexes, all of the methods are found to have modest accuracy, with the highest Pearson correlations reaching about 0.6. In particular, the most computationally intensive method, i.e., MD simulation, did not outperform several scoring functions. The optimized energy decomposition methods provided marginally higher accuracy, but at the expense of requiring experimental data for parametrization. Within each method class, we examined the effect of the number of independent computational replicates, i.e., modeled structures or reinitialized MD simulations, on the prediction accuracy. We suggest using about ten modeled structures for scoring methods, and about five simulation replicates for MD simulations as a rule of thumb for obtaining reasonable convergence. We anticipate that our study will be a useful resource for practitioners working to incorporate binding affinity calculations within their protein design and optimization process.

59 BASIC BIOLOGICAL SCIENCES↗

An Edge-Cloud Integrated Solution for Buildings Demand Response Using Reinforcement Learning

Buildings, as major energy consumers, can provide great untapped demand response (DR) resources for grid services. However, their participation remains low in real-life. One major impediment for popularizing DR in buildings is the lack of cost-effective automation systems that can be widely adopted. Existing optimization-based smart building control algorithms suffer from high costs on both building-specific modeling and on-demand computing resources. To tackle these issues, this paper proposes a cost-effective edge-cloud integrated solution using reinforcement learning (RL). Beside RL’s ability to solve sequential optimal decision-making problems, its adaptability to easy-to-obtain building models and the off-line learning feature are likely to reduce the controller’s implementation cost. Using a surrogate building model learned automatically from building operation data, an RL agent learns an optimal control policy on cloud infrastructure, and the policy is then distributed to edge devices for execution. Simulation results demonstrate the control efficacy and the learning efficiency in buildings of different sizes. A preliminary cost analysis on a 4-zone commercial building shows the annual cost for optimal policy training is only 2.25% of the DR incentive received. Results of this study show a possible approach with higher return on investment for buildings to participate in DR programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗