SEARCH · Engineering Papers

Results for “online”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

All sources NASA NTRS DOE OSTI

DOE OSTI2026

A transfer learning approach to energy-efficient control of small and medium-sized commercial buildings

Model-free reinforcement learning (RL) provides a data-driven and adaptive approach to optimize building energy use while satisfying occupant comfort. This powerful tool does not need any prior knowledge about the environment and system it is optimizing and can adapt its policy based on the changes in captures. Like any other data-driven tool, it faces high training costs due to the extensive agent-environment interactions required to capture long-term building dynamics and user comfort. Transfer learning, particularly policy distillation, offers a promising way to accelerate training by leveraging pretrained RL agents in different building and system types. Here, this study investigates online student distillation, in which the student model updates its neural network weights using outputs from teacher models. The work introduces a student distillation strategy designed for efficient knowledge transfer, along with a teacher selection method that ensures high-quality guidance. The approach is validated using a highly calibrated whole building energy model for a small/medium commercial building test facility. Results show substantial reductions in training time and data requirements while surpassing the performance of ASHRAE Guideline 36, an advanced rule-based control strategy. The distilled RL model required 45% less data and achieved 20% higher cumulative rewards than a state-of-the-art RL model, with faster convergence and lower energy consumption. These outcomes demonstrate that effective transfer learning enables a scalable and data-efficient energy management solution for commercial buildings.

ASHRAE guideline 36↗

DOE OSTI2022

Reinforcement Learning for feedback-enabled cyber resilience

The rapid growth in the number of devices and their connectivity has enlarged the attack surface and made cyber systems more vulnerable. As attackers become increasingly sophisticated and resourceful, mere reliance on traditional cyber protection, such as intrusion detection, firewalls, and encryption, is insufficient to secure the cyber systems. Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and uncertainties in real-time and strategically responds to them to maintain the critical functions of the cyber systems in the event of successful attacks. Feedback architectures play a pivotal role in enabling the online sensing, reasoning, and actuation process of the CRM. Reinforcement Learning (RL) is an important gathering of algorithms that epitomize the feedback architectures for cyber resilience. It allows the CRM to provide dynamic and sequential responses to attacks with limited or without prior knowledge of the environment and the attacker. In this work, we review the literature on RL for cyber resilience and discuss the cyber-resilient defenses against three major types of vulnerabilities, i.e., posture-related, information-related, and human-related vulnerabilities. Here we introduce moving target defense, defensive cyber deception, and assistive human security technologies as three application domains of CRMs to elaborate on their designs. The RL algorithms also have vulnerabilities themselves. We explain the major vulnerabilities of RL and present develop several attack models where the attacker target the information exchanged between the environment and the agent: the rewards, the state observations, and the action commands. We show that the attacker can trick the RL agent into learning a nefarious policy with minimum attacking effort. The paper introduces several defense methods to secure the RL-enabled systems from these attacks. However, there is still a lack of works that focuses on the defensive mechanisms for RL-enabled systems. Last but not least, we discuss the future challenges of RL for cyber security and resilience and emerging applications of RL-based CRMs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Continuous culture of anaerobic fungi enables growth and metabolic flux tuning without use of genetic tools

Anaerobic gut fungi (AGF) have potential to valorize lignocellulosic biomass owing to their diverse repertoire of carbohydrate-active enzymes (CAZymes). However, AGF metabolism is poorly understood, and no stable genetic tools are available to manipulate growth and metabolic flux to enhance production of specific targets, e.g., cells, CAZymes, or metabolites. Herein, a cost-effective, Arduino-based, continuous-flow anaerobic bioreactor with online optical density control is presented to probe metabolism and predictably tune fluxes in Caecomyces churrovis. Varying the C. churrovis turbidostat setpoint titer reliably controlled growth rate (from 0.04 to 0.20 h −1 ), metabolic flux, and production rates of acetate, formate, lactate, and ethanol. Bioreactor setpoints to maximize production of each product were identified, and all continuous production rates significantly exceed batch rates. Formate spike-ins increased lactate flux and decreased acetate, ethanol, and formate fluxes. The bioreactor and turbidostat culture schemes demonstrated here offer tools to tailor AGF fermentations to application-specific hydrolysate product profiles.

Agriculture↗

DOE OSTI2021

The future low-temperature geochemical data-scape as envisioned by the U.S. geochemical community

Data sharing benefits the researcher, the scientific community, and the public by allowing the impact of data to be generalized beyond one project and by making science more transparent. However, many scientific communities have not developed protocols or standards for publishing, citing, and versioning datasets. One community that lags in data management is that of low-temperature geochemistry (LTG). This paper resulted from an initiative from 2018 through 2020 to convene LTG and data scientists in the U.S. to strategize future management of LTG data. Through webinars, a workshop, a preprint, a townhall, and a community survey, the group of U.S. scientists discussed the landscape of data management for LTG – the data-scape. Currently this data-scape includes a “street bazaar” of data repositories. This was deemed appropriate in the same way that LTG scientists publish articles in many journals. The variety of data repositories and journals reflect that LTG scientists target many different scientific questions, produce data with extremely different structures and volumes, and utilize copious and complex metadata. Nonetheless, the group agreed that publication of LTG science must be accompanied by sharing of data in publicly accessible repositories, and, for sample-based data, registration of samples with globally unique persistent identifiers. LTG scientists should use certified data repositories that are either highly structured databases designed for specialized types of data, or unstructured generalized data systems. Recognizing the need for tools to enable search and cross-referencing across the proliferating data repositories, the group proposed that the overall data informatics paradigm in LTG should shift from “build data repository, data will come” to “publish data online, cybertools will find”. Funding agencies could also provide portals for LTG scientists to register funded projects and datasets, and forge approaches that cross national boundaries. Finally, the needed transformation of the LTG data culture requires emphasis in student education on science and management of data.

58 GEOSCIENCES↗

DOE OSTI2021

Multiscale CFD simulation of biomass fast pyrolysis with a machine learning derived intra-particle model and detailed pyrolysis kinetics

Coupling particle and reactor scale models is as essential as reactor fluid dynamics and particle motion for accurate Computational Fluid Dynamic (CFD) simulations of biomass fast pyrolysis reactors due to intraparticle heat transfer and chemical reactions controlling conversion time and product distributions. Direct online coupling of a particle model with a reactor model is computationally expensive, while offline coupling is case-dependent. In this research, solutions from a series of particle pyrolysis simulations were regressed with Artificial Neural Network (ANN). Furthermoer, this machine learning-derived model predicted the same temperature and conversion profiles compared with particle resolved simulation while the isothermal approach overpredicted the temperature by 130 K and underpredicted the conversion time by 30 s. The ANN model was then integrated into CFD simulations of fluidized bed biomass fast pyrolysis with varied feedstocks via coupling PyTorch and MFiX. The averaged error of simulation predicted bio-oil yields with four feedstocks is 6.4%. This multi-scale approach provides an efficient tool for the coupled particle and reactor scale simulations of biomass pyrolysis.

09 BIOMASS FUELS↗

DOE OSTI2022

Importance of residence-time control of industrial screw-conveying reactors: Application to dilute-acid hydrolysis of biomass

Horizontal screw reactors are utilized in biorefineries for acid-catalyzed hydrolysis of xylan, which is a multi-step chemical reaction requiring accurate residence-time control. However, it is difficult to obtain online analytical measurement of reactant species. In this work, a residence-time distribution (RTD) is exhibited whose characteristics influence species yields. Sensitivity of product yield to RTD was investigated to understand the relative importance of operating control vs. inherent reactor dispersion. We find that reactor operation using a commonly used theoretical residence-time relationship can result in substantial yield losses. Instead, a model that accounts for the actual reactor RTD provides much improved results. The dispersion caused by reactor conditions only slightly hinders achieving theoretical optimal xylose yield (less than 3% yield loss for coefficient of variation less than 0.35), provided a validated RTD model is used to target the desired mean residence-time. In contrast, neglecting to account for the RTD by using the simplistic theoretical calculation results in xylose yields that are as much as 16% lower than the theoretical maximum.

09 BIOMASS FUELS↗

DOE OSTI2023

Discovery of multi-functional polyimides through high-throughput screening using explainable machine learning

Polyimides have been widely used in modern industries because of their excellent mechanical and thermal properties, e.g., high-temperature fuel cells, displays, and aerospace composites. However, it usually takes decades of experimental efforts to develop a successful product. Aiming to expedite the discovery of high-performance polyimides, we utilize computational methods of machine learning (ML) and molecular dynamics (MD) simulations. Our study provides compelling evidence for the effectiveness of a data-driven approach in discovering novel polyimides. We first build a comprehensive library of more than 8 million hypothetical polyimides based on the polycondensation of existing dianhydride and diamine/diisocyanate molecules. Then we establish multiple ML models for the thermal and mechanical properties of polyimides based on their experimentally reported values, including glass transition temperature, Young’s modulus, and tensile yield strength. The obtained ML models demonstrate excellent predictive performance in identifying the key chemical substructures influencing the thermal and mechanical properties of polyimides. The use of explainable machine learning describes the effect of chemical substructures on individual properties, from which human experts can understand the cause of the ML model decision. Applying the well-trained ML models, we obtain property predictions of the 8 million hypothetical polyimides. Then, we screen the whole hypothetical dataset and identify three (3) best-performing novel polyimides that have better-combined properties than existing ones through Pareto frontier analysis. For an easy query of the discovered high-performing polyimides, we also create an online platform https://polyimide-explorer.herokuapp.com/ that embeds the developed ML model with interactive visualization. Furthermore, we validate the ML predictions through all-atom MD simulations and examine their synthesizability. The MD simulations are in good agreement with the ML predictions and the three novel polyimides are predicted to be easy to synthesize via Schuffenhauer’s synthetic accessibility score. Following the proposed ML guidance, we successfully synthesized a novel polyimide and the experimentally obtained high glass transition/thermal decomposition temperature demonstrated its excellent thermal stability. Here our study demonstrates an efficient way to expedite the discovery of novel polymers using ML prediction and MD validation. The high-throughput screening of a large computational dataset can serve as a general approach for new material discovery in other polymeric material exploration problems, such as organic photovoltaics, polymer membranes, and dielectrics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Probabilistic-learning-based stochastic surrogate model from small incomplete datasets for nonlinear dynamical systems

We consider a high-dimensional nonlinear computational model of a dynamical system, parameterized by a vector-valued control parameter, in the presence of uncertainties represented by an uncontrolled parameter modeled by a vector-valued random variable, and possibly with stochastic excitation. The objective is to construct a statistical surrogate model where the input is any deterministic value of the control parameter, and the output is a vector-valued observation of the computational model, which is a random vector whose probability measure is updated using a target dataset. To construct this statistical surrogate model, the stochastic response of the computational model must be built, which is a vector-valued time-discretized stochastic process in high dimension, depending on the control parameter. It is assumed that the computational cost of a single evaluation of the deterministic model is high. For the probabilistic updating, we consider a subset of the components of the observation of the computational model, defined as the “identification observation” of the computational model, for which a small target dataset is available. Therefore, the target dataset is associated with partial observability, corresponding to an incomplete data case. Given a prior probability model of the random control and uncontrolled parameters, a training dataset is constructed, consisting of realizations of the random triplet composed of the stochastic response, the random identification observation, and the random control parameter. Since the computational cost of a single evaluation of the deterministic model is assumed to be large, the training dataset is also of small size. The main challenges in this problem are the high dimensionality, partial observability leading to incomplete data in the target dataset for the identification observation of the computational model (which is not sufficient to identify the computational stochastic responses), and the availability of a small training dataset. To address these challenges, we propose a methodology based on statistical methods for constructing necessary reduced representations, direct probabilistic learning under constraints using probabilistic learning on manifolds (PLoM) constrained by the target dataset, and the use of a weak formulation of the Fourier transform of probability measures. Statistical conditioning is also employed to explore the learned dataset. The constructed predictive statistical surrogate model can be implemented in the context of online computation. Here, we apply this approach to a problem of nonlinear stochastic dynamics in high dimensions within the framework of deformable solids mechanics.

Engineering↗

DOE OSTI2024

Dynamic flux surrogate-based partitioned methods for interface problems

Loosely coupled partitioned methods for multiphysics problems treat each subproblem as a separate entity and advance them independently in time. In so doing these methods enable code reuse, increase concurrency and provide a convenient framework for plug-and-play multiphysics simulations. However, mathematically loosely coupled schemes are equivalent to a single step of an iterative solution method, which can compromise their accuracy and stability. We present a new data-driven partitioned method for coupled parametric PDEs that can improve upon the accuracy of traditional loosely coupled methods without incurring a performance penalty. To that end, we replace conventional field transfers across the interface by a surrogate for the dynamics of the interface flux exchanged between the subdomains. To develop this surrogate we apply dynamic mode decomposition to a non-standard staggered-in-time state, comprising the interface flux and small solution patches near the interface. The new approach shifts the main computational burden to an offline training phase, whereas application of the surrogate in the online phase amounts to a single matrix–vector multiplication. In conclusion, we provide stability analysis of the surrogate-based partitioned scheme and include numerical results that demonstrate its potential.

Dynamic mode decomposition (DMD)↗

DOE OSTI2024

Development of algorithms for augmenting and replacing conventional process control using reinforcement learning

Here, this work seeks to allow for the online operation and training of model-free reinforcement learning (RL) agents but limit the risk to system equipment and personnel. The parallel implementation of RL alongside more conventional process control (CPC) allows for the RL algorithm to learn from CPC. The past performance of both methods are assessed on a continuous basis allowing for a transition from CPC to RL and, if needed, transitioning back to CPC from RL. This allows for the RL algorithm to slowly and safely assume control of the process without significant degradation in control performance. It is shown that the RL can derive a near optimal policy even when coupled with a suboptimal CPC. It is also demonstrated that the coupled RL-CPC algorithm learns at a faster rate than traditional RL methods of exploration while the algorithm’s performance does not deteriorate below CPC, even when exposed to an unknown operating condition.

30 DIRECT ENERGY CONVERSION↗

DOE OSTI2022

Aggregate attack surface management for network discovery of operational technology

Interconnectivity has become a substratum of technology as the benefits of data-driven functionality are being realized in nearly all industries. Increased connectivity of Operational Technology (OT) exacerbates cyber risks because Industrial Control Systems (ICS) are becoming exposed to the Internet. These exposures are often done inadvertently through misconfigurations as additional network devices come online. Attack surface management (ASM) platforms can be used to identify vulnerabilities by performing external network discovery over the Internet using web spiders. These web spiders enable big data analytics of Internet of Things (IoT) devices as identifiable information of Internet-exposed equipment are archived in searchable databases that are made publicly available. There are a multitude of ASM service providers on the market. Here, this study was conducted to evaluate several commonly known tools to determine the aggregate attack surface of control systems. Queries were crafted by targeting commonly known manufacturers and communication protocols found in OT networks. Identified devices were that categorized based on technology types. Each query was replicated between several tools to target identical ICS equipment. Findings in this paper suggested a significant variance in the exposures discovered by each tool, but unique contributions were identified for each tool when a merged attack surface was derived. Therefore, all tools should be used in aggregate.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Machine learning in nuclear materials research

Nuclear materials are often demanded to function for extended time in extreme environments, including high radiation fluxes with associated transmutations, high temperature and temperature gradients, mechanical stresses, and corrosive coolants. They also have a wide range of microstructural and chemical makeups, resulting in multifaceted and often out-of-equilibrium interactions. Machine learning (ML) is increasingly being used to tackle these complex time-dependent interactions and aid researchers in developing models and making predictions, sometimes with better accuracy than traditional modeling that focuses on one or two parameters at a time. Conventional practices of acquiring new experimental data in nuclear materials research are often slow and expensive, limiting the opportunity for data-centric ML, but new methods are changing that paradigm. Here we review high-throughput computational and experimental data approaches, especially robotic experimentation and active learning that is based on Gaussian process and Bayesian optimization. We show ML examples in structural materials (e.g., reactor pressure vessel (RPV) alloys and radiation detecting scintillating materials) and highlight new techniques of high-throughput sample preparation and characterizations, and automated radiation/environmental exposures and real-time online diagnostics. Herein, this review suggests that ML models of material constitutive relations in plasticity, damage, and even electronic and optical responses to radiation are likely to become powerful tools as they develop. Finally, we speculate on how the recent trends of using natural language processing (NLP) to aid the collection and analysis of literature data, interpretable artificial intelligence (AI), and the use of streamlined scripting, database, workflow management, and cloud computing platforms that will soon make the utilization of ML techniques as commonplace as the spreadsheet curve-fitting practices of today.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Field surveying data of low-cost networked flood sensors in southeast Texas

Floods are common natural disasters worldwide and pose substantial risks to life, property, food production, and natural resources. Effective measures for flood mitigation and warning are essential. Southeast Texas is still at significant risk of flooding, and Lamar University is assisting the region with asset management of a flood sensor network for flooding events. This network provides real-time water stage information. Lamar University developed a survey program to measure elevation and coordinates at each sensor site location to make this data more useful for flood monitoring and mapping. This paper overviews the measurement of the elevation and coordinates of 74 networked flood sensors and various flood stage thresholds at critical points that flood decision-makers can use for reference at each site. In the first phase of this program, these sensors were deployed throughout a 7-county region spanning nearly 6,000 square miles in Southeast Texas. The latitude and longitude of the sensors and their elevations were determined using survey-grade Global Navigation Satellite System (GNSS) technology. Various Continually Operating Reference Stations (CORS) were utilized for post-processing to achieve sub-inch resolution. The flood stage thresholds, water level sensors elevation, and the elevations and positions of other critical surrounding points are viewable to the public through two online repositories and a web-based sensor management dashboard. The data is used to aid with decisions related to road closures or modeling efforts by mitigation decision-makers, emergency managers, and the public, including the Texas Department of Transportation, Houston Transtar, the National Weather Service, and the Sabine River Authority of Texas (SRA).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Dataset describing two reference models for full-spectral lighting and daylight simulations together with implementations for two software systems

A dataset of two spectral lighting simulation reference models - one office and one factory hall - is presented. It aims to demonstrate and support full-spectral daylight and electric lighting simulations and facilitate evaluation of non-visual effects of light. The dataset includes Rhino CAD geometry, comprehensive spectral material and light source data and window system BSDF data. Example implementations in the two software tools, Radiance and OWL, enable reproducible workflows and support adoption in other software. The dataset is openly available on Zenodo. The office model reproduces Room 518 at the University of Innsbruck, including a west-facing façade and interior furnishings. The factory hall model follows the proposed geometry in the European standard 15193 for building energy performance. Interior reflectances in the office were measured in-situ using a handheld spectrometer. Exterior spectra and factory hall materials matching specified reflectances were obtained from an online spectral materials database. Glazing transmittance was derived from IGDB data using LBNL Optics/WINDOW. BSDFs for venetian blinds at various tilt angles, and for a diffusing pane adapted from the Complex Glazing Database, were generated in WINDOW. Luminaires in both models are specified with photometric files (Eulumdat/IES) and lamp spectra (Fluorescent 840, 4000 K LED). The provided example implementations (Radiance, OWL) include prepared input data and scripts to run first spectral simulations; example results are also included. The dataset is prepared to support reuse by researchers, designers and software developers for method validation, software engineering and comparison, and development of spectral metrics and controls.

Geisler-Moroder, David↗

DOE OSTI2021

A reporting format for field measurements of soil respiration

Field observations of the soil-to-atmosphere CO2 flux–soil respiration, RS–are a prime example of ‘long tail’ data that historically have had neither centralized databases nor an agreed-upon reporting format. This has hindered scientific transparency, analytical reproducibility, and novel syntheses with respect to this globally-important component of the carbon cycle. Here we propose a new data and metadata reporting format for RS data, based on engagement with a wide range of researchers in the field as well as expert advisory panels. Our goal was a reporting format that would be relevant and useful for synthesis activities, and optimizing data discoverability and usability while not placing an undue burden on data contributors. We describe previous RS data collection efforts, lessons learned from related databases and data-oriented networks (e.g. FLUXNET) in earth and ecological sciences, and the process of community consultation. The proposed reporting format focuses on chamber-level data and metadata, specifying measurement conditions and, for a given measurement period defined by beginning and ending timestamps, a mean RS flux (or CO2 concentration) and associated ancillary measurements. Fundamentally, this format aims to enable findable, accessible, interoperable, and reusable data, while providing ‘future-proofing’ capabilities to support reanalyses using as yet unknown algorithms or approaches. Finally, this proposed RS reporting format is available online, and is intended to be a dynamic document, subject to further community feedback and/or change in the future.

Bond-Lamberty, Benjamin↗

DOE OSTI2022

Best practice reporting guideline for building stock energy models

Buildings are responsible for 38% of global greenhouse gas (GHG) emissions and, therefore, pathways to reduce their impact are crucial to achieve climate targets. Building stock energy models (BSEMs) have long been used as a tool to assess the current and future energy demand and environmental impact of building stocks. BSEMs have become more and more complex and are often tailored to case-specific datasets, which results in a high degree of heterogeneity among models. This heterogeneity, together with a lack of consistency in the reporting hinders the understanding of these models and, thereby, an accurate interpretation and comparison of results. In this paper we present a reporting guideline in order to improve reporting practices of BSEMs. The guideline was developed by experts as part of the IEA's Annex 70 and builds upon reporting guidelines from other fields. It consists of five topics (Overview, Model Components, Input and Output, Quality Assurance and Additional Information), which are further subdivided into subtopics. We explain which model aspects should be described in each subtopic, and provide illustrative examples on how to apply the guideline. In closing, the reporting guideline is consistent with the model classification framework and online model registry also developed in the Annex.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2022

Sharing is caring: An extensive analysis of parameter-based transfer learning for the prediction of building thermal dynamics

In recent years deep neural networks have been proposed as a lightweight data-driven model to capture high-dimensional, nonlinear physical processes to predict building thermal responses. However, the need of a large amount of data for the training process of deep neural networks clashes with the potential limited data availability in most existing or new buildings. Transfer learning aims to enhance the performance of a target learner exploiting knowledge from related and similar environments. This study conducted a suite of experiments that leveraged 250 data-driven models based on a synthetic dataset of a building archetype to study the influence of data availability, energy efficiency level, occupancy and climate for the transfer process of thermal dynamics. The performance of the transfer learning process was compared against a classical machine learning approach. Here, the results suggest that building thermal dynamics can be effectively transferred under the same climatic conditions, increasing performance when dealing with different occupancy schedules, efficiency levels and low data availability. Furthermore, the paper compares the performance of both transfer learning and machine learning approaches in an online fashion, to support the implementation in real-world deployment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2023

How good are learning-based control v.s. model-based control for load shifting? Investigations on a single zone building energy system

Both model predictive control (MPC) and deep reinforcement learning control (DRL) have been presented as a way to approximate the true optimality of a dynamic programming problem, and these two have shown significant operational cost saving potentials for building energy systems. Furthermore, there is still a lack of in-depth quantitative studies on their approximation levels to the true optimality, especially in the building energy domain. To fill in the gap, this paper provides a numerical framework that enables the evaluation of the optimality levels of different controllers for building energy systems. This framework is then used to comprehensively compare the optimal control performance of both MPC and DRL controllers with given computation budgets for a single zone fan coil unit system. Note the optimality is estimated based on a user-specific selection of trade-off weights among energy costs, thermal comfort and control slew rates. Compared with the best optimality we can find through expensive optimization simulations, the best DRL agent can maximally approximate the optimality by 96.54%, which outperforms the best MPC whose optimality level is 90.11%. However, due to the stochasticity, the DRL agent is only expected to approximate the optimality by 90.42%, which is almost equivalent to the best MPC. Except for Proximal Policy Optimization (PPO), all DRL agents can have a better approximation to the optimality than the best MPC, and are expected to have better approximation than the MPC with a prediction horizon of 32 steps (15 min per step). In terms of reducing energy cost and thermal discomfort, MPC can outperform the rule-based control (RBC) by 18.47%–25.44%. DRL can be expected to outperform RBC by 18.95%–25.65% ,and the best DRL control policy can outperform RBC by 20.29%–29.72%. Although the comparison of the optimality level is performed in a perfect setting, e.g., MPC assumes perfect models, and DRL assumes a perfect offline training process and online deployment process, this can shed insight on their capabilities of approximating to the original dynamic programming problem.

24 POWER TRANSMISSION AND DISTRIBUTION↗