Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance Tuning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

On the Feasibility of Using Reduced-Precision Tensor Core Operations for Graph Analytics

Today’s data-driven analytics and machine learning workload have been largely driven by the General-PurposeGraphics Processing Units (GPGPUs). To accelerate dense matrix multiplications on the GPUs, Tensor Core Units (TCUs) have been introduced in recent years. In this paper, we study linear-algebra-based and vertex-centric algorithms for various graph kernels on the GPUs with an objective of applying this new hardware feature to graph applications. We identify the potential stages in these graph kernels that can be executed on the Tensor Core Units. In particular, we leverage the reformulation of the reduction and scan operations in terms of matrix multiplication [1]on the TCUs. We demonstrate that executing these operations on the TCUs, available inside different graph kernels, can assist in establishing an end-to-end pipeline on the GPGPUs without depending on hand-tuned external libraries and still can deliver comparable performance for various graph analytics.

Graph algorithms, GPU computing↗
DOE OSTI2022

Discovering and Demonstrating a Novel High-Performing 2D-Patterned Electrode for Proton-Exchange Membrane Water Electrolysis Devices

Proton-exchange membrane water electrolysis (PEMWE) produces hydrogen with high efficiency and purity but uses high-loading platinum-group metal (PGM) catalysts. Such concerns call for the development of novel electrode architectures to improve catalyst utilization and mass activity, thus promoting PEMWE cost competitiveness for large-scale implementation. In this study, we demonstrated, for the first time, a novel two-dimensional (2D)-patterned electrode with edge effects to address these challenges. The edge effect was induced by membrane properties, potential distribution, and counter electrode coverage and could be optimized by tuning the catalyst layer dimensions. To achieve identical PEMWE performance, the optimal pattern saved the 21% anode PGM catalyst compared with the conventional catalyst fully covered electrode. The PGM catalyst could be further reduced by 61% to boost mass activity with no significant performance loss. The results also indicated that the electrode uniformity in PEMWE cells might not be as critical as that in PEM fuel cells. Finally, the novel 2D-patterned electrode could effectively reduce PGM catalyst loading, accelerating affordable and large-scale production of hydrogen and other value-added chemicals via electrolysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Advanced Pt-Based Core–Shell Electrocatalysts for Fuel Cell Cathodes

Proton-exchange membrane fuel cells (PEMFCs) are highly efficient energy storage and conversion devices. Thus, the platinum group metal (PGM)-based catalysts which are the dominant choice for the PEMFCs have received extensive interest during the past couple of decades. However, the drawbacks in the existing PGM-based catalysts (i.e., high cost, slow kinetics, poor stability, etc.) still limit their applications in fuel cells. The Pt-based core–shell catalysts potentially alleviate these issues through the low Pt loading with the associated low cost and the high corrosion resistance and further improve the oxygen reduction reaction’s (ORR’s) activity and stability. This Account focuses on the synthetic strategies, catalytic mechanisms, factors influencing enhanced ORR performance, and applications in PEMFCs for the Pt-based core–shell catalysts. We first highlight the synthetic strategies for Pt-based core–shell catalysts including the galvanic displacement of an underpotentially deposited non-noble metal monolayer, thermal annealing, and dealloying methods, which can be scaled-up to meet the requirements of fuel cell operations. Subsequently, catalytic mechanisms such as the self-healing mechanism in the Pt monolayer on Pd core catalysts, the pinning effect of nitrogen (N) dopants in N-doped PtNi core–shell catalysts, and the ligand effect of the ordered intermetallic structure in L10-Pt/CoPt core–shell catalysts and their synergistic effects in N-doped L10-PtNi catalysts are described in detail. Additionally, the core–shell structure in the Pt-based catalysts have two main effects for enhanced ORR performance: (i) the interaction between Pt shells and core substrates can tune the electronic state of the surface Pt, thus boosting the ORR activity and stability, and (ii) the outer Pt shell with modest thickness can enhance the oxidation and dissolution resistance of the core, resulting in improved durability. We then review the recent attempts to optimize the ORR performance of the Pt-based core–shell catalysts by considering the shape, composition, surface orientation, and shell thickness. The factors influencing the ORR performance can be grouped into two categories: the effect of the core and the effect of the shell. In the former, PtM core–shell catalysts which use different non-PGM element cores (M) are summarized, and in the latter, Pt-based core–shell catalysts with different shell structures and compositions are described. The modifications of the core and/or shell structure can not only optimize the intermediate-binding energetics on the Pt surface through tuning the strain of the surface Pt, which increases the intrinsic activity and stability, but also offer a significantly decreased catalyst cost. Finally, we discuss the membrane electrode assembly performance of Pt-based core–shell catalysts in fuel cell cathodes and evaluate their potential in real PEMFCs for light-duty and heavy-duty vehicle applications. Even though some challenges to the activity and lifetime in the fuel cells remain, the Pt-based core–shell catalysts are expected to be promising for many practical PEMFC applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Revolutionizing Porous Liquids: Stabilization and Structural Engineering Achieved by a Surface Deposition Strategy

Facile approaches capable of constructing stable and structurally diverse porous liquids (PLs) that can deliver high-performance applications are a long-standing, captivating, and challenging research area that requires significant attention. Here, in this work, a facile surface deposition strategy is demonstrated to afford diverse type III-PLs possessing ultra-stable dispersion, external structure modification, and enhanced performance in gas storage and transformation by leveraging the expeditious and uniform precipitation of selected metal salts. The Ag(I) species-modified zeolite nanosheets are deployed as the porous host to construct type III-PLs with ionic liquids (ILs) containing bromide anion , leading to stable dispersion driven by the formation of AgBr nanoparticles. The as-afforded type-III PLs display promising performance in CO 2 capture/conversion and ethylene/ethane separation. Property and performance of the as-produced PLs can be tuned by the cation structure of the ILs, which can be harnessed to achieve polarity reversal of the porous host via ionic exchange. The surface deposition procedure can be further extended to produce PLs from Ba(II)-functionalized zeolite and ILs containing [SO 4 ] 2– anion driven by the formation of BaSO 4 salts. The as-produced PLs are featured by well-maintained crystallinity of the porous host, good fluidity and stability, enhanced gas uptake capacity, and attractive performance in small gas molecule utilization.

36 MATERIALS SCIENCE↗
DOE OSTI2022

High‐Performance and Stable Semi‐Transparent Perovskite Solar Cells through Composition Engineering

Abstract Semi‐transparent perovskite solar cells (ST‐PeSCs) have tremendous potential as solar windows owing to their higher efficiency and visible transmittance. However, studies toward this application are still nascent, particularly in unraveling the interplay between how the perovskite composition impacts the achievable device performance and stability. Here, the role of A‐ and X‐site modification in APbX 3 perovskites is studied to understand their influence on these factors. Through detailed experimental and simulation work, it is found that a perovskite composition consisting of cesium (Cs) and formamidinium (FA) at the A‐site delivers the best device performance over a range of band gaps, which are tuned by changes to the X‐site anion. Using this optimized perovskite composition, power conversion efficiencies of 15.5% and 4.1% are achieved for ST‐PeSCs with average visible transmittance values between 20.7% and 52.4%, respectively. Furthermore, the CsFA‐based ST‐PeSCs show excellent long‐term stability under continuous illumination and heating. The stability of the precursor solutions across each of the studied compositions has also been considered, showing dramatic differences in the structural properties of the perovskites and their device performance for all mixed A‐site compositions possessing the archetypal methyl ammonium species, while also confirming the superior stability of the CsFA precursor solutions.

14 SOLAR ENERGY↗
DOE OSTI2021

Controlling Heterogeneous Catalysis with Organic Monolayers on Metal Oxides

A key theme of heterogeneous catalysis research is achieving control of the environment surrounding the active site to precisely steer the reactivity toward desired reaction products. One method toward this goal has been the use of organic ligands or self-assembled monolayers (SAMs) on metal nanoparticles. Metal-bound SAMs are typically employed to improve catalyst selectivity but often decrease the reaction rate as a result of site blocking from the ligands. Recently, the use of metal oxide-bound organic modifiers such as organophosphonic acid (PA) SAMs has shown promise as an additional method for tuning reactions on metal oxide surfaces as well as modifying oxide-supported metal catalysts. In this Account, we summarize recent approaches to enhance catalyst performance with oxide-bound monolayers. These approaches include (1) modification of metal oxide catalysts to tune surface reactions, (2) formation of SAMs on the oxide component of supported metal catalysts to modify sites at the metal–support interface, and (3) enhancement of catalyst performance (e.g., stability) through modification of sites remote from the active sites. Further, both the headgroups and organic tail groups of PA SAMs or other ligands can influence reactions on metal oxide surfaces. Binding of the headgroup can selectively poison certain active sites, altering the selectivity in a manner analogous to metal-bound ligands (at the expense of active site quantity). Moreover, tail groups can be functionalized to interact favorably with reactants and intermediates, for instance through dipole–dipole interactions. On supported metal catalysts like Pt/Al 2 O 3 , PA SAMs can selectively form on the oxide support. This selective deposition allows for modification of the metal–support interface with minimal blockage of metal sites. PA headgroups were shown to provide tunable acid sites at the interface, dramatically improving hydrodeoxygenation rates of various alcohols. Additionally, organic tail functionality was used to activate or stabilize specific reactants at the interface, such as with the use of amine-functionalized PAs to stabilize chemisorption of CO 2 during the reverse water gas shift reaction. PAs have also been found to affect the electronic properties of bulk metal sites through long-range electron withdrawal via the oxide, providing an additional avenue to tune catalytic behavior. Finally, organic modifiers were shown to enhance catalytic performance without directly modifying the active site. For instance, in biphasic liquid environments the modification of catalyst particles with hydrophobic or hydrophilic SAMs shifts the selectivity of multipath reactions on the basis of the hydrophobicities of different intermediates and products. As another “long-range” effect, the deposition of ligands on oxide supports improved catalyst stability through both improved resistance to sintering and suppression of active site poisoning. The recent contributions discussed in this Account demonstrate the versatility and significant potential for the approach of modifying catalysts with oxide-bound organic monolayers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Text Mining for Process–Structure–Properties Relationships in Metals

With the advent of large language models (LLMs), the vast unstructured text within millions of academic papers is increasingly accessible for materials discovery—although significant challenges remain. While LLMs offer promising few- and zero-shot learning capabilities, particularly valuable in the materials domain where expert annotations are scarce, general-purpose LLMs often fail to address key materials-specific queries without further adaptation. To bridge this gap, fine-tuning LLMs on human-labeled data is essential for effective structured knowledge extraction (Liu in The Importance of Human-Labeled Data in the Era of LLMs, 2023). Here, in this study, we introduce a novel annotation schema designed to extract generic process–structure–properties relationships from scientific literature. We demonstrate the utility of this approach using a dataset of 128 abstracts, with annotations drawn from two distinct domains: high-temperature materials (Domain I) and uncertainty quantification in simulating materials microstructure (Domain II). Initially, we developed a conditional random field (CRF) model based on MatBERT—a domain-specific BERT variant—and evaluated its performance on Domain I. Subsequently, we compared this model with a fine-tuned LLM (GPT-4o from OpenAI) under identical conditions. Our results indicate that fine-tuning LLMs can significantly improve entity extraction performance over the BERT-CRF baseline on Domain I. However, when additional examples from Domain II were incorporated, the performance of the BERT-CRF model became comparable to that of the GPT-4o model. These findings underscore the potential of our schema for structured knowledge extraction and highlight the complementary strengths of both modeling approaches.

Materials science↗
DOE OSTI2025

Fermilab Booster loss modelling and rebalancing using Bayesian methods

Fermilab Booster is being upgraded for the PIP-II project to support 20Hz ramp rate at higher intensities. Loss trip limits determine the achievable peak power. To meet PIP-II requirements, losses need to be halved as compared to current levels. Losses primarily occur at injection and transition crossing, with both gradually increasing and threshold-like intensity-dependent behaviors. The existing simulation models are not yet good enough for quantitative loss predictions. In practice, it will be necessary to tune up the Booster using iterative methods and operator intuition. In this paper we present an effort to systematically model Booster losses using active learning (Bayesian exploration) techniques, and subsequently to rebalance them for higher trip limit margins. We first created several sets of spatially and temporally isolated orbit and optics knobs, and trained Gaussian process models for each beam loss monitor as well as beam current. This is a complex task due to safety and timing requirements – we discuss mitigations such as uncertainty constraints and approximate fitting. Once models are stable, we perform large-scale single and multi-objective tuning using scalarized objectives made up of critical beam loss locations. Our results demonstrate significant rebalancing of losses, increasing trip margins, as well as an overall improvement in beam transmission efficiency. We are exploring how to combine existing simulations with experimental data and automate the collection procedure so that more advanced surrogate models can be created over time.

Kuklev, Nikita [Fermilab]↗
DOE OSTI2022

Transferring Liquid Metal to form a Hybrid Solid Electrolyte via a Wettability-Tuning Technology for Lithium-Metal Anodes

Integrating solid-state electrolyte (SSE) into Li-metal anodes has demonstrated great promise to unleash the high energy density of rechargeable Li-metal batteries. However, fabricating a highly cyclable SSE/Li-metal anode remains a major challenge because the densification of the SSE is usually incompatible with the reactive Li metal. Here, a liquid-metal-derived hybrid solid electrolyte (HSE) is proposed, and a facile transfer technology to construct an artificial HSE on the Li metal is reported. By tuning the wettability of the transfer substrates, electron- and ion-conductive liquid metal is sandwiched between electron-insulating and ion-conductive LiF and oxides to form the HSE. The transfer technology renders the HSE continuous, dense, and uniform. The HSE, having high ion transport, electron shut-off, and mechanical strength, makes the composite anode deliver excellent cyclability for over 4000 h at 0.5 mA cm(-2) and 1 mAh cm(-2) in a symmetrical cell. When pairing with LiFePO4 and sulfur cathodes, the HSE-coated Li metal dramatically enhances the performance of full cells. Therefore, this work demonstrates that tuning the interfacial wetting properties provides an alternate approach to build a robust solid electrolyte, which enables highly efficient Li-metal anodes.

li-metal anodes↗
DOE OSTI2025

Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction

Abstract The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified Swin Transformer V2 (SwinV2) on ERA5 data and find that it attains superior skill in terms of mean-square errors of deterministic forecasts when compared against the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS). Almost all DL–NWP systems share a core set of hyperparameters and design decisions. To aid and expedite future DL–NWP research, we present an in-depth, systematic exploration of different loss functions, model sizes and depths, patch sizes, and multistep training objectives. We also examine the model performance with metrics beyond the typical accuracy (ACC) and RMSE and investigate how the performance scales with model size. Through our open-source code, scoring pipelines, and models, we share our findings on key aspects of the training pipeline. These ablations reduce the necessity for expensive hyperparameter tuning and lower the barrier to entry for future DL–NWP research. Significance Statement This study investigates the potential of using large-scale transformer-based models for weather prediction, showing that it is possible to achieve high forecast accuracy with simpler, off-the-shelf architectures. By training a minimally modified SwinV2 transformer on ERA5 data, we show that the model achieves competitive forecast skill in terms of mean-square error for key variables, outperforming the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS) at all lead times. Our findings suggest that effective training strategies, such as multistep fine-tuning and channel-weighted losses, significantly enhance the model’s performance. However, we also highlight that these improvements come with trade-offs in other areas, such as ensemble spread and high-frequency spatial detail. This work highlights the promise of deep learning in improving weather forecasts, which could lead to better preparedness and response to weather events, ultimately benefiting society by providing more reliable weather predictions.

Willard, Jared D. [Lawrence Berkeley National Labo↗
DOE OSTI2022

Managing Sn-Supply to Tune Surface Characteristics of Vapor-Diffusion Coating of Nb₃Sn

Nb₃Sn promises better RF performance (Q and E_{acc}) than niobium at any given temperature because of superior superconducting properties. Nb₃Sn-coated SRF cavities are now produced routinely by growing a few microns thick Nb₃Sn films inside Nb cavities via the tin vapor diffusion technique. Sn evaporation and consumption during the growth process notably affect the quality of the coating. Aiming at favorable surface characteristics that could enhance the RF performance, many coatings were produced by varying Sn sources and temperature profiles. Coupon samples were examined using different material characterization techniques, and a selected few sets of coating parameters were used to coat 1.3 GHz single-cell cavities for RF testing. The Sn supply’s careful tuning is essential to manage the microstructure, roughness, and overall surface characteristics of the coating. We summarize the material analysis of witness samples and discuss the performance of several Nb₃Sn-coated single-cell cavities linked to Sn-source characteristics and observed Sn consumption during the film growth process.

Pudasaini, U.↗
DOE OSTI2021

McMillan electron lens in a system with space charge

Space charge (SC) forces of a circulating beam in a ring have both linear (defocusing) and nonlinear components, due to a nonuniform beam distribution. The linear component of SC forces produces a betatron tune shift, which is the largest for a zero-amplitude particle, while the nonlinear component produces an amplitude-dependent betatron tune spread. These SC effects are responsible for several undesirable phenomena in accelerators: emittance growth, particle losses, beam halo, etc. In this paper, we investigate the possibility to mitigate the distributed SC forces by a thin McMillan lens, providing an axially-symmetric kick, which is qualitatively opposite to the accumulated effect of beam's own SC. Experimentally, the proposed concept can be tested in Fermilab's IOTA ring. A thin McMillan lens can be implemented by a short (70) insertion of an electron beam with a specifically chosen density distribution in transverse directions. In this article, to test if McMillan lenses can reduce the tune spread induced by SC, we make several simulations with a 6-D particle tracking code, Synergia. We choose such beam and lattice parameters that the SC tune spread is roughly 0.5 and the emittance growth due to the half-integer resonance is clearly observed without the SC compensation. Then, we focus on reducing the emittance growth by adjusting the bare betatron tunes using the ring quadrupoles, and reducing the tune spread by the McMillan lenses. The results of reducing a large tune spread (≈ 0.5), reported here, are not perfect, but substantial. There is still room for further investigation. The simulations performed so far in this work indicate that McMillan lenses can cope with an SC tune spread of ≈ 0.1 per lens.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Validation of edge turbulence codes against the TCV-X21 diverted L-mode reference case

Self-consistent full-size turbulent-transport simulations of the divertor and scrape-off-layer (SOL) of existing tokamaks have recently become feasible. This enables the direct comparison of turbulence simulations against experimental measurements. In this work, we perform a series of diverted ohmic L-mode discharges on the tokamak à configuration variable (TCV) tokamak, building a first-of-a-kind dataset for the validation of edge turbulence models. This dataset, referred to as TCV-X21, contains measurements from five diagnostic systems from the outboard midplane (OMP) to the divertor targets—giving a total of 45 one- and two-dimensional comparison observables in two toroidal magnetic field directions. The experimental dataset is used to validate three flux-driven 3D fluid-turbulence models—GBS, GRILLIX and TOKAM3X. With each model, we perform simulations of the TCV-X21 scenario, individually tuning the particle and power source rates to achieve a reasonable match of the upstream separatrix value of density and electron temperature. We find that the simulations match the experimental profiles for most observables at the OMP—both in terms of profile shape and absolute magnitude—while a comparatively poorer agreement is found towards the divertor targets. The match between simulation and experiment is seen to be sensitive to the value of the resistivity, the heat conductivities, the power injection rate and the choice of sheath boundary conditions. Additionally, despite targeting a sheath-limited regime, the discrepancy between simulations and experiment also suggests that the neutral dynamics should be included. The results of this validation show that turbulence models are able to perform simulations of existing devices and achieve reasonable agreement with experimental measurements. Where disagreement is found, the validation helps to identify how the models can be improved. By publicly releasing the experimental dataset and validation analysis, this work should help to guide and accelerate the development of predictive turbulence simulations of the edge and SOL.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Intrinsic scintillation performance & europium concentration effects in RbSr 2 I 5 and RbSr 2 Br 5 scintillators

Scintillators play crucial roles in homeland security applications like gamma ray spectroscopy and high energy X-ray radiography. For promising new scintillators, fine-tuning the luminescent dopant concentration is one avenue to further improve their performance and tailor their properties. In this work, the effects of europium dopant concentrations on the crystal growth, luminescence and scintillation properties of RbSr 2 Br 5 and RbSr 2 I 5 crystals was investigated. Nine transparent 7 mm diameter single crystals were grown via the Vertical Bridgman method. Here, the optical band gap of RbSr 2 Br 5 was 5.9 eV and that of RbSr 2 I 5 was 4.7 eV. High scintillation performance was achieved with a relatively low europium concentration of 1 mol%. For both RbSr 2 Br 5 :Eu and RbSr 2 I 5 :Eu crystals, light yield of 60–90,000 ph/MeV, energy resolution 2.8–4.0 % at 662 keV, and X-ray afterglow 0.79–1.5 % at 2 ms were obtained.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks

Abstract Networks composed of nanoscale memristive components, such as nanowire and nanoparticle networks, have recently received considerable attention because of their potential use as neuromorphic devices. In this study, we explore ergodicity in memristive networks, showing that the performance on machine leaning tasks improves when these networks are tuned to operate at the edge between two global stability points. We find this lack of ergodicity is associated with the emergence of memory in the system. We measure the level of ergodicity using the Thirumalai-Mountain metric, and we show that in the absence of ergodicity, two different memristive network systems show improved performance when utilized as reservoir computers (RC). We highlight that it is also important to let the system synchronize to the input signal in order for the performance of the RC to exhibit improvements over the baseline.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Ergodicity, lack thereof, and the performance of reservoir computing with memristive networks and nanowire

Networks composed of nanoscale memristive components, such as nanowire and nanoparticle networks, have recently received considerable attention because of their potential use as neuromorphic devices. In this study, we explore ergodicity in memristive networks, showing that the performance on machine leaning tasks improves when these networks are tuned to operate at the edge between two global stability points. We find this lack of ergodicity is associated with the emergence of memory in the system. We measure the level of ergodicity using the Thirumalai-Mountain metric, and we show that in the absence of ergodicity, two different memristive network systems show improved performance when utilized as reservoir computers (RC). We highlight that it is also important to let the system synchronize to the input signal in order for the performance of the RC to exhibit improvements over the baseline.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Performance Evaluation of Heterogeneous GPU Programming Frameworks for Hemodynamic Simulations

Preparing for the deployment of large scientific and engineering codes on upcoming exascale systems with GPU-dense nodes is made challenging by the unprecedented diversity of device architectures and heterogeneous programming models. In this work, we evaluate the process of porting a massively parallel, fluid dynamics code written in CUDA to SYCL, HIP, and Kokkos with a range of backends, using a combination of automated tools and manual tuning. We use a proxy application along with a custom performance model to inform the results and identify additional optimization strategies. At scale performance of the programming model implementations are evaluated on pre-production GPU node architectures for Frontier and Aurora, as well as on current NVIDIA device-based systems Summit and Polaris. Real-world workloads representing 3D blood flow calculations in complex vasculature are assessed. Our analysis highlights critical trade-offs between code performance, portability, and development time.

Martin, Aristotle↗
DOE OSTI2022

Two-Stage Reinforcement Learning Policy Search for Grid-Interactive Building Control

This paper develops an intelligent grid-interactive building controller, which optimizes building operation during both normal hours and demand response (DR) events. To avoid costly on-demand computation and to adapt to non-linear building models, the controller utilizes reinforcement learning (RL) and makes real-time decisions based on a near-optimal control policy. Learning such a policy typically amounts to solving a hard non-convex optimization problem. We propose to address this problem with a novel global-local policy search method. In the first stage, an RL algorithm based on zero-order gradient estimation is leveraged to search for the optimal policy globally, due to its scalability and the potential to escape some poor performing local optima. The obtained policy is then fine-tuned locally to bring the first-stage solution closer to that of the original unsmoothed problem. Experiments on a simulated five-zone commercial building demonstrate the advantages of the proposed method over existing learning approaches. They also show that the learned control policy outperforms a pragmatic linear model predictive controller (MPC) and approaches the performance of an oracle MPC in testing scenarios. Using a state-of-the-art advanced computing system, we demonstrate that the controller can be learned and deployed within hours of training.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗