Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “hybrid algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Development of Real-Time System Identification to Detect Abnormal Operations in a Gas Turbine Cycle

Here, we present a novel online system identification methodology for monitoring the performance of power systems. This methodology was demonstrated in a gas turbine recuperated power plant designed for a hybrid configuration. A 120-kW Garrett microturbine modified to test dynamic control strategies for hybrid power systems designed at the National Energy Technology Laboratory (NETL) was used to implement and validate this online system identification methodology. The main component of this methodology consists of an empirical transfer function model implemented in parallel to the turbine speed operation and the fuel control valve, which can monitor the process response of the gas turbine system while it is operating. During fully closed-loop operations or automated control, the output of the controller, fuel valve position, and the turbine speed measurements were fed for a given period of time to a recursive algorithm that determined the transfer function parameters during the nominal condition. After the new parameters were calculated, they were fed into the transfer function model for online prediction. The turbine speed measurement was compared against the transfer function prediction, and a control logic was implemented to capture when the system operated at nominal or abnormal conditions. To validate the ability to detect abnormal conditions during dynamic operations, drifting in the performance of the gas turbine system was evaluated. A leak in the turbomachinery working fluid was emulated by bleeding 10% of the airflow from the compressor discharge to the atmosphere, and electrical load steps were performed before and after the leak. This tool could detect the leak 7 s after it had occurred, which accounted for a fuel flow increase of approximately 15.8% to maintain the same load and constant turbine speed operations.

algorithms↗
NASA NTRS1992

Estimation of the absolute position of mobile systems by an optoelectronic processor

A method that determine the absolute position of a mobile system with a hybrid optoelectronic processor has been developed. Position estimates are based on an analysis of circular landmarks that are detected by a TV camera attached to the mobile system. The difference between the known shape of the landmark and its image provides the information needed to determine the absolute position of the mobile system. For robust operation, the parameters of the landmark image are extracted at high speeds using an optical processor that performs an optical Hough transform. The coordinates of the mobile system are computed from these parameters in a digital co-processor using fast algorithms. Different sources of position estimation errors have also been analyzed, and consequent algorithms to improve the navigation performance of the mobile system have been developed and evaluated by both computer simulation and experiments.

Feng, Liqiang↗
NASA NTRS2000

HTMT-class Latency Tolerant Parallel Architecture for Petaflops Scale Computation

Computational Aero Sciences and other numeric intensive computation disciplines demand computing throughputs substantially greater than the Teraflops scale systems only now becoming available. The related fields of fluids, structures, thermal, combustion, and dynamic controls are among the interdisciplinary areas that in combination with sufficient resolution and advanced adaptive techniques may force performance requirements towards Petaflops. This will be especially true for compute intensive models such as Navier-Stokes are or when such system models are only part of a larger design optimization computation involving many design points. Yet recent experience with conventional MPP configurations comprising commodity processing and memory components has shown that larger scale frequently results in higher programming difficulty and lower system efficiency. While important advances in system software and algorithms techniques have had some impact on efficiency and programmability for certain classes of problems, in general it is unlikely that software alone will resolve the challenges to higher scalability. As in the past, future generations of high-end computers may require a combination of hardware architecture and system software advances to enable efficient operation at a Petaflops level. The NASA led HTMT project has engaged the talents of a broad interdisciplinary team to develop a new strategy in high-end system architecture to deliver petaflops scale computing in the 2004/5 timeframe. The Hybrid-Technology, MultiThreaded parallel computer architecture incorporates several advanced technologies in combination with an innovative dynamic adaptive scheduling mechanism to provide unprecedented performance and efficiency within practical constraints of cost, complexity, and power consumption. The emerging superconductor Rapid Single Flux Quantum electronics can operate at 100 GHz (the record is 770 GHz) and one percent of the power required by convention semiconductor logic. Wave Division Multiplexing optical communications can approach a peak per fiber bandwidth of 1 Tbps and the new Data Vortex network topology employing this technology can connect tens of thousands of ports providing a bi-section bandwidth on the order of a Petabyte per second with latencies well below 100 nanoseconds, even under heavy loads. Processor-in-Memory (PIM) technology combines logic and memory on the same chip exposing the internal bandwidth of the memory row buffers at low latency. And holographic storage photorefractive storage technologies provide high-density memory with access a thousand times faster than conventional disk technologies. Together these technologies enable a new class of shared memory system architecture with a peak performance in the range of a Petaflops but size and power requirements comparable to today's largest Teraflops scale systems. To achieve high-sustained performance, HTMT combines an advanced multithreading processor architecture with a memory-driven coarse-grained latency management strategy called "percolation", yielding high efficiency while reducing the much of the parallel programming burden. This paper will present the basic system architecture characteristics made possible through this series of advanced technologies and then give a detailed description of the new percolation approach to runtime latency management.

Sterling, Thomas↗
NASA NTRS2012

Control Algorithms Charge Batteries Faster

On March 29, 2011, NASA s Mercury Surface, Space Environment, Geochemistry and Ranging (MESSENGER) spacecraft beamed a milestone image to Earth: the first photo of Mercury taken from orbit around the solar system s innermost planet. (MESSENGER is also the first spacecraft to orbit Mercury.) Like most of NASA s deep space probes, MESSENGER is enabled by a complex power system that allows its science instruments and communications to function continuously as it travels millions of miles from Earth. "Typically, there isn't one particular power source that can support the entire mission," says Linda Taylor, electrical engineer in Glenn Research Center s Power Systems Analysis Branch. "If you have solar arrays and you are in orbit, at some point you re going to be in eclipse." Because of this, Taylor explains, spacecraft like MESSENGER feature hybrid power systems. MESSENGER is powered by a two-panel solar array coupled with a nickel hydrogen battery. The solar arrays provide energy to the probe and charge the battery; when the spacecraft s orbit carries it behind Mercury and out of the Sun s light, the spacecraft switches to battery power to continue operations. Typically, hybrid systems with multiple power inputs and a battery acting alternately as storage and a power source require multiple converters to handle the power flow between the devices, Taylor says. (Power converters change the qualities of electrical energy, such as from alternating current to direct current, or between different levels of voltage or frequency.) This contributes to a pair of major concerns for spacecraft design. "Weight and size are big drivers for any space application," Taylor says, noting that every pound added to a space vehicle incurs significant costs. For an innovative solution to managing power flows in a lightweight, cost-effective manner, NASA turned to a private industry partner.

Source record↗
NASA NTRS2014

The FIELDS Instrument Suite on MMS: Scientific Objectives, Measurements, and Data Products

The FIELDS instrumentation suite on the Magnetospheric Multiscale (MMS) mission provides comprehensive measurements of the full vector magnetic and electric fields in the reconnection regions investigated by MMS, including the dayside magnetopause and the night-side magnetotail acceleration regions out to 25 Re. Six sensors on each of the four MMS spacecraft provide overlapping measurements of these fields with sensitive cross-calibrations both before and after launch. The FIELDS magnetic sensors consist of redundant flux-gate magnetometers (AFG and DFG) over the frequency range from DC to 64 Hz, a search coil magnetometer (SCM) providing AC measurements over the full whistler mode spectrum expected to be seen on MMS, and an Electron Drift Instrument (EDI) that calibrates offsets for the magnetometers. The FIELDS three-axis electric field measurements are provided by two sets of biased double-probe sensors (SDP and ADP) operating in a highly symmetric spacecraft environment to reduce significantly electrostatic errors. These sensors are complemented with the EDI electric measurements that are free from all local spacecraft perturbations. Cross-calibrated vector electric field measurements are thus produced from DC to 100 kHz, well beyond the upper hybrid resonance whose frequency provides an accurate determination of the local electron density. Due to its very large geometric factor, EDI also provides very high time resolution (∼1 ms) ambient electron flux measurements at a few selected energies near 1 keV. This paper provides an overview of the FIELDS suite, its science objectives and measurement requirements, and its performance as verified in calibration and cross-calibration procedures that result in anticipated errors less than 0.1 nT in B and 0.5 mV/m in E. Summaries of data products that result from FIELDS are also described, as well as algorithms for cross-calibration. Details of the design and performance characteristics of AFG/DFG, SCM, ADP, SDP, and EDI are provided in five companion papers.

Electromagnetic field measurements↗
DOE OSTI2025

Novel challenges in tracking self-interacting dark matter subhalos

Cosmological N-body simulations are among the primary tools for studying structure formation in the Universe. Analyses of these simulations critically depend on accurately identifying and tracking dark matter subhalos over time. In recent years, several new algorithms have been developed to improve the accuracy and consistency of subhalo tracking in cold dark matter (CDM) simulations. These algorithms should be revisited in the context of new physics beyond gravity, which can modify the evolution and final properties of subhalo populations. In this work, we apply the particle-tracking-based subhalo finder Symfind to velocity-dependent self-interacting dark matter (SIDM) simulations with large cross section amplitudes to assess the performance of particle-tracking methods beyond the CDM paradigm. We find that the core-particle-tracking technique, which is key to the success of these algorithms in CDM, does not always yield accurate results in SIDM. In particular, the interplay between dark matter self-interactions and tidal stripping can cause the diffusion of core particles to larger radii, leading particle-tracking-based algorithms to prematurely lose track of SIDM subhalos. For massive core-expansion subhalos and core-collapse subhalos that experience close or repeated pericentric passages, a significant fraction of core particles can be lost, and particle-tracking-based finders such as Symfind offer no clear advantage over traditional methods that rely on identifying phase-space overdensities. On the other hand, for subhalos with large pericentric distances or fewer, more distant passages, Symfind tends to outperform. These differences depend sensitively on the cross section amplitude and turnover velocity of the underlying SIDM model. We therefore recommend a hybrid approach that leverages the strengths of both techniques to produce complete and robust catalogs of core-expansion and core-collapse SIDM subhalos.

cosmological simulations↗
DOE OSTI2025

A Morphological Model to Separate Resolved–Unresolved Sources in the DESI Legacy Surveys: Application in the LS4 Alert Stream

Separating resolved and unresolved sources in large imaging surveys is a fundamental step to enable downstream science, such as searching for extragalactic transients in wide-field time-domain surveys. Here we present our method to effectively separate point sources from the resolved, extended sources in the Dark Energy Spectroscopic Instrument (DESI) Legacy Surveys (LS). We develop a supervised machine learning model based on the Gradient Boosting algorithm XGBoost. The features input to the model are purely morphological and are derived from the tabulated LS data products. We train the model using ∼2 × 10 5 LS sources in the COSMOS field with HST morphological labels and evaluate the model performance on LS sources with spectroscopic classification from the DESI Data Release 1 (∼2 × 10 7 objects) and the Sloan Digital Sky Survey Data Release 17 (∼3 × 10 6 objects), as well as on ∼2 × 10 8 Gaia stars. A significant fraction of LS sources are not observed in every LS filter, and we therefore build a “Hybrid” model as a linear combination of two XGBoost models, each containing features combining aperture flux measurements from the “blue” (gr) and “red” (iz) filters. The Hybrid model shows a reasonable balance between sensitivity and robustness, and achieves higher accuracy and flexibility compared to the LS morphological typing. With the Hybrid model, we provide classification scores for ∼3 × 10 9 LS sources, making this the largest ever machine learning catalog separating resolved and unresolved sources. The catalog has been incorporated into the real-time pipeline of the La Silla Schmidt Southern Survey (LS4), enabling the identification of extragalactic transients within the LS4 alert stream.

astrostatistics↗
NASA NTRS2024

Universal Coating by 3D Hybrid Programmable Matter

Motivated by the prospect of nano-robots that assist human physiological functions at the nanoscale, we investigate the coating problem in the three-dimensional model for hybrid programmable matter. In this model, a single agent with strictly limited viewing range and the computational capability of a deterministic finite automaton can act on passive tiles by picking up a tile, moving, and placing it at some spot. The goal of the coating problem is to fill each node of some surface graph of size n with a tile. We first solve the problem on a restricted class of graphs with a single tile type, and then use constantly many tile types to encode this graph in certain surface graphs capturing the surface of 3D objects. Our algorithm requires O(n^2) steps, which is worst-case optimal compared to an agent with global knowledge and no memory restrictions.

programmable matter↗
NASA NTRS2018

Comparing and Integrating Constraint Programming and Temporal Planning for Quantum Circuit Compilation

Recently, the makespan-minimization problem of compiling a general class of quantum algorithms into near-term quantum processors has been introduced to the AI community. The research demonstrated that temporal planning is a strong solution approach for the studied class of quantum circuit compilation (QCC) problems. In this paper, we explore the use of methods from operations research, specifically constraint programming (CP), as an alternative and complementary approach to temporal planning. We also extend previous work by introducing two new problem variations that incorporate important characteristics identified by the quantum computing community. We apply temporal planning and CP to the baseline and extended QCC problems as both stand-alone and hybrid approaches. The hybrid method uses solutions found by temporal planning to warm-start CP, leveraging the ability of temporal planning to find satisficing solutions to problems with a high degree of task optionality, an area that CP typically struggles with. These solutions are then used to seed the CP formulation which significantly benefits from inferred bounds on planning horizon and task counts provided by the warm-start. Our extensive empirical evaluation indicates that while stand-alone CP is not competitive with temporal planning, except for the smallest problems, CP in a hybrid setting is beneficial for all temporal planners in all problem classes.

quantum circuit compilation↗
NASA NTRSDate not supplied

Exploring Quantum State Preparation Using Tensor Networks and Sparse Wavefunction Simulations

The variational quantum eigenvalue solver is a powerful hybrid quantum-classical approach that has been suggested as a candidate method to run on near-term quantum hardware for computing ground state electronic energies of molecular systems. However, even for small molecules, the number of variational parameters and qubits required to minimize the electronic energy is beyond the reach of current quantum computers except for small basis sets. We explore a new paradigm for state preparation where we test how much of the optimization can be approximately prepared with classical computers to reduce the number of optimization steps performed using a quantum device. By adapting a recent algorithm for the factorized form of the UCC ansatz, we can study molecular electronic structure problems with up to 64 qubits. In addition, we also test a related approach of using tensor networks to optimize quantum circuits in order to benchmark various lattice models. We present results using these approaches and discuss strategies for incorporating these ideas into variational algorithms involving near-term quantum computers. Our results help demonstrate the strength of the UCC ansatz and address pressing questions about optimal initial parameterizations and circuit construction.

quantum computing↗
DOE OSTI2023

Hierarchical Flexibility Offering Strategy for Integrated Hybrid Resources in Real-time Energy Markets

This paper proposes a hierarchical model for determining the energy flexibility offering strategy of integrated hybrid resources (IHRs) in power distribution systems to participate in real-time energy markets. The proposed model utilizes the scalability, fast response time, and uncertainty observation of deep reinforcement learning (DRL) to overcome the scalability issue of operating numerous flexible resources and deliverability of energy flexibility to the real-time markets in the presence of the network constraints. To that end, the power distribution system is divided into multiple IHRs, where different types of flexible loads, energy storage systems, and solar plants with controllable inverters are operated through local IHR controllers, trained by deep deterministic policy gradient (DDPG) algorithm. Active power request and reactive power capacity of IHRs are then transmitted to a central flexibility controller, where a quadratic optimization model ensures the deliverability of the energy flexibility to the real-time energy market by satisfying the distribution network constraints. The proposed model is implemented on the 123-bus test power distribution system, demonstrating the capability of DRL-based hierarchical model for scalable operation of IHRs in order to offer deliverable energy flexibility to the real-time energy market.

Majidi, Majid↗
DOE OSTI2020

Learning to Count Grave Sites for Cemetery Observation Models With Satellite Imagery

Understanding how people occupy open spaces is important for research in support of population modeling, policy, national security, emergency response, and sustainability. For the past decade, there has been an increase in research toward capturing and reporting population dynamics and patterns of life at the building level and in some open public spaces such as cemeteries and parks. This is done through observation models developed from local sociocultural information acquired at various spatiotemporal scales to inform night, day, and episodic population occupancy estimates (people/1000 sq ft). Sociocultural information for cemeteries and parks is scarcely available and often collected manually. The process is not only marred by inconsistencies but is laborious and time consuming. In this study, we leverage convolutional neural networks (CNNs) and satellite imagery to derive grave site counts as proxy variables to support scalable and accurate sociocultural data required in a population observation model. Through a hybrid workflow (weak localization plus regression model), we characterize a large scale automation process to counting of grave sites. We evaluate and demonstrate the efficacy of proposed workflow using out-of-data set large satellite imagery and establish its broader impact on cemetery observation models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

MrHyDE v.1.0

SAND2024-01324O MrHyDE, which stands for Multi-resolution Hybridized Differential Equations, is a general-purpose C++ package for the solution of coupled multiphysics and multiscale systems on massively parallel computing systems. MrHyDE is designed to enable moving beyond forward simulation for multiscale applications which includes optimization, control, uncertainty quantification, and stochastic inversion. The framework provides interfaces to several packages within the Trilinos framework and leverages automatic differentiation to enable adjoint capabilities for large-scale, gradient-based optimization. MrHyDE provides automated multiscale capabilities through a subgrid model interface and multiscale Dirichlet-to-Neumann maps. For extreme-scale applications, MrHyDE provides in situ data-compression algorithms to reduce memory requirements while maintaining performance. MrHyDE is a general-purpose, computational framework for the solution of multiscale and multiphysics applications. It uses a combination of structure-preserving, physics-compatible discretizations, fully implicit methods, multi-resolution schemes, or fully explicit methods. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
NASA NTRS2002

High-Efficiency High-Resolution Global Model Developments at the NASA Goddard Data Assimilation Office

The Data Assimilation Office (DAO) has been developing a new generation of ultra-high resolution General Circulation Model (GCM) that is suitable for 4-D data assimilation, numerical weather predictions, and climate simulations. These three applications have conflicting requirements. For 4-D data assimilation and weather predictions, it is highly desirable to run the model at the highest possible spatial resolution (e.g., 55 km or finer) so as to be able to resolve and predict socially and economically important weather phenomena such as tropical cyclones, hurricanes, and severe winter storms. For climate change applications, the model simulations need to be carried out for decades, if not centuries. To reduce uncertainty in climate change assessments, the next generation model would also need to be run at a fine enough spatial resolution that can at least marginally simulate the effects of intense tropical cyclones. Scientific problems (e.g., parameterization of subgrid scale moist processes) aside, all three areas of application require the model's computational performance to be dramatically improved as compared to the previous generation. In this talk, I will present the current and future developments of the "finite-volume dynamical core" at the Data Assimilation Office. This dynamical core applies modem monotonicity preserving algorithms and is genuinely conservative by construction, not by an ad hoc fixer. The "discretization" of the conservation laws is purely local, which is clearly advantageous for resolving sharp gradient flow features. In addition, the local nature of the finite-volume discretization also has a significant advantage on distributed memory parallel computers. Together with a unique vertically Lagrangian control volume discretization that essentially reduces the dimension of the computational problem from three to two, the finite-volume dynamical core is very efficient, particularly at high resolutions. I will also present the computational design of the dynamical core using a hybrid distributed-shared memory programming paradigm that is portable to virtually any of today's high-end parallel super-computing clusters.

Lin, Shian-Jiann↗
NASA NTRS2002

High-Efficiency High-Resolution Global Model Developments at the NASA Goddard Data Assimilation Office

The Data Assimilation Office (DAO) has been developing a new generation of ultra-high resolution General Circulation Model (GCM) that is suitable for 4-D data assimilation, numerical weather predictions, and climate simulations. These three applications have conflicting requirements. For 4-D data assimilation and weather predictions, it is highly desirable to run the model at the highest possible spatial resolution (e.g., 55 kin or finer) so as to be able to resolve and predict socially and economically important weather phenomena such as tropical cyclones, hurricanes, and severe winter storms. For climate change applications, the model simulations need to be carried out for decades, if not centuries. To reduce uncertainty in climate change assessments, the next generation model would also need to be run at a fine enough spatial resolution that can at least marginally simulate the effects of intense tropical cyclones. Scientific problems (e.g., parameterization of subgrid scale moist processes) aside, all three areas of application require the model's computational performance to be dramatically improved as compared to the previous generation. In this talk, I will present the current and future developments of the "finite-volume dynamical core" at the Data Assimilation Office. This dynamical core applies modem monotonicity preserving algorithms and is genuinely conservative by construction, not by an ad hoc fixer. The "discretization" of the conservation laws is purely local, which is clearly advantageous for resolving sharp gradient flow features. In addition, the local nature of the finite-volume discretization also has a significant advantage on distributed memory parallel computers. Together with a unique vertically Lagrangian control volume discretization that essentially reduces the dimension of the computational problem from three to two, the finite-volume dynamical core is very efficient, particularly at high resolutions. I will also present the computational design of the dynamical core using a hybrid distributed- shared memory programming paradigm that is portable to virtually any of today's high-end parallel super-computing clusters.

Lin, Shian-Jiann↗
DOE OSTI2020

TensorFlow Quantum: A Software Framework for Quantum Machine Learning

We introduce TensorFlow Quantum (TFQ), an open source library for the rapid prototyping of hybrid quantum-classical models for classical or quantum data. This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators. We provide an overview of the software architecture and building blocks through several examples and review the theory of hybrid quantum-classical neural networks. We illustrate TFQ functionalities via several basic applications including supervised learning for quantum classification, quantum control, simulating noisy quantum circuits, and quantum approximate optimization. Moreover, we demonstrate how one can apply TFQ to tackle advanced quantum learning tasks including meta-learning, layerwise learning, Hamiltonian learning, sampling thermal states, variational quantum eigensolvers, classification of quantum phase transitions, generative adversarial networks, and reinforcement learning. We hope this framework provides the necessary tools for the quantum computing and machine learning research communities to explore models of both natural and artificial quantum systems, and ultimately discover new quantum algorithms which could potentially yield a quantum advantage.

Broughton, Michael↗
NASA NTRS2013

Scheduling Operations for Massive Heterogeneous Clusters

High-performance computing (HPC) programming has become increasingly difficult with the advent of hybrid supercomputers consisting of multicore CPUs and accelerator boards such as the GPU. Manual tuning of software to achieve high performance on this type of machine has been performed by programmers. This is needlessly difficult and prone to being invalidated by new hardware, new software, or changes in the underlying code. A system was developed for task-based representation of programs, which when coupled with a scheduler and runtime system, allows for many benefits, including higher performance and utilization of computational resources, easier programming and porting, and adaptations of code during runtime. The system consists of a method of representing computer algorithms as a series of data-dependent tasks. The series forms a graph, which can be scheduled for execution on many nodes of a supercomputer efficiently by a computer algorithm. The schedule is executed by a dispatch component, which is tailored to understand all of the hardware types that may be available within the system. The scheduler is informed by a cluster mapping tool, which generates a topology of available resources and their strengths and communication costs. Software is decoupled from its hardware, which aids in porting to future architectures. A computer algorithm schedules all operations, which for systems of high complexity (i.e., most NASA codes), cannot be performed optimally by a human. The system aids in reducing repetitive code, such as communication code, and aids in the reduction of redundant code across projects. It adds new features to code automatically, such as recovering from a lost node or the ability to modify the code while running. In this project, the innovators at the time of this reporting intend to develop two distinct technologies that build upon each other and both of which serve as building blocks for more efficient HPC usage. First is the scheduling and dynamic execution framework, and the second is scalable linear algebra libraries that are built directly on the former.

Humphrey, John↗
DOE OSTI2023

Role of reinforcement learning for risk-based robust control of cyber-physical energy systems

Critical infrastructures such as cyber-physical energy systems (CPS-E) integrate information flow and physical operations that are vulnerable to natural and targeted failures. Safe, secure, and reliable operation and control of CPS-E is critical to ensure societal well-being and economic prosperity. Automated control is key for real-time operations and may be mathematically cast as a sequential decision-making problem under uncertainty. Emergence of data-driven techniques for decision making under uncertainty, such as reinforcement learning (RL), have led to promising advances for addressing sequential decision-making problems for risk-based robust CPS-E control. However, existing research challenges include understanding the applicability of RL methods across diverse CPS-E applications, addressing the effect of risk preferences across multiple RL methods, and development of open-source domain-aware simulation environments for RL experimentation within a CPS-E context. This article systematically analyzes the applicability of four types of RL methods (model-free, model-based, hybrid model-free and model-based, and hierarchical) for risk-based robust CPS-E control. Problem features and solution stability for the RL methods are also discussed. We demonstrate and compare the performance of multiple RL methods under different risk specifications (risk-averse, risk-neutral, and risk-seeking) through the development and application of an open-source simulation environment. Motivating numerical simulation examples include representative single-zone and multizone building control use cases. Finally, six key insights for future research and broader adoption of RL methods are identified, with specific emphasis on problem features, algorithmic explainability, and solution stability.

24 POWER TRANSMISSION AND DISTRIBUTION↗