Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “simulation modeling training analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2014

An efficient optimization of well placement and control for a geothermal prospect under geological uncertainty

This work applies an efficient optimization technique based on a multivariate adaptive regression spline (MARS) technique to determine the optimal design and engineering of a potential geothermal production operation at a prospect near Superstition Mountain in Southern California, USA. The faster MARS-based statistical model is used as a surrogate for higher-fidelity physical models within the intensive optimization process. Its use allows for the exploration of the impacts of specific engineering design parameters in the context of geologic uncertainty as a means to both understand and maximize profitability of the production operation. The MARS model is initially developed from a training dataset generated by a finite set of computationally complex hydrothermal models applied to the prospect. Its application reveals that the optimal engineering design variables can differ considerably assuming different choices of hydrothermal flow properties, which, in turn, indicates the importance of reducing the uncertainty of key geologic properties. The major uncertainty sources in the natural-system are identified and ranked first by an efficient MARS-enabled total order sensitivity quantification, which is then used to assist evaluating the effect of geological uncertainties on optimized results. At the Southern California prospect, this parameter sensitivity analysis suggests that groundwater circulation through high permeable structures, rather than heat conduction through impermeable granite, is the primary heat transfer method during geothermal extraction. Reservoir histories simulated using optimal parameters with different constraints are analyzed and compared to investigate the longevity and maximum profit of the geothermal resources. The comparison shows that the longevity and profit are very likely to be overestimated by optimizations without appropriate constraints on natural conditions. In addition to geothermal energy production, this optimization approach can also be used to manage other geologic resource operations, such as hydrocarbon production or CO 2 sequestration, under uncertain reservoir conditions.

15 GEOTHERMAL ENERGY↗
DOE OSTI2021

Probabilistic Deep Learning Approach to Automate the Interpretation of Multi-phase Diffraction Spectra

Autonomous synthesis and characterization of inorganic materials requires the automatic and accurate analysis of X-ray diffraction spectra. For this task, we designed a probabilistic deep learning algorithm to identify complex multi-phase mixtures. At the core of this algorithm lies an ensemble convolutional neural network trained on simulated diffraction spectra, which are systematically augmented with physics-informed perturbations to account for artifacts that can arise during experimental sample preparation and synthesis. In this work, larger perturbations associated with off-stoichiometry are also captured by supplementing the training set with hypothetical solid solutions. Spectra containing mixtures of materials are analyzed with a newly developed branching algorithm that utilizes the probabilistic nature of the neural network to explore suspected mixtures and identify the set of phases that maximize confidence in the prediction. Our model is benchmarked on simulated and experimentally measured diffraction spectra, showing exceptional performance with accuracies exceeding those given by previously reported methods based on profile matching and deep learning. We envision that the algorithm presented here may be integrated in experimental workflows to facilitate the high-throughput and autonomous discovery of inorganic materials.

36 MATERIALS SCIENCE↗
NASA NTRS2000

Efficient Analysis of Complex Structures

Last various accomplishments achieved during this project are : (1) A Survey of Neural Network (NN) applications using MATLAB NN Toolbox on structural engineering especially on equivalent continuum models (Appendix A). (2) Application of NN and GAs to simulate and synthesize substructures: 1-D and 2-D beam problems (Appendix B). (3) Development of an equivalent plate-model analysis method (EPA) for static and vibration analysis of general trapezoidal built-up wing structures composed of skins, spars and ribs. Calculation of all sorts of test cases and comparison with measurements or FEA results. (Appendix C). (4) Basic work on using second order sensitivities on simulating wing modal response, discussion of sensitivity evaluation approaches, and some results (Appendix D). (5) Establishing a general methodology of simulating the modal responses by direct application of NN and by sensitivity techniques, in a design space composed of a number of design points. Comparison is made through examples using these two methods (Appendix E). (6) Establishing a general methodology of efficient analysis of complex wing structures by indirect application of NN: the NN-aided Equivalent Plate Analysis. Training of the Neural Networks for this purpose in several cases of design spaces, which can be applicable for actual design of complex wings (Appendix F).

Kapania, Rakesh K.↗
NASA NTRS2003

Experimental Evaluation of Verification and Validation Tools on Martian Rover Software

To achieve its science objectives in deep space exploration, NASA has a need for science platform vehicles to autonomously make control decisions in a time frame that excludes intervention from Earth-based controllers. Round-trip light-time is one significant factor motivating autonomy capability, another factor is the need to reduce ground support operations cost. An unsolved problem potentially impeding the adoption of autonomy capability is the verification and validation of such software systems, which exhibit far more behaviors (and hence distinct execution paths in the software) than is typical in current deepspace platforms. Hence the need for a study to benchmark advanced Verification and Validation (V&V) tools on representative autonomy software. The objective of the study was to access the maturity of different technologies, to provide data indicative of potential synergies between them, and to identify gaps in the technologies with respect to the challenge of autonomy V&V. The study consisted of two parts: first, a set of relatively independent case studies of different tools on the same autonomy code, second a carefully controlled experiment with human participants on a subset of these technologies. This paper describes the second part of the study. Overall, nearly four hundred hours of data on human use of three different advanced V&V tools were accumulated, with a control group that used conventional testing methods. The experiment simulated four independent V&V teams debugging three successive versions of an executive controller for a Martian Rover. Defects were carefully seeded into the three versions based on a profile of defects from CVS logs that occurred in the actual development of the executive controller. The rest of the document is structured a s follows. In section 2 and 3, we respectively describe the tools used in the study and the rover software that was analyzed. In section 4 the methodology for the experiment is described; this includes the code preparation, seeding of defects, participant training and experimental setup. Next we give a qualitative overview of how the experiment went from the point of view of each technology; model checking (section 5), static analysis (section 6), runtime analysis (section 7) and testing (section 8). The find section gives some preliminary quantitative results on how the tools compared.

Brat, Guillaume↗
DOE OSTI2024

Screening Cu-Zeolites for Methane Activation Using Curriculum-Based Training

Machine learning (ML), when used synergistically with atomistic simulations, has recently emerged as a powerful tool for accelerated catalyst discovery. However, the application of these techniques has been limited by the lack of interpretable and transferable ML models. In this work, we propose a curriculum-based training (CBT) philosophy to systematically develop reactive machine learning potentials (rMLPs) for high-throughput screening of zeolite catalysts. Our CBT approach combines several different types of calculations to gradually teach the ML model about the relevant regions of the reactive potential energy surface. The resulting rMLPs are accurate, transferable, and interpretable. We further demonstrate the effectiveness of this approach by exhaustively screening thousands of [CuOCu] 2+ sites across hundreds of Cu-zeolites for the industrially relevant methane activation reaction. Specifically, this large-scale analysis of the entire International Zeolite Association (IZA) database identifies a set of previously unexplored zeolites (i.e., MEI, ATN, EWO, and CAS) that show the highest ensemble-averaged rates for [CuOCu] 2+ -catalyzed methane activation. We believe that this CBT philosophy can be generally applied to other zeolite-catalyzed reactions and, subsequently, to other types of heterogeneous catalysts. Thus, this represents an important step toward overcoming the long-standing barriers within the computational heterogeneous catalysis community.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Fast Gaussian Process Estimation for Large-Scale In Situ Inference using Convolutional Neural Networks

Exascale computing will bring with it significant I/O limitations. One foreseeable consequence of such restrictions is that the user can save only a small fraction of complex simulation data to disk for subsequent analysis. An alternative is to fit statistical models to data in situ, that is, inside the simulation as it runs. This option requires extremely fast statistical estimation to avoid slowing down the simulation. Gaussian processes (GPs) have state-of-the-art predictive performance for modeling spatial data. However, standard estimation methods for GPs scale quite poorly to large data sets as parameter estimation requires inverting a covariance matrix to the size of the data set. In the presented work, we use a convolutional neural network (CNN) to predict the GP parameters for a spatial data set, from a simulation or otherwise, rather than optimize the parameters directly. Here, our presented case study models spatial data from E3SM, the Department of Energy’s Exascale climate model. The CNN is trained on synthetic data simulated from GP models with known parameters and then applied to data from the climate simulation. In the presented examples, the neural network scheme produces parameter estimates that compare well with standard methods such as maximum likelihood estimation in predictive performance but is obtained four orders of magnitude faster.

big data↗
NASA NTRSDate not supplied

Combining Data with Physical Knowledge for Uncertainty Quantification in Certification and Reliability Analysis

Unifying empirical data with predictive models can enable engineering cost-savings through certification by analysis and reliability-based design. Both concepts require rigorous uncertainty quantification (UQ) and robust understanding and treatment of relevant physics. Combining sampling-based UQ algorithms with high-fidelity simulations creates a computational bottleneck that is often alleviated through the use of machine learning (ML). ML can be used to create computationally efficient surrogates for simulations of complex or high-dimensional physical interactions (e.g., multi-phase interactions associated with melt pools in laser powder bed fusion or spatially-dependent material properties in functionally graded materials). However, negative side effects of ML may include a lack of interpretability and negative correlation between event rarity and simulation accuracy due to a lack of training data. As such, it is important to infuse ML algorithms with physics-based guardrails to provide confidence in their predictions. This talk will provide a brief review of recent NASA research at this intersection of physics-based simulation, ML, and UQ with a focus on certification and reliability analysis.

uncertainty quantification↗
DOE OSTI2026

Stochastic Thermo-Hydro Modeling and Neural Network Surrogate Development for Thermal Resource Assessment of the Galleries-to-Calories Geobattery

The Galleries-to-Calories Geobattery concept explores the use of abandoned coal mine workings for large-scale thermal energy transport and storage. The system involves injecting waste heat from a supercomputing facility into flooded mine galleries, where groundwater flow can store and transport thermal energy for potential recovery in downgradient district heating and cooling applications. To evaluate the feasibility and performance of the Geobattery under geological and operational uncertainty, we developed a suite of stochastic thermo-hydrological (TH) simulations using Monte Carlo sampling of key uncertain parameters (e.g., permeability, porosity, thermal conductivity, specific heat capacity) and operating conditions (e.g., injection rate, injection temperature). Results identified injection rate and temperature as the most influential parameters governing thermal front propagation, while the geometry of the room-and-pillar structure played a critical role in directing the extent and orientation of thermal advancement. Optimal combinations of material properties for maximizing heat recovery were also determined. To address the high computational cost of coupled-process stochastic modeling, we trained a neural network surrogate model on 24,000 physics-based realizations, achieving an R² > 0.99 and MAE < 0.1 for temperature predictions at monitoring locations. This surrogate enabled an additional 100,000 realizations for global sensitivity analysis and probabilistic thermal resource assessment. The integrated stochastic physics–surrogate modeling framework offers a computationally efficient tool for quantifying uncertainty, identifying key drivers, and informing early-stage design decisions for Geobattery systems.

15 - GEOTHERMAL ENERGY↗
DOE OSTI2014

Breaking Computational Barriers: Real-time Analysis and Optimization with Large-scale Nonlinear Models via Model Reduction

Model reduction for dynamical systems is a promising approach for reducing the computational cost of large-scale physics-based simulations to enable high-fidelity models to be used in many- query (e.g., Bayesian inference) and near-real-time (e.g., fast-turnaround simulation) contexts. While model reduction works well for specialized problems such as linear time-invariant systems, it is much more difficult to obtain accurate, stable, and efficient reduced-order models (ROMs) for systems with general nonlinearities. This report describes several advances that enable nonlinear reduced-order models (ROMs) to be deployed in a variety of time-critical settings. First, we present an error bound for the Gauss-Newton with Approximated Tensors (GNAT) nonlinear model reduction technique. This bound allows the state-space error for the GNAT method to be quantified when applied with the backward Euler time-integration scheme. Second, we present a methodology for preserving classical Lagrangian structure in nonlinear model reduction. This technique guarantees that important properties--such as energy conservation and symplectic time-evolution maps--are preserved when performing model reduction for models described by a Lagrangian formalism (e.g., molecular dynamics, structural dynamics). Third, we present a novel technique for decreasing the temporal complexity --defined as the number of Newton-like iterations performed over the course of the simulation--by exploiting time-domain data. Fourth, we describe a novel method for refining projection-based reduced-order models a posteriori using a goal-oriented framework similar to mesh-adaptive h -refinement in finite elements. The technique allows the ROM to generate arbitrarily accurate solutions, thereby providing the ROM with a 'failsafe' mechanism in the event of insufficient training data. Finally, we present the reduced-order model error surrogate (ROMES) method for statistically quantifying reduced- order-model errors. This enables ROMs to be rigorously incorporated in uncertainty-quantification settings, as the error model can be treated as a source of epistemic uncertainty. This work was completed as part of a Truman Fellowship appointment. We note that much additional work was performed as part of the Fellowship. One salient project is the development of the Trilinos-based model-reduction software module Razor , which is currently bundled with the Albany PDE code and currently allows nonlinear reduced-order models to be constructed for any application supported in Albany. Other important projects include the following: 1. ROMES-equipped ROMs for Bayesian inference: K. Carlberg, M. Drohmann, F. Lu (Lawrence Berkeley National Laboratory), M. Morzfeld (Lawrence Berkeley National Laboratory). 2. ROM-enabled Krylov-subspace recycling: K. Carlberg, V. Forstall (University of Maryland), P. Tsuji, R. Tuminaro. 3. A pseudo balanced POD method using only dual snapshots: K. Carlberg, M. Sarovar. 4. An analysis of discrete v. continuous optimality in nonlinear model reduction: K. Carlberg, M. Barone, H. Antil (George Mason University). Journal articles for these projects are in progress at the time of this writing.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1995

O/S analysis of conceptual space vehicles

The application of recently developed computer models in determining operational capabilities and support requirements during the conceptual design of proposed space systems is discussed. The models used are the reliability and maintainability (R&M) model, the maintenance simulation model, and the operations and support (O&S) cost model. In the process of applying these models, the R&M and O&S cost models were updated. The more significant enhancements include (1) improved R&M equations for the tank subsystems, (2) the ability to allocate schedule maintenance by subsystem, (3) redefined spares calculations, (4) computing a weighted average of the working days and mission days per month, (5) the use of a position manning factor, and (6) the incorporation into the O&S model of new formulas for computing depot and organizational recurring and nonrecurring training costs and documentation costs, and depot support equipment costs. The case study used is based upon a winged, single-stage, vertical-takeoff vehicle (SSV) designed to deliver to the Space Station Freedom (SSF) a 25,000 lb payload including passengers without a crew.

Ebeling, Charles E.↗
DOE OSTI2018

Machine Learning–Based Reduce Order Crystal Plasticity Modeling for ICME Applications

Abstract Crystal plasticity simulation is a widely used technique for studying the deformation processing of polycrystalline materials. However, inclusion of crystal plasticity simulation into design paradigms such as integrated computational materials engineering (ICME) is hindered by the computational cost of large-scale simulations. In this work, we present a machine learning (ML) framework using the material information platform, Open Citrination, to develop and calibrate a reduced order crystal plasticity model for face-centered cubic (FCC) polycrystalline materials, which can be both rapidly exercised and easily inverted. The reduced order model takes crystallographic texture, constitutive model parameters, and loading condition as inputs and returns the stress-strain curve and final texture. The model can also be inverted and take a stress-strain curve, loading condition, and final texture as inputs and return the initial texture and constitutive model parameters as outputs. Principal component analysis (PCA) is used to develop an efficient description of the crystallographic texture. A viscoplastic self-consistent (VPSC) crystal plasticity solver is used to create the training data by modeling the stress-strain behavior and evolution of texture during deformation processing.

Yuan, Mengfei↗
DOE OSTI2023

A Deep Potential model for liquid–vapor equilibrium and cavitation rates of water

Computational studies of liquid water and its phase transition into vapor have traditionally been performed using classical water models. Here, we utilize the Deep Potential methodology—a machine learning approach—to study this ubiquitous phase transition, starting from the phase diagram in the liquid–vapor coexistence regime. The machine learning model is trained on ab initio energies and forces based on the SCAN density functional, which has been previously shown to reproduce solid phases and other properties of water. Here, we compute the surface tension, saturation pressure, and enthalpy of vaporization for a range of temperatures spanning from 300 to 600 K and evaluate the Deep Potential model performance against experimental results and the semiempirical TIP4P/2005 classical model. Moreover, by employing the seeding technique, we evaluate the free energy barrier and nucleation rate at negative pressures for the isotherm of 296.4 K. Further, we find that the nucleation rates obtained from the Deep Potential model deviate from those computed for the TIP4P/2005 water model due to an underestimation in the surface tension from the Deep Potential model. From analysis of the seeding simulations, we also evaluate the Tolman length for the Deep Potential water model, which is (0.091 ± 0.008) nm at 296.4 K. Finally, we identify that water molecules display a preferential orientation in the liquid–vapor interface, in which H atoms tend to point toward the vapor phase to maximize the enthalpic gain of interfacial molecules. We find that this behavior is more pronounced for planar interfaces than for the curved interfaces in bubbles. This work represents the first application of Deep Potential models to the study of liquid–vapor coexistence and water cavitation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

SUNBIRD : a simulation-based model for full-shape density-split clustering

Combining galaxy clustering information from regions of different environmental densities can help break cosmological parameter degeneracies and access non-Gaussian information from the density field that is not readily captured by the standard two-point correlation function (2PCF) analyses. However, modelling these density-dependent statistics down to the non-linear regime has so far remained challenging. We present a simulation-based model that is able to capture the cosmological dependence of the full shape of the density-split clustering (DSC) statistics down to intra-halo scales. Our models are based on neural-network emulators that are trained on high-fidelity mock galaxy catalogues within an extended-ΛCDM framework, incorporating the effects of redshift-space, Alcock–Paczynski distortions, and models of the halo–galaxy connection. Our models reach sub-percent level accuracy down to $1 \, h^{-1}\text{Mpc}$ and are robust against different choices of galaxy–halo connection modelling. When combined with the galaxy 2PCF, DSC can tighten the constraints on ω cdm , σ 8 , and n s by factors of 2.9, 1.9, and 2.1, respectively, compared to a 2PCF-only analysis. DSC additionally puts strong constraints on environment-based assembly bias parameters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Measurement of the muon neutrino charged-current mesonless cross section in the NOvA near detector

NOvA is a long-baseline accelerator neutrino experiment at Fermilab. Its physics goals include precision neutrino oscillation measurements, neutrino interaction cross-section measurements and beyond Standard Model explorations. We present a measurement of muon neutrino charged-current cross section with zero mesons in the final state at the NOvA near detector. This measurement is performed as a function of the kinematics of the final state muon. Our chosen interaction channel is especially sensitive to quasielastic and meson exchange current interactions and it provides handles for constraining the cross section systematic uncertainties in oscillation analyses in present and future experiments. For particle identification, we use a convolutional neural network (CNN) trained on individual particles simulated in the NOvA Near detector. This allows us to select the desired signal while reducing the potential bias from neutrino interaction modeling. We study strategies for constraining the remaining charged-pion background via Michel electron information in a template fitting approach. The main experimental result is a two-dimensional differential cross section as a function of final-state muon kinetic energy and polar angle. The parameters of this measurement, including binning and unfolding, were optimized to reduce the expected systematic uncertainty in the total cross section. The final result shows good agreement with the main GENIE-based simulation framework that was independently fine-tuned in NOvA. We finally propose improvements and subsequent steps that build on this analysis and further dissect the final states of neutrino interactions. This work has been supported by US DOE grant DE-SC0015684.

Sánchez Falero, Sebastián Jesús [Iowa State U.]↗
DOE OSTI2017

Exploring Electric Polarization Mechanisms in Multiferroic Oxides

Multiferroic oxides are a class of systems which exhibit coupling between the electrical polarization and the magnetization. These materials show promise to lead to devices in which ferromagnetic memory can be written with magnetic fields or magnetic bits can be written by an electric field. The work conducted in our research focuses on single phase materials. We studied the detailed coupling of the spin and lattice correlations in these systems. In the first phase of the proposal, we explored the complex spin spiral systems and low temperature behavior of hexagonal layered REMnO 3 (RE= rare earth, Y and Sc) system following the detailed structural changes which occurred on crossing into the magnetic states. The techniques were applied to other layered materials such as superconductors and thermoelectric where the same layered motif exists. The second phase of the proposal focused on understanding the mechanisms involved in the onset high temperature ferroelectricity ion hexagonal REMnO 3 and at low temperature in E-Type magnetic ordered perovskite REMnO 3 . We wsynthesized preovskite small A site multiferroics by high pressure and high temperature methods. Detailed measurement of the structural properties and dynamics were conducted over a range of length scales from atomic to mesoscopic scale using, x-ray absorption spectroscopy, x-ray diffuse scattering, x-ray and neutron pair distribution analysis and high resolution x-ray diffraction. Changes in vibration modes which occur with the onset of polarization were probed with temperature and pressure dependent infrared absorption spectroscopy. In addition the orthorhombic system (small radius RE ions) which is believed to exhibit electronically driven ferroelectricity and is also not understood was examined. The multiple length scale synchrotron based measurements may assist in developing more detailed models of these materials and possibly lead to device applications. The experimental work was complemented by density functional methods to determine the magnetic ground states and ab initio molecular dynamics methods (AIMD) to determine the high temperature structures. Simulation were carried out on supercomputers at the National Energy Research Scientific Computing Center (NERSC). An important contribution of this work was the training of graduate students and postdoctoral researchers in materials synthesis, high pressure methods and synchrotron based spectroscopy and x-ray scattering techniques.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2012

Reduced-Order Modeling for Flutter/LCO Using Recurrent Artificial Neural Network

The present study demonstrates the efficacy of a recurrent artificial neural network to provide a high fidelity time-dependent nonlinear reduced-order model (ROM) for flutter/limit-cycle oscillation (LCO) modeling. An artificial neural network is a relatively straightforward nonlinear method for modeling an input-output relationship from a set of known data, for which we use the radial basis function (RBF) with its parameters determined through a training process. The resulting RBF neural network, however, is only static and is not yet adequate for an application to problems of dynamic nature. The recurrent neural network method [1] is applied to construct a reduced order model resulting from a series of high-fidelity time-dependent data of aero-elastic simulations. Once the RBF neural network ROM is constructed properly, an accurate approximate solution can be obtained at a fraction of the cost of a full-order computation. The method derived during the study has been validated for predicting nonlinear aerodynamic forces in transonic flow and is capable of accurate flutter/LCO simulations. The obtained results indicate that the present recurrent RBF neural network is accurate and efficient for nonlinear aero-elastic system analysis

Yao, Weigang↗
DOE OSTI2018

Data-Driven Wind Farm Optimization Incorporating Effects of Turbulence Intensity

We demonstrate assimilation of field data from a nacelle-mounted lidar and meteorological tower into a medium-fidelity Reynolds-Averaged Navier Stokes wind farm flow model to better predict the effects of atmospheric stability. Increased predictability under a variety of atmospheric conditions can lead to more effective control design and optimization of a wind farm. In particular, atmospheric stability affects wind turbine wake propagation and, therefore, aspects of wind farm control and performance, such as active wind farm control, layout optimization, and power output. Accurately modeling wakes in different stability conditions remains a persistent challenge. This paper presents an optimization framework that leverages high-fidelity field or simulation data to correct a lower-fidelity flow model. Optimal model corrections are found by solving a regularized, high-dimensional, gradient-based optimization problem using adjoint flowfield information. We validate the trained model against large eddy simulation results, and perform separate gradient-based layout optimizations of a simulated utility-scale wind farm to maximize power. Using the data-driven model corrections, we find that atmospheric stability significantly impacts layout optimization and power production: the optimal layout for stable conditions produced 9.1% more power than the optimal layout for unstable conditions, and the optimal layout for neutral conditions underperformed by 8.5% in unstable conditions.

17 WIND ENERGY↗
DOE OSTI2022

RLC4CLR (Reinforcement Learning Controller for Critical Load Restoration Problems)

RLC4CLR demonstrates using a reinforcement learning controller (RLC) to solve a critical load restoration (CLR) problem, which improves the grid resilience after a substation outage event. RLC4CLR consists of two parts. (1) RL environment: This environment encapsulates the CLR problem to be solved and provides interfacing functions to follow the standard OpenAI Gym format. A power system simulator, i.e., OpenDSS, is included to provide the power flow solution. Controller inputs and outputs (RL state and action) as well as the reward are defined in this environment as well. In summary, the RL environment is the problem formulation from which the RL agent can learn. (2) RL training script: The training script enables the RL agent to learn its control policy by interacting with the RL environment. For RL training, an open-sourced RL library, i.e., RLlib, is leveraged which is based on a distributed computing framework (Ray). The training script is designed to be able to be run on both local machine or the NREL HPC system. Other components of RLC4CLR include input data, e.g., grid model (standard IEEE test feeders), and other files used for results analysis.

Zhang, Xiangyu↗