SEARCH · Engineering Papers

Results for “software development framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2013

Management of the decommissioning of the Thetis reactor

The Thetis research reactor on the site of the Nuclear Sciences Institute of the Ghent University has been in operation from 1967 until December 2003. This light-water moderated graphite-reflected low-enriched uranium pool-type reactor has been used for various purposes e.g. the production of radioisotopes and activation analyses. During the first years its core power was 15 kW. In the early '70, a core enlargement allowed for operation at typically 150 kW, while the maximum was allowed to be 250 kW In September 2007, Ghent University entrusted to SCK-CEN the management of the back-end of the spent fuel and the decommissioning of the reactor. In 2010, the spent fuel was removed from the reactor and transported to Belgoprocess for cementation in 400 l drums and interim storage awaiting final disposal. This activity allows tackling the decommissioning of the reactor. The objective is to complete its decommissioning by the end of 2014. In the framework of the decommissioning of the Thetis reactor, SCK-CEN set-up the final decommissioning plan and the decommissioning licensing file. These documents include among others a radiological inventory of the reactor. The graphite moderator blocks, the control and the safety pates, the liner of the pool were modeled to assess the activation products (isotopic vector and intensity). At the end of the unloading of the reactor in 2010 a brief mapping of the equipment's and internals of the reactor pool was performed. In 2012, we realized a more detailed mapping. These results confirmed those performed earlier and allowed to confirm the assumptions made in the final decommissioning plan. We set-up the terms of reference for the first decommissioning phase of the reactor namely the dismantling of the reactor i.e. reactor pool, circuits and rabbit system, equipment's and ventilation ducts. The removal of asbestos is also included into this phase. We conducted the selection process and the awarding of this decommissioning job. We gained the decommissioning license in May 2012. We also prepared the software tool allowing managing the decommissioning project by updating the inventory and recording the progress, the characterization measurements and the material and waste production. This software allows also to trace all the material streams and to report to the Authorities. This software is a simplified release of the ones developed by SCK-CEN in the framework of other decommissioning projects like BR3 and Belgonucleaire. The dismantling of the reactor i.e. reactor pool, circuits and rabbit system, will be performed in 2013. In 2014, it is planned to map all the surfaces of the infrastructure to highlight residual contamination of floor, walls and ceiling. The contaminated surfaces will be decontaminated and controlled. The objective is to reach the free release of the reactor building and laboratories by the end of 2014. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2015

WE-G-BRA-02: SafetyNet: Automating Radiotherapy QA with An Event Driven Framework

Purpose: Quality assurance is an essential task in radiotherapy that often requires many manual tasks. We investigate the use of an event driven framework in conjunction with software agents to automate QA and eliminate wait times. Methods: An in house developed subscription-publication service, EventNet, was added to the Aria OIS to be a message broker for critical events occurring in the OIS and software agents. Software agents operate without user intervention and perform critical QA steps. The results of the QA are documented and the resulting event is generated and passed back to EventNet. Users can subscribe to those events and receive messages based on custom filters designed to send passing or failing results to physicists or dosimetrists. Agents were developed to expedite the following QA tasks: Plan Revision, Plan 2nd Check, SRS Winston-Lutz isocenter, Treatment History Audit, Treatment Machine Configuration. Results: Plan approval in the Aria OIS was used as the event trigger for plan revision QA and Plan 2nd check agents. The agents pulled the plan data, executed the prescribed QA, stored the results and updated EventNet for publication. The Winston Lutz agent reduced QA time from 20 minutes to 4 minutes and provided a more accurate quantitative estimate of radiation isocenter. The Treatment Machine Configuration agent automatically reports any changes to the Treatment machine or HDR unit configuration. The agents are reliable, act immediately, and execute each task identically every time. Conclusion: An event driven framework has inverted the data chase in our radiotherapy QA process. Rather than have dosimetrists and physicists push data to QA software and pull results back into the OIS, the software agents perform these steps immediately upon receiving the sentinel events from EventNet. Mr Keranen is an employee of Varian Medical Systems. Dr. Moran’s institution receives research support for her effort for a linear accelerator QA project from Varian Medical Systems. Other quality projects involving her effort are funded by Blue Cross Blue Shield of Michigan, Breast Cancer Research Foundation, and the NIH.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2023

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

A Statistical Testing Approach for Quantifying Software Reliability; Application to an Example System

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2016

A Statistical Testing Approach for Quantifying Software Reliability; Application to an Example System

The U.S. Nuclear Regulatory Commission (NRC) encourages the use of probabilistic risk assessment (PRA) technology in all regulatory matters, to the extent supported by the state-of-the-art in PRA methods and data. Although much has been accomplished in the area of risk-informed regulation, risk assessment for digital systems has not been fully developed. The NRC established a plan for research on digital systems to identify and develop methods, analytical tools, and regulatory guidance for (1) including models of digital systems in the PRA's of nuclear power plants (NPPs), and, (2) incorporating digital systems in the NRC's risk-informed licensing and oversight activities. Under NRC's sponsorship, Brookhaven National Laboratory (BNL) explored approaches for addressing the failures of digital instrumentation and control (I and C) systems in the current NPP PRA framework. Specific areas investigated included PRA modeling digital hardware , development of a philosophical basis for defining software failure , and identification of desirable attributes of quantitative software reliability methods . Based on the earlier research, statistical testing is considered a promising method for quantifying software reliability. It is widely recognized that software failures are due to the triggering of pre-existing defects by the software's operational environment. Software defects can arise from errors made in user requirements or coding errors introduced during the developmental process. The software's operational environment includes factors such as the time history of digital system inputs, communication interfaces, the internal state of the digital system, and external conditions. Thus, software reliability is a function of both the number of pre-existing defects and the presence of a triggering condition caused by the manner in which the software is used. In this paper, we describe a statistical software-testing approach for quantifying software reliability and applied it to the loop-operating control system (LOCS) of an experimental loop of the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL). The work involved collaboration between BNL and INL. The objectives of the study include: (1) Development of a statistical testing approach for estimating software failure probability on demand, the results of which are suitable for including in a probabilistic risk assessment (PRA); and, (2) Application of the approach to the LOCS to estimate its failure probability, and obtain insights into the feasibility, practicality, and usefulness of the estimation in models of digital systems for inclusion in nuclear power plants' PRAs. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

LLNL Response to the DOE ASCR RFI, "Stewardship of Software for Scientific and High-Performance Computing"

For decades, Lawrence Livermore National Laboratory (LLNL) has been engaged in significant research, development, and support for software to enable scientific computing and, particularly, the use of high performance computing (HPC) in the NNSA mission space. In particular, the move in the mid-1990’s to simulation as a leading component of stockpile stewardship through the ASCI and the successor ASC programs, as well as the need for reliable data acquisition and control software for the National Ignition Facility, have been important drivers in building expertise in production-quality software development at LLNL. LLNL has also been a leader in the DOE SciDAC FASTMath Institute and the DOE Exascale Computing Project (ECP), both of which have striven to make scientific computing software – in particular, the enabling technologies underpinning simulation capabilities – more widely adopted and sustainable. As such, we believe that our experience can inform the broader goal of software stewardship for scientific and high-performance computing. LLNL strongly supports the formation of a new DOE ASCR program element in software stewardship and sustainment. Historically, DOE ASCR has funded applied mathematics and computer science research that has led to the development of important new capabilities and algorithms that are expressed as artifacts in research software. Such frameworks, libraries, and tools have seldom been directly funded to address the important issues of code maintenance, documentation, robustness, and community building. Software engineering and support have typically been done on the side in support of the ASCR-driven research products. DOE funding priorities have been slow to recognize that good software engineering, the kind that ensures research investments have more adoption and longevity, requires significant resources. Based upon our experiences, we have prepared this response to highlight the concerns and issues we believe to be important as DOE ASCR considers its role in scientific software stewardship. We believe that role is important and will require a significant investment of new funding to legitimately support the technologies past and future DOE ASCR investments have and will produce to facilitate their uptake and adoption in the broader scientific computing community. Following a summary of our involvement in scientific software development, the remainder our response is organized around the nine topics specifically identified in the RFI.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Integrate FARM with PID controllers: IES Simulation Ecosystem Control System Development

The FARM (Feasible Actuator Range Modifier) software module is a component of the RAVEN-based FORCE framework for analysis of Integrated Energy Systems (IES). FARM was designed to support the HERON software module in the solution of the optimal dispatch problem for IES units. As the result of HERON-FARM dispatch simulation, the set-point trajectories are optimized to meet constraints on both the production variables (i.e., the variables to be optimized such as the electrical power, the hydrogen production rate, etc.) and the process variables tied to the service life of equipment (e.g., steam flowrate, vessel pressure, turbine firing temperature, etc.) at a coarse time resolution (every 10 or 100 seconds) over long time horizons (several days or weeks). In case the operational constraints need to be met at finer time resolution, the computational burden of HERON-FARM would linearly increase with the sampling rate, and sub-optimal solutions might be obtained. System responses characterized by overshoots and damped oscillations temporarily violating the imposed constraints might occur during abrupt power transients. In this report, a hierarchical control system architecture for the operation of the Dynamic Energy Technology and Integration Laboratory (DETAIL) facility constructed at INL was proposed. First, the preliminary studies on the proposed control strategy for operating the facility and the designed PI controllers were reviewed. In particular, the current approach for generating the set-point trajectories was studied, and its limits were identified. To this aim, the inclusion of a Supervisory Control layer embedding a modified version of the FARM algorithm for preserving the system safe operation over both long and real-time horizons was proposed. In this way, FARM would be applied twice, i.e., the original version (“FARM Validator”) aiding the solution of the power dispatch problem, and the modified version (“FARM Supervisory” coordinating the PI controllers to address the real-time control tasks. Despite the kernel of the two modules is the same algorithm, their roles, tasks, and capabilities are quite different. A detailed description of the role of FARM at addressing low-level control tasks is provided, along with tentative operational procedures for training the models embedded into the algorithm by using the collected experimental data.

42 ENGINEERING↗

DOE OSTI2013

AutoDrug: fully automated macromolecular crystallography workflows for fragment-based drug discovery

New software has been developed for automating the experimental and data-processing stages of fragment-based drug discovery at a macromolecular crystallography beamline. A new workflow-automation framework orchestrates beamline-control and data-analysis software while organizing results from multiple samples. AutoDrug is software based upon the scientific workflow paradigm that integrates the Stanford Synchrotron Radiation Lightsource macromolecular crystallography beamlines and third-party processing software to automate the crystallography steps of the fragment-based drug-discovery process. AutoDrug screens a cassette of fragment-soaked crystals, selects crystals for data collection based on screening results and user-specified criteria and determines optimal data-collection strategies. It then collects and processes diffraction data, performs molecular replacement using provided models and detects electron density that is likely to arise from bound fragments. All processes are fully automated, i.e. are performed without user interaction or supervision. Samples can be screened in groups corresponding to particular proteins, crystal forms and/or soaking conditions. A single AutoDrug run is only limited by the capacity of the sample-storage dewar at the beamline: currently 288 samples. AutoDrug was developed in conjunction with RestFlow, a new scientific workflow-automation framework. RestFlow simplifies the design of AutoDrug by managing the flow of data and the organization of results and by orchestrating the execution of computational pipeline steps. It also simplifies the execution and interaction of third-party programs and the beamline-control system. Modeling AutoDrug as a scientific workflow enables multiple variants that meet the requirements of different user groups to be developed and supported. A workflow tailored to mimic the crystallography stages comprising the drug-discovery pipeline of CoCrystal Discovery Inc. has been deployed and successfully demonstrated. This workflow was run once on the same 96 samples that the group had examined manually and the workflow cycled successfully through all of the samples, collected data from the same samples that were selected manually and located the same peaks of unmodeled density in the resulting difference Fourier maps.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2021

Pronghorn: A Multidimensional Coarse Mesh Application for Advanced Reactor Thermal-Hydraulics

This paper presents an overview of Pronghorn, a multiscale thermal-hydraulic (T/H) application developed by Idaho National Laboratory and the University of California, Berkeley. Pronghorn, built on the open-source finite element Multiphysics Object-Oriented Simulation Environment (MOOSE), leverages state-of-the-art physical models, numerical methods, and nonlinear solvers to deliver fast-running advanced reactor T/H simulation capabilities within a modern software engineering environment. This work summarizes the physical models, multiphysics and multiscale coupling, and numerical discretization in Pronghorn with emphasis on our initial target application to pebble bed reactors (PBRs). A diverse set of applications are shown to depressurized natural circulation in the SANA experiments, forced convection in the Pebble Bed Modular Reactor, three-dimensional (3-D)/one-dimensional coupling of Pronghorn and RELAP-7 systems T/H for loop analysis in the High Temperature Reactor Power Module, and forced convection in the Mark-1 Pebble Bed Fluoride-Salt-Cooled High-Temperature Reactor. A multiphysics coupling of Pronghorn, RELAP-7, and Griffin deterministic neutronics for a gas-cooled PBR demonstrates the capability of the MOOSE framework for reactor design calculations. These applications highlight the verification and validation underlying Pronghorn’s software development while emphasizing features that improve upon capabilities offered by legacy tools in areas such as 3-D unstructured meshing, physics modeling, and multiphysics coupling.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

WindWatts Computational Framework and Web UI [SWR-20-100]

This software provides a collection of algorithms, an API, and a functional Web UI for the Distributed Wind’s WindWatts project. DW WindWatts is a DOE WETO-funded project aimed at the development of tools to supporting the distributed wind industry, particularly with respect to siting and resource assessment. The computational framework and the back-end API are powering the easy to use front-end services available at: https://windwatts.nrel.gov. https://github.com/NREL/dw-tap-api; https://github.com/NREL/dw-tap; https://github.com/NREL/windwatts-data For reference, this software was previously known as DW TAP Computational Framework.

Phillips, Caleb↗

DOE OSTI2024

Physics-Informed Learning Machines for Multiscale and Multiphysics Problems (PHILMS) (Technical Report)

The research work at University of California Santa Barbara (UCSB) resulted in several new developments in the areas of scientific machine learning, numerical analysis, and practical methods for data-driven modeling, prediction, reductions, and simulation. Many of the projects were carried out in collaboration with members of the national laboratories at Sandia National Laboratories (SNL), Pacific Northwestern National Laboratories (PNNL), and other institutions. Results included developing new scientific machine learning methods, related theory and mathematical frameworks for analysis and training, data-driven numerical solvers, and related tools and software for scientific computation. During the support period, over 16+ papers were submitted for publication, and 4 open-source software packages were developed and released (available at http://atzberger.org/). In addition, 7+ students and 2 post-docs were mentored in collaboration with the laboratory staff for future careers in academia, government labs, and industry.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Quantum Networks for Resilient Power Grids: Theory and Simulated Evaluation

Quantum networks are considered the future of secure communication in the coming quantum era. Yet there lack significant efforts on developing practical quantum networks for power grids. Here, in this paper, we establish a quantum network-based power grid (QNetGrid) framework and develops a real-time, reliable, flexible, programmable, and cost-effective QNetGrid software testbed containing repeater-based quantum communication, quantum routing, real software-defined networking (SDN) switches, and real-time networked microgrids (NMs) operations. It makes the following contributions: 1) a repeater and routing based quantum network simulator (QNSim) is developed, 2) repeaters with and without quantum memories are respectively simulated in QNSim, 3) different routing scenarios in QNetGrid are investigated, 4) a real-time QNetGrid software testbed is built in RTDS incorporating QNSim and real SDN switches, and 5) various test cases are designed, and experimental results produced with the QNetGrid testbed provide valuable insights for building quantum networks in power grids.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Real Vector Framework

SAND2025-11463O The Real Vector Framework (RVF) is a modern and flexible C++ vector math library for developing scientific computing software that involves vector computations. RVF allows an opt-in approach to functionality that parallels the familiar base-class and override structures of object-oriented programming. Users can reuse and customize the code without inheritance entanglements and dynamic dispatch, while enabling seamless interoperability between diverse container types. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

von Winckel, Gregory [Sandia National Lab. (SNL-CA↗

DOE OSTI2015

D and D knowledge management information tool feasibility study for cross-platform mobile applications - 15704

The D and D Knowledge Management Information tool (D and D KM- IT) is a web-based knowledge management information tool built for the D and D user community. A feasibility study is being conducted on cross-platform application frameworks for their viability to the D and D KM-IT mobile needs. Currently, three main Operating Systems comprise the majority of the mobile market IOS, Android, and Windows, requiring a separate application for each device. Cross-development software allows the development of one application using one language, that can be ported or deployed on multiple devices with little effort. Using Xamarin, a native cross-platform framework, create a shared mobile application using Visual Studio. Build a D and D KMIT mobile application designed for the three major mobile platforms. Use Xamarin to compile the application into native applications for the three target platforms. Build and deploy the application on all three mobile platforms. Perform benchmark tests of the developed applications to analyze the performance on the three platforms. A large portion of the business logic of the application has been shared, eliminating development redundancy and minimizing planning, design, and validation time. The user interface development could not fully be shared cross-platform and some platform specific code must be individually implemented for each device. Maintenance, testing and design phases of the software engineering process were reduced from one for each device to having just one shared cycle processes. A less diverse skill-set for developers is required with cross-platform development. Some processes within the development cycle still require redundant activities.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2019

Reactivity Feedback Modeling in SAM

System Analysis Module (SAM) is being developed at Argonne National Laboratory as a modern system-level modeling and simulation tool for advanced non-light water reactor safety analyses. It utilizes the object-oriented application framework MOOSE to leverage the modern software environment and advanced numerical methods. The capabilities of SAM are being extended to enable the transient modeling, analysis, and design of various advanced nuclear reactor systems. This report presents the development of Point-Kinetics modeling and reactivity feedback mechanisms in SAM. It also presents a new capability for predicting the thermal expansion in various structural components by coupling SAM with an external thermomechanics module (Tensor Mechanics) from the MOOSE framework. The analysis of the transient behavior of a nuclear reactor requires the coupled simulation of reactor kinetics and thermal-hydraulics of the reactor core, especially for those unprotected transients where the reactor scram system may not function properly. The point kinetics model has been widely used for reactor safety analysis due to its simplicity to capture the transient behavior of the reactor. Various reactivity feedback models have been developed and integrated with the Point-Kinetics module, including fuel axial expansion, core radial expansion, fuel Doppler, and coolant density reactivity. The reactivity feedback models in SAM are similar to the respective models used in SAS4A/SASSYS-1. This report first presents the brief theory of the Point-Kinetics module and reactivity feedback models. A number of verification tests have been performed where the code simulations are compared to the analytical model results. The reactivity feedback due to the thermal deformation, such as the fuel axial expansion and core radial expansion, is important for SFR transient analysis. Simplified thermal expansion models for the fuel pin and reactor constrain system (e.g. grid plate) are developed and verified in SAM. Additionally, a coupling interface is developed to couple SAM with external thermomechanical analysis modules for more accurate predictions of the thermal expansion of different components during the transients. The current coupling interface has been tested with the Tensor Mechanics module from MOOSE. These point kinetics and reactivity feedback modeling capabilities have also been demonstrated by simulating the early stage of the unprotected loss-of-flow (ULOF) accident in the Advanced Burner Test Reactor (ABTR). Both the stand-alone SAM and coupled SAM and Tensor Mechanics simulations are performed. It is confirmed that the major physics phenomena in the heat transport system of the ABTR reactor are captured by SAM, and the point kinetics model, reactivity feedback models, and the coupling schemes are working well as expected.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2019

Reactivity Feedback Modeling in SAM

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2013

DASAO: software tool for the management of safeguards, waste and decommissioning

Decommissioning of nuclear facilities is a complex process involving operations such as detailed surveys, decontamination and dismantling of equipment's, demolition of buildings and management of resulting waste and nuclear materials if any. This process takes place in a well-developed legal framework and is controlled and followed-up by stakeholders like the Safety Authority, the Radwaste management Agency and the Safeguards Organism. In the framework of its nuclear waste and decommissioning program and more specifically the decommissioning of the BR3 reactor, SCK-CEN has developed different software tools to secure the waste and material traceability, to support the sound management of the decommissioning project and to facilitate the control and the follow-up by the stakeholders. In the case of Belgium, it concerns the Federal Agency for Nuclear Control, the National Agency for radioactive waste management and fissile material and EURATOM and IAEA. In 2005, Belgonucleaire decided to shutdown her Dessel MOX fuel fabrication plant and the production stopped in 2006. According to the final decommissioning plan ('PDF') approved by NIRAS, the decommissioning works should start in 2008 at the earliest. In 2006, the management of Belgonucleaire identified the need for an integrated database and decided to entrust SCK-CEN with its development, because SCK-CEN relies on previous experience in comparable applications namely already approved by authorities such as NIRAS, FANC and EURATOM. The main objectives of this integrated software tool are: - simplified and updated safeguards; - waste and material traceability; - computerized documentation; - support to project management; - periodic and final reporting to waste and safety authorities. The software called DASAO (Database for Safeguards, Waste and Decommissioning) was successfully commissioned in 2008 and extensively used from 2009 to the satisfaction of Belgonucleaire and the stakeholders. SCK-CEN is now implementing a simplified release of the software for the management of the decommissioning of the Thetis reactor. Its decommissioning will start in March 2013 and will be completed by the end of 2014. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2022

Hybrid-RL-MPC4CLR (Hybird-Reinforcement-Learning-Model-Predictive-Control-for-Reserve-Policy-Assisted-Critical-Load-Restoration-in-Distribution-Grids)

Hybrid-RL-MPC4CLR was developed as a hybrid controller for active distribution grid critical load restoration, combining deep reinforcement learning (RL) and model predictive control (MPC) aiming at maximizing total restored load following an extreme event. The RL determines a policy for quantifying operating reserve requirements, thereby hedging against uncertainty, while the MPC models grid operations incorporating the RL policy actions (i.e., reserve requirements), renewable (wind and solar) power predictions, and load demand forecasts. The developers formulated the reserve requirement determination problem as a sequential decision-making problem based on the Markov Decision Process (MDP) and design an RL learning environment based on the OpenAI Gym framework and MPC simulation. The RL agent reward and MPC objective function aim to maximize and monotonically increase total restored load and minimize load shedding and renewable power curtailment. The software is developed using various software packages in Python. The MPC's optimal power flow (OPF) model is implemented using the Pyomo package, the RL simulation environment is implemented using the MPC simulation with various scenarios of renewable energy and load demand profiles and power outage beginning times, based on the OpenAI Gym framework. The RL agent training is performed using the RLlib Ray package. The RL algorithm is trained offline using historical forecasts of renewable generation and load demand profiles. Simulation analysis and performance tests are conducted using a modified IEEE 13-bus distribution test feeder containing wind turbine, photovoltaic, microturbine, and battery.

Eseye, Abinet Tesfaye↗