Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed scheduling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗

AC Power Flow Based DLMP Calculation and Decomposition Method to Smooth Power Fluctuation of Distributed Renewable Energy Sources

As the penetration of renewable energy sources increases, the growing renewable power variability brings ramping issues to power systems. Meanwhile, the development of distributed energy resources (DERs) makes the distribution systems to provide both energy and ancillary services. To incentivise individual resources and customers to alleviate ramping issues on the demand side, a two-stage distribution locational marginal price (DLMP) calculation and decomposition method is developed to formulate the marginal power ramping price for DERs. In the first stage of the proposed method, a distribution system operator market scheduling model based on AC optimal power flow is designed to estimate the optimal operating point of the distribution system. Subsequently, the voltage and power flow constraints are linearised in stage two to calculate DLMP. Finally, based on the Lagrange function and sensitivity factors, DLMP is decomposed to the marginal costs for active/reactive power, voltage management, power loss and power variability. Case studies demonstrate that the proposed model can effectively smooth the power fluctuation and reduce the ramping flexibility requirements of distribution systems.

AC optimal power flow↗

Optimal pumping schedule with high-viscosity gel for uniform distribution of proppant in unconventional reservoirs

Hydrocarbon recovery from a hydraulic-fractured unconventional reservoir can be enhanced by replacing slickwater with non-Newtonian high-viscosity gel as a fracturing fluid. However, in the literature, the role of gel rheology on hydrocarbon production from a fractured reservoir is not very well understood. This lack of understanding makes it difficult for the industry to choose appropriate viscosity parameters of fracturing gels. Motivated by these limitations, first, we have developed a high-fidelity model of non-Newtonian fluid flow to simulate fracture propagation, fluid leak-off, and gel flowback processes to predict the oil production from a hydraulic fractured reservoir. Simulation results of the high-fidelity model show how fracture geometry, formation damage, and gel cleanup process varies with the fracturing fluid rheology. Therefore, the developed model is used to understand the effect of gel rheology on hydrocarbon production via a sensitivity analysis, which provides a set of rheological parameters that maximize the cumulative amount of oil production. Finally, utilizing this optimal high-viscosity gel, a model predictive controller is designed to obtain an optimal pumping schedule necessary for producing a target fracture geometry and proppant concentration. The closed-loop simulation results demonstrate that the obtained pumping schedule successfully achieved the desired fracture geometry and proppant concentration at the end of pumping, leading to the maximum oil production from an unconventional reservoir.

13 HYDRO ENERGY↗

On the Feasibility of Simulation-Driven Portfolio Scheduling for Cyberinfrastructure Runtime Systems

Runtime systems that automate the execution of applications on distributed cyberinfrastructures need to make scheduling decisions. Researchers have proposed many scheduling algorithms, but most of them are designed based on analytical models and assumptions that may not hold in practice. The literature is thus rife with algorithms that have been evaluated only within the scope of their underlying assumptions but whose practical effectiveness is unclear. It is thus difficult for developers to decide which algorithm to implement in their runtime systems.To obviate the above difficulty, we propose an approach by which the runtime system executes, throughout application execution, simulations of this very execution. Each simulation is for a different algorithm in a scheduling algorithm portfolio, and the best algorithm is selected based on simulation results. The main objective of this work is to evaluate the feasibility and potential merit of this portfolio scheduling approach, even in the presence of simulation inaccuracy, when compared to the traditional one-algorithm approach. We perform this evaluation via a case study in the context of scientific workflows. Our main finding is that portfolio scheduling can outperform the best one-algorithm approach even in the presence of relatively large simulation inaccuracies.

Casanova, Henri↗

A Modified Maximum Entropy Inverse Reinforcement Learning Approach for Microgrid Energy Scheduling

Increasing popularity of integrating distributed energy resources (DERs) into the power system brings a challenge to optimize the microgrid dispatch policy. The reinforcement learning methods suffer from a long-time problem with the theoretical assumption of the objective/reward function for the microgrid system. Although the traditional inverse reinforcement learning (IRL) approaches can solve this problem to some extent, they encounter a limitation of complex computations for state visitation frequency in the large and continuous state space. To alleviate this limitation, we propose a modified maximum entropy IRL (MMIRL) method to extract the reward function from the expert demonstrations for solving the microgrid energy scheduling problem. The proposed MMIRL algorithm is promising in recovering the reward function and learning the dispatch policy compared to conventional approaches. Case studies are performed in an energy arbitrage problem and a microgrid system with DERs. Results substantiate that the proposed MMIRL approach can learn the dispatch policy with more than 99% efficiency and outperforms other comparative methods.

artificial intelligence, reinforcement learning, m↗

An exploration of online-simulation-driven portfolio scheduling in Workflow Management Systems

Workflow Management Systems used to automate the execution of scientific workflow applications on parallel and distributed computing platforms must make scheduling decisions at runtime. A large number of workflow scheduling algorithms have been proposed in the literature, but often these algorithms are evaluated based on simplifying assumptions that may not hold in practice. Furthermore, published algorithm evaluation and/or comparison results are necessarily only for a subset of all possible scenarios, and thus may not include scenarios relevant to particular use-cases. Consequently, it is difficult for Workflow Management Systems (WMSs) developers to decide which scheduling algorithm should be implemented. To obviate this difficulty, one possible approach is to implement a portfolio of scheduling algorithms and select the most effective algorithm at runtime. One method for performing this selection is to run an online simulation for each algorithm in the portfolio. The algorithm that leads to the best performance, in simulation, is selected for future use. The above simulation-driven portfolio scheduling (SDPS) approach has been proposed in a few parallel and distributed computing contexts. The main objective of this work is to evaluate the feasibility and potential merit of SDPS if implemented in WMSs. Here we perform this evaluation using simulated WMS executions, where the simulations are instantiated from real-world platform and workflow configurations. Our main finding is that SDPS is on par with or outperforms an approach in which a single algorithm is used, where this algorithm is the one that performs best on average across all our experimental scenarios. Furthermore, we find that SDPS remains an attractive proposition even in the presence of high levels of simulation error and for simulators with relatively low levels of sophistication. In many of our experimental scenarios we find that mitigating simulation error at runtime can further improve performance. Finally, we show that simulation overhead can be made sufficiently low for SDPS to be feasible in practice.

97 MATHEMATICS AND COMPUTING↗

US 2023/0182605 A1 Network constraint energy management system for electric vehicle depot charging and scheduling

Network constraint energy management system for electric vehicle (EV) depot charging and scheduling. In an embodiment, a power schedule is received from an economic dispatch application for a charging depot comprising EV charging station(s) and distributed energy resource(s). The power schedule may be simulated on a distribution network model of the charging depot, according to load flow analysis, to determine whether any grid-code violations occur. In response to the detection of violation(s), a constraint may be generated for each violating node, and the economic dispatch application may be re-executed with the constraint(s) to produce a new power schedule, until no violations are detected. When not all load demand can be satisfied by the power schedule, a charging schedule may be adjusted to ensure that critical energy requirements are satisfied. The final power and charging schedules may be used to schedule and control power generation and charging in the charging depot.

Hafiz, Faeza↗

SMC 2021 : Analyzing Resource Utilization and User Behavior on Titan Supercomputer

Resource utilization statistics of submitted jobs on a supercomputer can help us understand how users from various scientific domains use HPC platforms and better design a job scheduler. We explore to generate insight regarding workload distribution and usage pattern domains from job scheduler trace, GPU failure information, and project-specific information collected from Titan supercomputer. Furthermore, we want to know how the scheduler performance varies over time and how the users' scheduling behavior changes following a system failure. These observations have the potential to provide valuable insight, which is helpful to prepare for system failures. These practices will help us develop and apply novel machine learning algorithms in understanding system behavior, requirement, and better scheduling of HPC systems. There are two datasets, RUR and GPU. RUR: This dataset is the job scheduler traces collected from the Titan supercomputerfrom 01/01/2015 to 07/31/2019 (2015.csv - 2019.csv). These were collected usingResource Utilization Report (RUR), a Cray-developed resource-usage data collectionand reporting system. It contains the usage information of its critical resources (CPU,Memory, GPU, and I/O) of each running job on Titan during that period [2]. ProjectAreas: Every job is associated with a project ID. TheProjectAreas.csvdatasetprovides a mapping of the project ID to its domain science. GPU: There have been some hardware-related issues in the GPUs in Titan that caused some GPUs to fail, sometimes irrecoverably during some job runs. This dataset provides information regarding these failures during the execution of the submitted jobs. GPUs on Titan are uniquely identified by a serial number (SN), and they are installed in a location. A GPU can be installed in a location, then removed from that location following a failure, and then re-installed in a different location after fixing the problem. If the failure can't be recovered, the GPU might be removed entirely from Titan. There are two prominent types of failures that resulted in the removal of GPUs from Titan: Double Bit Error (DBE) and Out of the Bus (OTB). The dataset (gc_full.csv) has the following fields: 1. SN : Serial number of a GPU 2. location : The location where it is installed 3. insert : The time when it was inserted into that location 4. remove : The time when it was removed from that location 5. duration : Amount of time the GPU spent in this location 6. out : If the device was taken out entirely w/o a re-installment into a new location. 7. event : If the GPU was taken out entirely, the reason for its removal. To learn more about this dataset, please refer to the git repositoryhttps://github.com/olcf/TitanGPULifeand the related publication [1]. References [1] George Ostrouchov, Don Maxwell, Rizwan A Ashraf, Christian Engelmann, MallikarjunShankar, and James H Rogers. Gpu lifetimes on titan supercomputer: Survival analysisand reliability. InSC20: International Conference for High Performance Computing,Networking, Storage and Analysis, pages 1-14. IEEE, 2020. [2] Feiyi Wang, Sarp Oral, Satyabrata Sen, and Neena Imam. Learning from five-yearresource-utilization data of titan system. In2019 IEEE International Conference onCluster Computing (CLUSTER), pages 1-6. IEEE, 2019.

42 ENGINEERING↗

SMC 2021 Data Challenge: Analyzing Resource Utilization and User Behavior on Titan Supercomputer

Resource utilization statistics of submitted jobs on a supercomputer can help us understand how users from various scientific domains use HPC platforms and better design a job scheduler. We explore to generate insight regarding workload distribution and usage pattern domains from job scheduler trace, GPU failure information, and project-specific information collected from Titan supercomputer. Furthermore, we want to know how the scheduler performance varies over time and how the users' scheduling behavior changes following a system failure. These observations have the potential to provide valuable insight, which is helpful to prepare for system failures. These practices will help us develop and apply novel machine learning algorithms in understanding system behavior, requirement, and better scheduling of HPC systems. There are two datasets, RUR and GPU: RUR dataset is the job scheduler traces collected from the Titan supercomputer from 01/01/2015 to 07/31/2019 (2015.csv - 2019.csv). These were collected using resource Utilization Report (RUR), a Cray-developed resource-usage data collection and reporting system. It contains the usage information of its critical resources (CPU, Memory, GPU, and I/O) of each running job on Titan during that period (https://ieeexplore.ieee.org/abstract/document/8891001). It includes ProjectAreas as additional information, every job is associated with a project ID. TheProjectAreas.csv dataset provides a mapping of the project ID to its domain science. GPU dataset has information regarding GPU failure on Titan. There have been some hardware-related issues in the GPUs in Titan that caused some GPUs to fail, sometimes irrecoverably during some job runs. This dataset provides information regarding these failures during the execution of the submitted jobs. GPUs on Titan are uniquely identified by a serial number (SN), and they are installed in a location. A GPU can be installed in a location, then removed from that location following a failure, and then re-installed in a different location after fixing the problem. If the failure can't be recovered, the GPU might be removed entirely from Titan. There are two prominent types of failures that resulted in the removal of GPUs from Titan: Double Bit Error (DBE) and Out of the Bus (OTB). The dataset (gc_full.csv) has seven attributes, we provided a short description of these attributes in the ReadMe file. To learn more about this dataset, please refer to the git repository https://github.com/olcf/TitanGPULife and the related publication (https://ieeexplore.ieee.org/abstract/document/9355319).

42 ENGINEERING↗

Managing Power Systems-Induced Wildfire Risks Using Optimal Scheduled Shutoffs

The increasing demands for electricity and the increase in extreme weather conditions are putting unprecedented pressure on our electric grids. Often, this pressure leads to electrical component failures, which might ignite wildfires. This work develops a novel model to balance the reliability of power network operations and the risk of wildfire ignition by opti- mizing the operational schedule of power transmission networks considering time-varying risk measures that include exogenous and operational factors. Energy storage systems are considered to deliver power during peak wildfire hours and enable temporal load shifting. The problem is formulated as a mixed-integer linear program that maximizes a weighted sum of the served power demand and the reduction in grid-induced wildfire risk. The results demonstrate the ability of the model to significantly reduce wildfire risk without considerable load shedding.

power systems operations↗

Co-optimization of repairs and dynamic network reconfiguration for improved distribution system resilience

In this work, a post-disaster distribution system repair and restoration (DSRR) strategy is proposed to improve distribution system resilience. The DSRR strategy is formulated as a two-stage optimization. The first stage is a comprehensive co-optimization of repair crew scheduling, dynamic network reconfiguration, and distributed energy resource (DER) dispatch based on the forecast load profile. The goal is to minimize the accumulative operating cost caused by the load reduction payment as well as DER operating cost. In particular, since the number of available repair crews is usually smaller than the number of faulted lines after a disaster event, the DSRR strategy determines the optimal scheduling for repairing faulted lines. The second stage is a re-dispatch of the DER power output and load shedding based on the real-time load demand of each bus. The proposed algorithm is validated by case studies of the IEEE 33-bus and 123-bus test systems. We consider those scenarios in which faults occur in multiple heavy-loaded feeders. The simulation results demonstrate that the DSRR strategy effectively coordinate the repair scheduling, network reconfiguration and load shedding to minimize the operating cost.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Prevalence of typical operational problems and energy savings opportunities in U.S. commercial buildings

In the United States, as much as 30% of the 19 EJ that commercial buildings consume is considered excess. Much of the excess energy is due to the inability to manage building operations efficiently. Because almost 20% of the total primary energy consumption is associated with commercial buildings, significant energy reductions in this sector are needed to mitigate climate change. Therefore, many cities and states are mandating periodic “tune-ups” of these buildings to eliminate excess energy consumption. Although the benefits of tune-ups and retro-commissioning are clear, focusing these mandates to look for specific opportunities has been a challenge because of the lack of studies that document the prevalence of opportunities. Therefore, we analyzed building automation system data from 151 buildings across the United States to document common operational problems and opportunities to improve building operations. This analysis showed that opportunities to improve building operations exist in almost every building. These opportunities were not strongly correlated with building vintage or size, but were reflective of how the buildings are operated. The prevalence of the top 20 opportunities ranged between 74% and 23%, with 40% of these associated with air-handling units. The rest of the opportunities are associated with schedules, chilled and hot-water distribution, and zone controls. Of the 151 buildings, 69 of them implemented corrective actions of some or all opportunities that were identified. Implementation varied across the Re-tuning categories, with 60% for schedule opportunities, 50% for zone opportunities, over 40% for the air-handling unit and hot-water opportunities, and 35% of the chilled-water opportunities. There was wide variation in whole building energy savings, ranging from 0 to 50% and 0 to 18 $/m2 with median percent annual whole building savings of 12% and median normalized annual cost savings of $1.75/m2. In addition to documenting these key findings, the paper provides a list of opportunities that can be automatically and continuously identified and corrected and offers a list of those opportunities that should be the focus of the mandates.

Katipamula, Srinivas↗

Tri-Level Scheduling Model Considering Residential Demand Flexibility of Aggregated HVACs and EVs Under Distribution LMP

Residential loads, especially heating, ventilation and air conditioners (HVACs) and electric vehicles (EVs), have great potentials to provide demand flexibility which is an attribute of grid-interactive efficient buildings (GEB). Under this new paradigm, first, EV and HVAC aggregator models are developed in this paper to represent the fleet of GEBs, in which the aggregated parameters are obtained based on a new approach of data generation and least squares parameter estimation (DG-LSPE), which can deal with heterogeneous HVACs. Then, a tri-level bidding and dispatching framework is established based on competitive distribution operation with distribution locational marginal price (DLMP). Furthermore, the first two levels form a bilevel model to optimize the aggregators’ payment and to represent the interdependency between load aggregators and the distribution system operator (DSO) using DLMP, and the third level is to dispatch the optimal load aggregation to all residents by the proposed priority list-based demand dispatching algorithm. Finally, case studies on a modified IEEE 33-Bus system illustrate three main technical reasons of payment reduction due to demand flexibility: load shift, DLMP step changes, and power losses. They can be used as general guidelines for better decision-making for future planning and operation of demand response programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimal Network Reconfiguration and Scheduling With Hardware-in-the-Loop Validation for Improved Microgrid Resilience

With the increased occurrence of various major extreme weather events, power outages and prompt power system restorations have recently drawn more attention to the resilience and recovery of power systems. From the perspective of a more resilient power delivery at the distribution grid, system restoration using network topology reconfiguration together with optimal scheduling of distributed energy resources are adopted in this paper. The proposed optimization model aims at minimizing the total load shedding cost and other operational costs, in which linearized topological constraints borrowed from graph theory and linearized DistFlow models are respectively used to maintain the radial network topology and power flow balance after system contingencies. To demonstrate the applicability of the proposed strategy, a real-world case study of a networked three-microgrid system in Adjuntas, Puerto Rico, is used with the consideration of different independent/interconnected microgrid scenarios, contingencies, and fairness settings. Furthermore, hardware-in-the-loop testing is conducted for the same three-microgrid network, where the closely matched results with the simulated ones have validated the effectiveness of the proposed restoration strategy, which is now ready to move one step forward towards field deployment. Finally, to test the proposed restoration strategy in a larger networked system, the modified IEEE-33 bus test distribution system is considered, and the results show a more resilient power delivery for critical loads under three and four line outages.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An MILP-Based Distributed Energy Management for Coordination of Networked Microgrids

An MILP-based distributed energy management for the coordination of networked microgrids is proposed in this paper. Multiple microgrids and the utility grid are coordinated through iteratively adjusted price signals. Based on the price signals received, the microgrid controllers (MCs) and distribution management system (DMS) update their schedules separately. Then, the price signals are updated according to the generation–load mismatch and distributed to MCs and DMS for the next iteration. The iteration continues until the generation–load mismatch is small enough, i.e., the generation and load are balanced under agreed price signals. Through the proposed distributed energy management, various microgrids and the utility grid with different economic, resilient, emission and socio-economic objectives are coordinated with generation–load balance guaranteed and the microgrid customers’ privacy preserved. In particular, a piecewise linearization technique is employed to approximate the augmented Lagrange term in the alternating direction method of multipliers (ADMM) algorithm. Thus, the subproblems are transformed into mixed integer linear programming (MILP) problems and efficiently solved by open-source MILP solvers, which would accelerate the adoption and deployment of microgrids and promote clean energy. The proposed MILP-based distributed energy management is demonstrated through various case studies on a networked microgrids test system with three microgrids.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Combining model-based and model-free methods for stochastic control of distributed energy resources

Modern distribution systems are experiencing a fast transformation with the growing penetration of distributed energy resources (DERs). Along with the economic and environmental benefits of DERs, challenges arise to address the uncertainties caused by their inherent volatility. If properly coordinated, however, DERs have the potential to provide the controllability that grid operators need. Here, we propose a hierarchical control framework that combines the model-based and model-free methods for stochastic DER control in distribution systems. The upper-level scheduler considers a chance-constrained optimal power flow problem (model-based) that schedules DER setpoints to minimize the operational cost and maintain the operating reserve. The lower-level distributed DER controllers absorb real-time disturbances and uncertainties using the extremum seeking control (model-free) to achieve grid objectives. The combination of model-based and model-free methods allows us to take the advantages of both methods to effectively manage the uncertainty in distribution systems. The proposed work is demonstrated on the IEEE 13-node feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Optimizing workplace charging facility deployment and smart charging strategies

This study introduces a workplace charging (WPC) optimization model that maximizes the total satisfied electric miles of employees’ plug-in electric vehicles, subject to a given annual budget. The model optimizes both planning decisions of charger number and power levels and operation decisions of charging spot assignment and charging schedule for the given temporal distribution of charging demands and varied electricity prices. Results of experiments based on national average travel data indicate that the actual WPC strategy varies by budget level. Through optimization, the strategy could reduce impacts of the varied electricity price by shifting charging schedules to periods when electricity prices are low. Also, the model is expanded to study the trade-off between providing WPC and addressing consequence of degraded charging service by including the per-mile shadow cost of unsatisfied charging demand. Finally, we observe that their relative competitiveness mainly depends on the actual shadow cost of WPC.

33 ADVANCED PROPULSION SYSTEMS↗

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows

This report details recent progress for the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows”. We refer to the project as IPPD/2, reflecting the 2017 renewal under expanded scope and partners In IPPD/2, we increased our research scope to include data motion. We are focusing on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. This new work on data motion will augment and complement IPPD/2’s research that focused on the computational aspects of tasks. We leverage and extend our existing tools and demonstrate our work on the Belle II workflow suite as well as on workflows from NSLS-II. The highlights of our work are as follows: Provenance for Workflows: Provenance is used to provide information enabling quality control, re-run computational workflows, and reproduce results. IPPD/2 has been building a scalable provenance management system that enables the capture of provenance from the high-level workflow through all relevant system levels in one integrated environment. Leveraging this work, our recent efforts have included using provenance as an enabling technique. Workload characterization: Leveraging provenance and analysis, we characterize data movement within network, storage, and memory over a variety of workloads. This characterization enables an understanding by performance analysts and application developers of the range of behaviors that could be expected. Performance Prediction for Workflows: The goal of modeling distributed workflows is to understand performance bottlenecks and enable more intelligent task scheduling to optimize selected metrics of interest (e.g., task throughput or output data rate). IPPD/2 has utilized both analytical and AI/ML modeling methodologies for performance modeling. Advanced Scheduling and Fault Modeling for Workflows: Scheduling of large-scale scientific workflows on geographically distributed resources is a challenging problem. To improve workflow throughput, we combined novel scheduling algorithms with task predictions from performance modeling and fault modeling. Dynamically Alleviating Bottlenecks in Workflows: Exploiting our provenance, analysis, and modeling efforts, we have explored and developed several techniques for dynamically detecting and alleviating bottlenecks in data movement. In particular, we have spent considerable effort demonstrating our techniques on production-like workflow configurations.

97 MATHEMATICS AND COMPUTING↗