Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed scheduling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Intelligent Partitioning based Fully Parallel AC Security-Constrained Optimal Power Flow

Today’s power grid is becoming more diverse and integrated with high-level distributed energy resources and smart control technologies that is creating a new set of grid management challenges in terms of large-scale, nonlinear, and non-convex problem modeling, complex and time-consuming computation, as well as difficult uncertainty handling. This project focused on solving a challenging multi-period security-constrained generation scheduling problem, which is of great importance for maximizing the social welfare of real-time dispatch, day-ahead market, as well as weekly planning of power systems. Our developed software explored parallel optimization algorithms for complex and realistic power system models, and develop fast, efficient, and robust grid optimization solutions on the high-performance computing platform that will enable increased grid economics, flexibility, resilience, as well as energy security in the United States.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Frontier Job-Centric Telemetry Dataset

Comprehensive analysis of high-performance computing (HPC) systems requires linking workload execution to system behavior. This kind of analysis is vital for diagnosing performance issues, managing capacity, detecting anomalous workloads, and understanding how applications interact with system hardware. This job-centric telemetry dataset unifies scheduler job records with node-level measurements, enabling direct association between workloads and their corresponding power, thermal, and performance characteristics. It contains sanitized, scheduler related metadata for 152,400 individual jobs that ran on the Frontier supercomputer and ended on selected days throughout 2024 and 2025, a subpopulation of ~6.8% of the total number of allocated jobs with non-zero run time on the system over that same period. Each is linked with files that contain telemetry time series records of the power utilization and temperature behavior of its allocated nodes and their processors during the run time of the job. Where available, a portion of the job files also contain network performance time series. Jobs are sampled from select days that reflect normal levels of user activity and possess job size distributions with large numbers of leadership class jobs (>20% of Frontier nodes). Jobs in this dataset attempt to best represent successful user workflows.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Asynchronous distributed-memory task-parallel algorithm for compressible flows on unstructured 3D Eulerian grids

Here, we discuss the implementation of a finite element method, used to numerically solve the Euler equations of compressible flows, using an asynchronous runtime system (RTS). The algorithm is implemented for distributed-memory machines, using stationary unstructured 3D meshes, combining data-, and task-parallelism on top of the Charm++ RTS. Charm++’s execution model is asynchronous by default, allowing arbitrary overlap of computation and communication. Task-parallelism allows scheduling parts of an algorithm independently of, or dependent on, each other. Built-in automatic load balancing enables continuous redistribution of computational load by migration of work units based on real-time CPU load measurement. The RTS also features automatic checkpointing, fault tolerance, resilience against hardware failure, and supports power-, and energy-aware computation. We demonstrate scalability up to 25 x 10 9 cells at $\mathscr{O}$10 4 compute cores and the benefits of automatic load balancing for irregular workloads. The full source code with documentation is available at https://quinoacomputing.org.

42 ENGINEERING↗

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

97 MATHEMATICS AND COMPUTING↗

Dynamic Risk-Aware Patch Scheduling

A program that defines and assesses the dynamic risk of software vulnerabilities and considers the dynamic risks into patch scheduling to reduce security risks posed by vulner­abilities and provide formal guidance to security operations at various organizations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Preparing Distribution Utilities for the Future - Unlocking Demand-Side Management Potential: A Novel Analytical Framework

The balance of supply and demand in the power systems has traditionally been served solely through generation and network capacity planning and operations. However, with increased requirements for flexibility due to the uptake in variable renewable generation sources such as wind and solar there is a need to increased demand-side flexibility. In addition, there are increased communications and flexibility capabilities emerging on the demand-side from the adoption of advanced metering infrastructures and smart meter deployment and intelligent loads such as smart thermostats and schedulable white goods (e.g. dishwashers and washing machines). Unlocking demand-side flexibility can bring system benefits from peak load reduction bringing about generation capacity and network upgrade deferral, to reducing demand and more efficient utilization of generation and network capacity. Unlocking demand-side flexibility is an evolving process for utilities and solutions must be tailored to each specific customer group. Demand-side management (DSM) is a broad set of tools that can include demand response (both dispatchable and non-dispatchable), energy efficiency and distributed energy resources and demand-side technologies. The National Renewable Energy Laboratory (NREL), in collaboration with BSES Rajdhani Power Ltd. (BRPL) and Deloitte, examined the potential of DSM in BRPL’s service territory, developing detailed information on customer classes and willingness to participate in DSM. The study developed modeling frameworks for load analysis and the analysis tools to assess the potential of time-of-use tariffs in motivating customers to reduce their peak period energy consumption. The study shows that BRPL customers, specifically their domestic customers, are willing to participate in DSM programs and that time-of-use pricing can help BRPL reduce their peak demand and help unlock demand-side flexibility.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Monitoring of corrosion effects in pipes with multi-mode acoustic signals

Current commercial long-range guided wave inspection and monitoring technologies for pipes usually require establishment of certain specific acoustic modes for wave propagation and use a large number of sensors in a certain geometric arrangement (for example, uniformly-spaced arrays of sensors on a collar, placed around a pipe). This paper, in contrast, presents a long-range corrosion detection method in pipes based on multi-mode guided acoustic waves, generated and received using simple, single-element sensors. In this technique, available guided wave modes in any interrogated zone of the structure are automatically down-selected by the medium, and changes in energy distribution over the selected modes are analyzed through signal processing to monitor material loss or addition. Here, to investigate the feasibility of the proposed method, we have performed both experimental and numerical simulation studies on 50.8-mm (2-in)-diameter schedule-40 carbon steel pipes. In these studies, a linear chirp signal is injected in compression mode, perpendicular to the surface of the pipe, generating multi-mode, multi-directional guided waves that travel within the pipe wall. Corrosion effects in the pipes are simulated numerically and experimentally. To reduce errors in the measurement technique due to changes in the local or global temperature of the measured media, a universal temperature compensation algorithm is introduced. The experimental results show good agreement with simulation results, indicating that the proposed multi-mode wave generation and detection in a pipe could be used for long-range detection and monitoring of corrosion effects, such as pipe wall losses or solid material deposits.

36 MATERIALS SCIENCE↗

Optimizing $\mathrm{ACRT}$ to reduce inclusion formation during the $\mathrm{VGF}$ growth of cadmium zinc telluride: II. Application to experiments

Here two different accelerated crucible rotation technique (ACRT) rotation schedules are assessed, via theoretical computations and growth experiments, according to their ability to reduce tellurium-rich inclusions during the vertical gradient freeze (VGF) of cadmium zinc telluride (CZT). A vigorous rotation schedule based on classical ACRT guidelines produces a well-mixed melt that is expected to reduce tellurium inclusions compared to growth without crucible rotation. In contrast, a new ACRT schedule, derived from model-based optimization for interface stability, employs slower rotation with longer periods and is predicted to further reduce inclusions. Growth experiments corroborate these expectations. Namely, both ACRT schedules result in crystals with inclusion size and volume significantly decreased from levels found in material grown with no rotation, and material grown using the computationally optimized ACRT parameters exhibits a median inclusion size that is smaller and with a sharper distribution than in material grown via classical ACRT.

36 MATERIALS SCIENCE↗

Secure Distributed Energy Resource Communications (sDERC)

The slides will be presented at a DOE CESER Peer Review for the Risk Management Tools and Technology (RMT) that provide an overview of the sDERC project. The slides discuss the project plan for this new project and how the results will be demonstrated at the conclusion of this project. The peer review is scheduled for August 27-29.

Chavez, Adrian R.↗

Solute diffusion behavior during heat treatment and its impact in Sc-microalloyed Al-Cu system

Scandium (Sc) is a promising microalloying element that enhances the strength and thermal stability of aluminum (Al) alloys. However, these benefits are not fully realized in aluminum-copper (Al-Cu) systems, and corrosion resistance often declines due to the complex phase evolution and diffusion behavior of Cu and Sc. Here, to clarify this, solute diffusion behavior and Cu-Sc interaction during heat treatment (HT) were investigated using in situ synchrotron-based transmission x-ray microscopy (TXM), wide-angle x-ray scattering (WAXS), and x-ray absorption near-edge structure (XANES) spectroscopy. The results, supported by electron microscopy, reveal strong Cu-Sc bonding that significantly impedes solute diffusion, leading to inhomogeneous solute distribution and non-uniform precipitation. Moreover, the Al-Cu-Sc eutectic phase exhibits high thermal stability, resisting dissolution even near the matrix liquidus. These findings quantitatively elucidate the sluggish diffusion kinetics of Cu and Sc, which can help redesign HT schedules to improve both mechanical properties and corrosion resistance in Sc-microalloyed Al-Cu alloys.

36 MATERIALS SCIENCE↗

Dynamic Distribution of High-Rate Data Processing from CERN to Remote HPC Data Centers

The prompt reconstruction of the data recorded from the Large Hadron Collider (LHC) detectors has always been addressed by dedicated resources at the CERN Tier-0. Such workloads come in spikes due to the nature of the operation of the accelerator and in special high load occasions experiments have commissioned methods to distribute (spill-over) a fraction of the load to sites outside CERN. The present work demonstrates a new way of supporting the Tier-0 environment by provisioning resources elastically for such spilled-over workflows onto the Piz Daint Supercomputer at CSCS. Furthermore, this is implemented using containers, tuning the existing batch scheduler and reinforcing the scratch file system, while still using standard Grid middleware. ATLAS, CMS and CSCS have jointly run selected prompt data reconstruction on up to several thousand cores on Piz Daint into a shared environment, thereby probing the viability of the CSCS high performance computer site as on demand extension of the CERN Tier-0, which could play a role in addressing the future LHC computing challenges for the high luminosity LHC.

97 MATHEMATICS AND COMPUTING↗

Biofoundries: Principles, Tools, and Applications

This chapter aims to provide a broad overview of biofoundries and introduces the principles, concepts, and case studies. We first outline the underlying principles of the Design-Build-Test-Learn (DBTL) framework and the role of automation, digital integration, and standardization. The chapter then explores core biofoundry technologies including robotic liquid handlers, high-throughput analytical instruments, and digital infrastructure for data management and workflow scheduling. Case studies spanning DNA assembly, protein engineering, metabolic engineering, and mammalian cell culture demonstrate the practical applications of the biofoundries. Economic and societal impacts are assessed alongside current limitations. We discuss the emerging trends including artificial intelligence integration and cloud-based distributed facilities to highlight its potential for biotechnology and the bioeconomy.

Singh, Nilmani↗

A Combined Day-ahead and Real-time approach for Scheduling Real and Reactive power dispatch of Battery Energy Storage System

A combined day-ahead (look-ahead) and real-time optimization approach is proposed to schedule BESS real and reactive power dispatch. The look ahead scheduling agent maximizes profit and essentially performs the function of energy arbitrage by looking at forecasted prices, network constraints and BESS operating conditions to generate real power dispatch schedule. While this schedule is followed, the real time optimization agent computes an optimum reactive power set-point (for a particular objective function) using optimal power flow framework. The concepts are implemented by co-simulating legacy tools and demonstrated on a large real world distribution system. In addition to the profits from energy arbitrage, optimum reactive power set point computation results in achieving desirable network attributes like flat voltage profiles and minimizing voltage deviations from desired value.

Bhatti, Bilal A.↗

Adaptive cold-load pickup considerations in 2-stage microgrid unit commitment for enhancing microgrid resilience

In an extended main grid outage spanning multiple days, load shedding serves as a critical mechanism for islanded microgrids to maintain essential power and energy reserves that are indispensable for fulfilling reliability and resiliency mandates. However, using load shedding for such purposes leads to increasing occurrence of cold load pickup (CLPU) events. Here, this study presents an innovative adaptive CLPU model that introduces a method for determining and incorporating parameters related to CLPU power and energy requirements into a two-stage microgrid unit commitment (MGUC) algorithm. In contrast to the traditional fixed-CLPU-curve approach, this model calculates CLPU duration, power, and energy demands by considering outage durations and ambient temperature variations within the MGUC process. By integrating the adaptive CLPU model into the MGUC problem formulation, it allows for the optimal allocation of energy resources throughout the entire scheduling horizon to fulfill the CLPU requirements when scheduling multiple CLPU events. The performance of the enhanced MGUC algorithm considering CLPU needs is assessed using actual load and photovoltaic (PV) data. Simulation results demonstrate significant improvements in dispatch optimality evaluated by the amount of load served, customer comfort, energy storage operation, and adherence to energy schedules. These enhancements collectively contribute to reliable and resilient microgrid operation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Hierarchical Control Architecture: Utilization of Behind-the-Meter Appliances with Increased Visibility and Controllability

Optimal operations of distribution systems to enhance reliability and resiliency can be achieved if information with higher fidelity can be accessed from grid premise devices. This paper discusses a hierarchical control architecture that can introduce higher visibility and controllability by utilizing the estimated injection and absorption of different behind-the-meter controllable loads, and DERs, including EVs, within a premise. The acquired information is used to make control decisions for scheduling the premise appliances to reduce energy consumptions. The premise controller can also optimize the consumptions on a day-to-day basis in addition to taking input signals from the utility operators to support grid-service events (e.g., peak shaving). The entire control architecture is designed in a modular and scalable fashion that provides benefits to the utility from the reuse of existing infrastructure while creating opportunities for various grid services to be enabled by the operation of premise controllers.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An Integrated Paradigm for the Management of Delivery Risk in Electricity Markets: From Batteries to Insurance and Beyond [Slides]

In wholesale electricity markets today, flexibility from a limited number of distributed energy resources (DERs) is offered daily, and the value of flexibility is not yet recognized for economic hedging of delivery risk. Under a three-year project funded by the ARPA-E PERFORM program, a collaborative team is working towards developing an integrated risk management framework that will leverage flexibility from distributed and bulk resources to cost-effectively and reliably manage delivery risk of intermittent resources. Two concepts are at the core of the proposed integrated risk management framework: (A) flexibility options, which are a novel type of options and enable wholesale electricity market participants to hedge uncertainty by buying flexibility. (B) DER flexibility scores, which provide a way for utilities or aggregators to classify assets in groups with different likelihood of delivering contracted flexibility. This report presentation will focus on the proposed ISO-product "flexibility options," which is complementary to ramp and other products being introduced by ISOs/RTOs to manage net load uncertainties. Participating resources with imbalance risk can buy flexibility options to hedge their production, whereas grid-connected resources that can provide physical flexibility can offer flexibility options. We will present basics of the formulation for a day-ahead ISO market that matches buyers and sellers of this hedge in coordination with existing capabilities to schedule energy and ancillary services, and outline how their settlements mitigate the impact of imbalance risk.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Multiscale simulation of integrated energy system and electricity market interactions

Accelerating the deep decarbonization of the world's electric grids requires the coordination of complex energy systems and infrastructures across timescales from seconds to decades. Here, we present a new multiscale simulation framework that integrates process- and grid-centric modeling paradigms to better design, operate, and control integrated energy systems (IESs), which combine multiple technologies, in wholesale energy markets. Traditionally, IESs are analyzed with a process-centric paradigm such as levelized cost of electricity (LCOE) or annualized net revenue, ignoring important interactions with electricity markets. This framework explicitly models the complex interactions between an IES's bidding, scheduling, and control decisions and the energy market's clearing and settlement processes, while incorporating operational uncertainties. Through two case studies, we show the importance of understanding and quantifying complex resource-grid interactions. In case study 1, we demonstrate that optimized bidding from one resource shifts the profit distribution for all energy systems in the market. This result suggests new and more flexible IES technologies can disrupt the economics of all market participants, possibly leading to accelerated retirements of less flexible resources. Interestingly, the optimized bidding has little impact on grid-level aggregate statistics, such as total generation costs and renewable penetration rate. While aggregate modeling strategies may remain valid under some IES adoption scenarios for analysis focused on regional outcomes, direct comparisons of IES technologies at specific locations without considering these interactions may lead to misleading or incorrect conclusions. In case study 2, we consider the design and flexible operation of IESs that hybridize conventional generators with energy storage. Through a sensitivity analysis, we find that as the size of the storage system increases, the total number of start-ups for coal- and natural gas-based IESs reduced by 25% and 33.6%, and the total thermal generator ramping (i.e., mileage) reduced by 86.5% and 62.5%, respectively. This shows the primary benefit of storage may not be reduced operational costs (which do not change significantly) but fewer start-ups and less ramping, which may greatly simplify the design, operation, and control of carbon capture systems. The new modeling and optimization capabilities from this work enable the coupling of rigorous, dynamic process models with grid-level production cost models to quantitatively identify the nuanced interdependencies across these vast timescales that must be addressed to realize clean, safe, and secure energy production. Moreover, the proposed general multiscale simulation framework is applicable to all IES technologies and can be easily extended to consider other energy carriers (e.g., hydrogen, ammonia) and energy infrastructures (e.g., natural gas pipelines).

24 POWER TRANSMISSION AND DISTRIBUTION↗

Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration

Here, this paper proposes a safe soft actor-critic reinforcement learning (RL) algorithm–based controller for networked microgrid restoration. It formulates the post black-start start as a finite-horizon constrained Markov decision process. The RL agent co-optimizes real and reactive power set-points for both grid-forming and grid-following inverters under explicit voltage and frequency constraints, while enforcing proper power sharing via the Mean Active Power Sharing Index (MPSI) and Mean Reactive Power Sharing Index (MQSI). Numerical results obtained on the IEEE 123-bus distribution system show that the proposed method achieves a mean voltage build-up time of 0.01 s without breaching the 5% sharing-violation budget under various load scenarios, considering MPSI and MQSI indices. These findings demonstrate that the proposed method yields fast and safe black-start schedules without resorting to heuristic penalties.

Selim, Alaa [Dartmouth College, Hanover, NH (Unite↗