SEARCH · Engineering Papers

Results for “Task scheduling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2024

IRIS-GNN: Leveraging Graph Neural Networks for Scheduling on Truly Heterogeneous Runtime Systems

The diversity of accelerators in computer systems poses significant challenges for software developers, such as managing vendor-specific compiler toolchains, code fragmentation requiring different kernel implementations, and performance portability issues. To address these, the Intelligent Runtime System (IRIS) was developed. IRIS works across various systems, from smartphones to supercomputers, enabling automatic performance scaling based on available accelerators. It introduces abstract tasks for seamless execution transitions between accelerators while ensuring memory consistency and task dependencies. Although IRIS simplifies system details, optimal dynamic scheduling still requires user input to understand workload structures. To address this, we introduce a new scheduling policy for IRIS, termed IRIS-GNN, which is the first IRIS hybrid policy that operates in conjunction with the dynamic policies. This policy employs a Graph-Neural Network (GNN) to conduct Graph Classification of any task graphs submitted to IRIS. This GNN analyzes the structure and attributes of the task graph, categorizing it as either locality, concurrency, or mixed. This classification subsequently guides the selection of the dynamic policy used by IRIS. We provide a comparison of the performance of IRIS-GNN against the complete spectrum of IRIS’s dynamic policies, assess the overhead introduced by the GNN within this scheduling framework, and ultimately explore its practical application in real-world scenarios.

Johnston, Beau↗

DOE OSTI2020

Locality-Aware Scheduling for Scalable Heterogeneous Environments

Heterogeneous computing promise boost performance of scientific applications by allowing massively parallel execution of computational tasks. However, manually managing extremely heterogeneous, multi-device systems is complicated and may result in sub-optimal performance. Specifically, data management is an extremely challenging problem on multi-device systems. In this work, we introduce two locality-aware schedulers for the Minos Computing Library (MCL), an asynchronous, task-based programming model and runtime for extremely heterogeneous systems. The first scheduler implements a pure locality-aware algorithm to maximize data reuse, though it might incur in ”hot-spots” that limit system utilization. The second scheduler mitigates this drawback by dynamically targeting between locality-awareness and system utilization based on the current workload and available computing devices. Our results show that locality-awareness greatly benefit applications that exhibit data reuse, providing up to 6.9x and 7.9x over the original MCL scheduler and equivalent OpenCL implementations, respectively. Moreover, our schedulers introduce negligible overhead compared with the original MCL scheduler and achieve similar performance for applications that don’t benefit from data locality.

Architecture, co-design, Task-based programming mo↗

DOE OSTI2023

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Applications which fuse machine learning and simulation are rarely best served by a single computing resource. Highly parallel simulation codes are best deployed on super- computers, while AI tasks used to decide which simulations to perform may be best suited to specialized accelerators. Here we present a Function-as-a-Service (FaaS) system for executing complex, distributed computational campaigns that achieves performance parity with conventional workflow systems without the complexities of secure network connections between compute providers. One innovation enabling high performance is a subsystem that directly moves task data between sites, separate from the cloud-hosted FaaS system used to distribute task instructions. We also introduce a flexible scheduling system that allows us access factor of 2 trade offs between the amount of resources required to solve a problem at each compute site. We anticipate that this system will upgrade multi-site applications from demonstration projects to routine practice in computational science.

Ward, Logan↗

DOE OSTI2019

Exploring the use of novel programming models in land surface models

A wide range of programming models are currently under rapid development to meet the needs of application developers looking to work on more complex machines. These models fill a variety of roles. Some look to abstract supercomputer architecture, including both processors and memory, to present a strategy for portable performance across a wide range of machines. Others look to expose concurrency by explicitly constructing task-driven dependency graphs that allow a scheduler to find parallelism. Here we explore the implications for application codes of adopting two such programming models, Kokkos and Legion, one from each class of models. We specifically focus on the software design implications on refactoring existing applications, rather than the performance and performance tuning of these models. We identify a strategy for refactoring the Energy Exascale Earth System Model's Land Surface Model, an extremely complex code for climate applications, and prototype a series of mini-apps that explore the adoption of Kokkos and Legion. In doing this, we identify commonalities across the models, leading to a series of conclusions about application software design and refactoring for the adoption of novel programming models.

Coon, Ethan↗

DOE OSTI2022

MSASGCN : Multi-Head Self-Attention Spatiotemporal Graph Convolutional Network for Traffic Flow Forecasting

Traffic flow forecasting is an essential task of an intelligent transportation system (ITS), closely related to intelligent transportation management and resource scheduling. Dynamic spatial-temporal dependencies in traffic data make traffic flow forecasting to be a challenging task. Most existing research cannot model dynamic spatial and temporal correlations to achieve well-forecasting performance. The multi-head self-attention mechanism is a valuable method to capture dynamic spatial-temporal correlations, and combining it with graph convolutional networks is a promising solution. Therefore, we propose a multi-head self-attention spatiotemporal graph convolutional network (MSASGCN) model. It can effectively capture local correlations and potential global correlations of spatial structures, can handle dynamic evolution of the road network, and, in the time dimension, can effectively capture dynamic temporal correlations. Experiments on two real datasets verify the stability of our proposed model, obtaining a better prediction performance than the baseline algorithms. The correlation metrics get significantly reduced compared with traditional time series prediction methods and deep learning methods without using graph neural networks, according to MAE and RMSE results. Compared with advanced traffic flow forecasting methods, our model also has a performance improvement and a more stable prediction performance. We also discuss some problems and challenges in traffic forecasting.

Cao, Yang↗

DOE OSTI2018

SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks

Going deeper and wider in neural architectures improves their accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) practitioners either need to change to less de- sired network architectures, or nontrivially dissect a network across multiGPUs. These distract DL practitioners from concentrating on their original machine learning tasks. We present SuperNeurons: a dynamic GPU memory scheduling runtime to enable the network training far beyond the GPU DRAM capacity. SuperNeurons features 3 memory optimizations, Liveness Analysis, Unified Tensor Pool, and Cost-Aware Recomputation; together they effectively reduce the network-wide peak memory usage down to the maximal memory usage among layers. We also address the performance issues in these memory-saving techniques. Given the limited GPU DRAM, SuperNeurons not only provisions the necessary memory for the training, but also dynamically allocates the memory for convolution workspaces to achieve the high performance. Evaluations against Caffe, Torch, MXNet and TensorFlow have demonstrated that SuperNeurons trains at least 3.2432 deeper network than current ones with the leading performance. Particularly, SuperNeurons can train ResNet2500 that has 104 basic network layers on a 12GB K40c.

Wang, Linnan↗

DOE OSTI2019

symPACK v2.0

SymPACK is a parallel direct solver for sparse symmetric systems of linear equations. SymPACK is implemented for large- scale distributed-memory platforms. It employs several innovative ideas, including the use of an asynchronous task paradigm, one-sided communication, and dynamic scheduling.

Ng, EsmondG↗

DOE OSTI2021

Production and Performance of High Sulfur Concentration Waste Glasses (FY21)

In FY21 Savannah River National Laboratory (SRNL) was tasked to investigate the practical aspects of low activity waste (LAW) glass production with high sulfur loading for the Office of River Protection (ORP). Relationships between sulfur solubility, retention, and melter resonance need to be evaluated to refine waste loading limits and the super saturated melting (triple melt, grind, and wash) methodology. The impact of high sulfur concentration on durability and phase stability (crystallization and liquid-liquid immiscibility) during slow cooling also needs to be further evaluated. To accomplish this, SRNL proposed scaled melter testing using simulants with matched physicochemical properties to melter feed, as opposed to laboratory reagents. This type of testing is expected to validate applicability to design parameters and functional criteria expected during unit operations. This work was expected to commence in FY21, with the first melter tests being performed near the end of the FY. However, reduced access to Savannah River Site and personnel availability during the COVID-19 pandemic significantly reduced the amount of time and resources available to perform the task. This report documents the experimental plan and progress towards this task. The following is a summarized list of scheduled testing that will be performed on feed, condensate, cold cap and glass: 1) Melter runs; 2) Product consistency test (PCT); 3) Inductively coupled plasma optical emission spectroscopy (ICP-OES); and 4) Optical (OM) and scanning electron microscopy (SEM).

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2023

E-Area Low-Level Waste Facility Inadvertent Human Intruder Limits and Doses in Support of the PA2022

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

E-Area Low-Level Waste Facility Inadvertent Human Intruder Limits and Doses in Support of the PA2022

This report documents the inadvertent human intruder (IHI) analysis for the E-Area Low-Level Waste Facility (ELLWF) at the Savannah River Site (SRS), near Aiken, South Carolina. This analysis supports the revised ELLWF Performance Assessment (PA), complying with the Department of Energy standard for operation of low-level waste disposal facilities (USDOE, 2017). The ELLWF is an operating waste disposal facility and is scheduled to continue accepting waste to 2065. One task of the revised PA is to establish waste inventory limits for the various disposal units at ELLWF. This is done by modeling future contaminant release and transport through applicable pathways to human receptors, comparing predicted doses per disposed curie with applicable performance measures, to obtain inventory limits which will assure that doses to receptors do not exceed performance measures. This report documents results of modeling future doses to one class of receptor, the inadvertent human intruder. It is assumed that after site closure, public knowledge of the site is lost, and IHIs will engage in activities on the ELLWF that will disrupt the closure cap, causing dose to the IHI. Following USDOE (2017), six different stylized exposure scenarios are considered, simulating activities by an IHI which could result in a radiological dose. The six scenarios are: • Acute – Basement Construction: IHI constructs a basement and encounters waste during excavation which is inadvertently mixed with clean soil and diluted. • Acute – Well Drilling: IHI drills a water well through waste and is exposed to drill cuttings mixed with clean soil that are brought to the surface. • Acute – Discovery: IHI begins constructing a basement but stops when encountering the riprap in the final closure cap and is exposed to photon radiation from unexcavated material residing in the undisturbed waste zone. • Chronic – Agriculture: Resident IHI is exposed to waste that was excavated for basement construction and mixed with native soil in the intruder’s vegetable garden. • Chronic – Post-Drilling: Resident IHI is exposed to waste from drill cuttings mixed with native soil and scattered in the garden area. • Chronic – Residential: Resident IHI is exposed to external radiation while in home located above waste with shielding provided by the concrete basement floor and any soil or engineered material remaining between the basement and waste. Dose calculations are performed using the SRNL Dose Toolkit (Aleman, 2023), following the approach of Smith et al (2019). Calculations are performed separately for 27 of the 33 disposal units (DUs) at ELLWF and are radionuclide specific. The results of the IHI analysis include: • Dose Factors: mrem per disposed curie (acute) and mrem/yr per disposed curie (chronic) for each parent radionuclide, for each DU. • Inventory Limits: in curies, for each parent radionuclide, for each DU. • Estimated Dose to IHI: mrem (acute) and mrem/yr (chronic), for each DU, given its projected closure inventory without inventory biases applied. Most DU-specific IHI inventory limits are in the range of 10 3 to 10 7 curies per nuclide. The lowest inventory limits are associated with gamma-emitters such as Sn-126, Ra-226, Th-232, and Cm-248. Radionuclides with short half-lives such as Pu-241, and nuclides which are pure beta emitters or which decay by electron capture, such as Ni-59 and Ni-63, have the highest limits. For the 27 evaluated DUs, predicted IHI doses are shown in Table ES-1. The maximum acute dose is 1.18 mrem, at ST23, much less than the DOE performance measure of 500 mrem (USDOE, 2017). The highest chronic dose is 37.2 mrem/yr at ST02, below the DOE performance measure of 100 mrem/yr. Also shown are estimated inventory sums of fractions (SOFs) at closure in 2065, for groundwater (GW) and IHI pathways. For each DU, the inventory is constrained by the GW pathway. For most DUs, the IHI SOFs are approximately 1000 times lower than the GW SOF values, and the IHI pathway does not drive risk for any disposal unit.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

Energy Improvements of Fire Station 71

Since 2018, the City of Shawnee, Kansas has completed two phases of the State of Kansas Facility Conservation Improvement Program (FCIP), an initiative that guarantees operational cost and energy savings through targeted construction improvements on City facilities and infrastructure. The City is currently in the third phase of this FCIP, where one of the projects included an investment in energy improvements for Fire Station 71 (FS 71). The City partnered with Navitas, an Energy Service Company (ESCO), to implement a Photovoltaic Solar Array on FS71. The purpose of this project was to invest in sustainable building improvements with Energy Conservation Measures (ECM) to bring cost savings to the City and to provide sustainable benefits to the residents of Shawnee. In the first task of the project, Navitas collaborated with the City of Shawnee and the Community Development Department to determine the optimal layout and schedule for the installation of the solar array on FS 71. In the second task of the project, Navitas installed the 99.8 kW DC Photovoltaic solar array system. This system installation comprised of racking, inverters, optimizers, load center, and disconnect, which were all installed at a total ECM price of $\$$247,948. The third task focused on start-up and commissioning of the array. Navitas installed a real-time data analytics information management system integrated with utility meters, which evaluates the operations of the utility system and verifies operation of equipment and ensures optimum operation for energy efficiency. In the final task of this project, this analytics system was used for monitoring and verification, which will continue to be used to evaluate the success of the project for the coming years. The primary goal of the project was to install the 99.8 kW DC PV solar array at FS 71 to demonstrate the viability of solar energy systems in essential municipal facilities. Fire stations are energy demanding structures, as they require a constant intake of power and have a high baseline energy usage. The success of solar arrays on a fire station exemplifies their energy efficiency and effectiveness and displays their potential for application on other city facilities. By installing a solar array at such a facility, the City sought not only to offset electricity usage but also to serve as a model for ECMs in other municipal facilities and infrastructure projects. From an economic standpoint, this project demonstrates the feasibility of renewable energy at the municipal level. The total project cost of $\$$247,948 was split evenly between city funds and award funding, minimizing financial risk while ensuring guaranteed long-term savings. Any excess savings that are beyond the guaranteed minimums remain with the city, which enables future investment in sustainable energy initiatives. This project provides many benefits to the public. In addition to reducing the environmental footprint of city operations, it lowers taxpayer-funded utility spending and improves the energy security of a critical facility. The knowledge gained from this implementation motivates the City to focus on similar efforts across other public facilities in future FCIP phases and other City projects.

14 SOLAR ENERGY↗

DOE OSTI2019

Scale-Up of Coal-Based Supercritical CO 2 cycle technology

The Allam Cycle is a transformational technology that enables a step change in the performance, efficiency, and cost of electricity for electrical generating units by using supercritical CO 2 (sCO 2 ) as the working fluid in the turbine for power generation. The objectives of this Phase I effort were to select a host site for a large pilot system through a process of down selection, develop a conceptual design, and conduct a pre-front-end engineering design (pre-FEED) study for a sCO 2 cycle pilot plant that uses domestic coal reserves as the primary feedstock, thereby furthering technology development of the coal-based Allam Cycle. The project was divided into four main tasks: 1) Project Management and Planning; 2) Conceptual Pilot Plant Definition and Host Site Selection; 3) Preliminary Design, Cost, and Schedule Development; and 4) Commitments and Environmental Information Volume (EIV). All project management activities occurred in Task 1. Host site selection was conducted in Task 2, while conceptual design and pre-FEED activities took place under Task 3. Task 4 enhanced the project by securing commitments for the next phase of the program and completing an initial EIV. The Great Plains Synfuels Plant (GPSP) has been selected as the host site for the large-scale pilot, and its owner, Dakota Gasification Company (DGC), is a committed partner. GPSP was compared to two other host site candidates during Phase I: the existing natural gas Allam Cycle facility in La Porte, Texas, and the Energy & Environmental Research Center’s (EERC) high-bay demonstration facility in Grand Forks, North Dakota. Costs for hosting the pilot plant were estimated for each facility, and those costs are shown in Table ES-1. Based on these estimates, the DGC site was clearly shown to be the lowest cost option and was selected as the host site for the project.

01 COAL, LIGNITE, AND PEAT↗

DOE OSTI2021

Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)

The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Lessons Learned From Decommissioning B411 Decontamination Centre at Whiteshell Laboratories - 18211

Whiteshell Laboratories, located in Manitoba, Canada, provided research facilities for the Canadian nuclear industry since the early 1960's. The facility was centered on the largest organically cooled, heavy water moderated nuclear reactor in the world, the WR-1. It is now operated by Canadian Nuclear Laboratories (CNL) and is the first major decommissioning project of a nuclear research establishment in Canada. One of the major projects to support the closure of Whiteshell Laboratories (WL) is the decommissioning and demolition of the Building 411 (B411) Decontamination Centre, which provided a decontamination service for maintaining research and development of experimental rigs, equipment and tools in a safe usable state. It also provided a laundry service for radioactively contaminated clothing. The team successfully self-performed their first demolition of a nuclear facility, but not without challenges. The baseline decommissioning strategy assumed most of the facility would meet unconditional clearance levels; however when characterization revealed the extent of contamination exceeded initial assumptions projected from process knowledge, in conjunction with resource challenges and a waste disposal pathway not firmly established, active remediation of the facility was undertaken. With an unknown waste acceptance criteria for the future waste disposal facility, sludge that had accumulated in a tank used to collect xylene and organic coolant from contaminated tools and equipment and the selection of appropriate waste packages for waste storage, transportation and disposal forced the project team to adjust their demolition timeline. This, coupled with developing an open air demolition plan and managing an uncertain end state criteria, were just some of the major tasks the team successfully navigated. The impact on both schedule and cost performance caused by the change in strategy, and the lessons learned experienced by the WL Decommissioning Project Team that are being applied as the Whiteshell Laboratories Closure Project (WLCP) moves forward with the next major decommissioning challenge, are highlighted in this paper. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2026

Q-IRIS: The Evolution of the IRIS Task-Based Runtime to Enable Classical-Quantum Workflows

Extreme heterogeneity in emerging HPC systems are starting to include quantum accelerators, motivating runtimes that can coordinate between classical and quantum workloads. We present a proof-of-concept hybrid execution framework integrating the IRIS asynchronous task-based runtime with the XACC quantum programming framework via the Quantum Intermediate Representation Execution Engine (QIR-EE). IRIS orchestrates multiple programs written in the quantum intermediate representation (QIR) across heterogeneous backends (including multiple quantum simulators), enabling concurrent execution of classical and quantum tasks. Although not a performance study, we report measurable outcomes through the successful asynchronous scheduling and execution of multiple quantum workloads. To illustrate practical runtime implications, we decompose a four-qubit circuit into smaller subcircuits through a process known as quantum circuit cutting, reducing per-task quantum simulation load and demonstrating how task granularity can improve simulator throughput and reduce queueing behavior -- effects directly relevant to early quantum hardware environments. We conclude by outlining key challenges for scaling hybrid runtimes, including coordinated scheduling, classical-quantum interaction management, and support for diverse backend resources in heterogeneous systems.

Miniskar, Narasinga Rao [ORNL] (ORCID:000000018259↗

DOE OSTI2024

Improving I/O-aware Workflow Scheduling via Data Flow Characterization and trade-off Analysis

The scientific computing paradigm has transitioned from compute-intensive to I/O-intensive and memory-intensive in the past decade, especially when data-driven science has become common practice. Numerous empirical I/O-aware scheduling optimizations have been developed by incorporating I/O capacity and bandwidth as constraints into scheduling. Unfortunately, there is a lack of data flow (I/O) characterization tool and an understanding of trade-offs between concurrency, locality, and I/O bandwidth. To bridge the gap, this work 1) presents a set of descriptors to characterize, organize, and visualize I/O profiles, including flow size, I/O bandwidth, and operation count, which group data flows by I/O types, tasks, and files; 2) proposes an I/O Roofline model-based trade-off analysis to find the optimal trade-off between flow operational intensity, concurrency, and flow performance. The I/O descriptors generate useful insights into complicated I/O behaviors, suggesting distinct concurrency, storage, and scheduling to be used by types, tasks, and files. The proposed trade-off analysis guides scheduling decisions that generate resource assignment with the best flow parallelism. We evaluate our I/O-aware scheduling methodology on a highly I/O-intensive workflow–1000 Genomes. The experimental results demonstrate speedups of up to 2.4× compared to the state-of-the- art methods.

Guo, Luanzheng [BATTELLE (PACIFIC NW LAB)]↗

DOE OSTI2023

Usage Pattern Analysis for the Summit Login Nodes

High performance computing (HPC) users interact with Summit through dedicated gateways, also known as login nodes. The performance and stability of these login nodes can have a significant impact on the user experience. In this study, the performance and stability of Summit’s five login nodes are evaluated by analyzing the log data from 2020 and 2021. The analysis focuses on the computing capability (CPU average load, users and tasks) and the storage performance, along with the associated job scheduler activity. The outcome of this study can serve as the foundation of a predictive modeling framework that enables the system admin of an HPC system to preemptively deploy countermeasures before the onset of a system failure.

Eiffert, Brett↗

DOE OSTI2018

Programmable In Situ System for Iterative Workflows

We describe an in situ system for solving iterative problems. We specifically target inverse problems, where expensive simulations are approximated using a surrogate model. The model explores the parameter space of the simulation through iterative trials, each of which becomes a job managed by a parallel scheduler. Our work extends Henson [1], a cooperative multi-tasking system for in situ execution of loosely coupled codes.

Lohrmann, Erich↗