Results for “job execution” — Page 5

DOE OSTI2026

The LCLStream Ecosystem for Multi-Institutional Dataset Exploration

We describe a new end-to-end experimental data streaming framework designed from the ground up to support new types of applications – AI training, extremely high-rate X-ray time-of-flight analysis, crystal structure determination with distributed processing, and custom data science applications and visualizers yet to be created. Throughout, we use design choices merging cloud microservices with traditional HPC batch execution models for security and flexibility. This project makes a unique contribution to the DOE Integrated Research Infrastructure (IRI) landscape. By creating a flexible, API-driven data request service, we address a significant need for high-speed data streaming sources for the X-ray science data analysis community. With the combination of data request API, mutual authentication web security framework, job queue system, high-rate data buffer, and complementary nature to facility infrastructure, the LCLStreamer framework has prototyped and implemented several new paradigms critical for future generation experiments.

Rogers, David [ORNL] (ORCID:0000000251871768)↗

DOE OSTI2020

BEE - FY20 P6-3: Release BEEWorkflowManager, BEETaskManager, and client application 2.3.6.01 – LANL ATDM ST / STNS01-4 P6 Milestone Completion Documentation

Release BEEWorkflowManager, BEETaskManager, and client software. The BEEWorkflowManager daemon runs on the HPC cluster login node. It accepts workflows submitted by the BEE client. These workflows are specified using the Common Workflow Language (CWL) standard. The BEEWorkflowManager loads workflows into the Neo4j graph database to create the workflow directed acyclic graph (DAG), and submits the workflow tasks to the BEETaskManager for execution. The BEEWorkflowManager records the state of the workflow and its tasks, and communicates this state to the BEE client. The BEEWorkflowManager will start, pause, and cancel a running workflow and its tasks at the command of the BEE client. The BEETaskManager daemon runs on the HPC cluster login node. It accepts tasks from the BEEWorkflowManager, turns those tasks into HPC resource manager jobs (e.g. a slurm job script), and submits the job to the cluster resource manager. The BEETaskManager then tracks the status of the job (pending, running, complete) and updates the BEEWorkflowManager. The BEETaskManager will also cancel a queued or running job when commanded to do so by the BEEWorkflowManager. The first release of the BEETaskManager will support the Slurm resource manager and the Charliecloud linux container runtime.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Profiles of upcoming HPC Applications and their Impact on Reservation Strategies

With the expected convergence between HPC, BigData and AI, new applications with different profiles are coming to HPC infrastructures. Here, we aim at better understanding the features and needs of these applications in order to be able to run them efficiently on HPC platforms. The approach followed is bottom-up: we study thoroughly an emerging application from the neuroscience community (SLANT) to understand its behavior. Based on these observations, we derive a generic, yet simple, application model (namely, a linear sequence of stochastic jobs). We expect this model to be representative for a large set of upcoming applications that require the computational power of HPC clusters without fitting the typical behavior of large-scale traditional applications. In a second step, we show how one can manipulate this generic model in a scheduling framework. Specifically we consider the problem of making reservations (both time and memory) for an execution on an HPC platform. We derive solutions using the model of the first step of this work. We experimentally show the robustness of the model, even with very few data or with another application, to generate the model, and provide performance gains with regards to standard and more recent approaches used in the neuroscience community.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Repurposing of the Run 2 CMS High Level Trigger Infrastructure as a Cloud Resource for Offline Computing

The former CMS Run 2 High Level Trigger (HLT) farm is one of the largest contributors to CMS compute resources, providing about 25k job slots for offline computing. This CPU farm was initially employed as an opportunistic resource, exploited during inter-fill periods, in the LHC Run 2. Since then, it has become a nearly transparent extension of the CMS capacity at CERN, being located on-site at the LHC interaction point 5 (P5), where the CMS detector is installed. This resource has been configured to support the execution of critical CMS tasks, such as prompt detector data reconstruction. It can therefore be used in combination with the dedicated Tier 0 capacity at CERN, in order to process and absorb peaks in the stream of data coming from the CMS detector. The initial configuration for this resource, based on statically configured VMs, provided the required level of functionality. However, regular operations of this cluster revealed certain limitations compared to the resource provisioning and use model employed in the case of WLCG sites. A new configuration, based on a vacuum-like model, has been implemented for this resource in order to solve the detected shortcomings. This paper reports about this redeployment work on the permanent cloud for an enhanced support to CMS offline computing, comparing the former and new models’ respective functionalities, along with the commissioning effort for the new setup.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2026

Dataset of Generative AI Workload Power Profiles

This dataset provides a collection of high-resolution (5/10 Hz or every 0.2/0.1 seconds) power consumption profiles for generative artificial intelligence (GenAI) workloads executed on NLR's High Performance Computing (HPC) platform Kestrel. The dataset also includes examples of representative whole-facility power profiles generated using a bottom-up, event-driven, data center energy model . This dataset is designed to support research in energy modeling, infrastructure planning, energy system integration, and sustainability analysis for AI-driven computing systems. The dataset captures time-resolved electrical power measurements across a diverse set of configurations, including variations in job type (inference vs. training), workload (LLM vs. image generation), datasets, and number of compute nodes. Power traces are provided in a standardized format and include both raw/instantaneous and aggregated files. Each profile is accompanied by metadata describing workload parameters, enabling reproducibility and cross-study comparison. The dataset is intended for use in applications such as data center infrastructure planning, energy modeling, demand response and grid impact studies, and development and validation of system-level simulation tools. By making these workload-specific power profiles publicly available, this dataset aims to address the current lack of open, empirical energy data for generative AI systems and to facilitate transparent, reproducible research on the energy and environmental impacts of large-scale AI deployment. If you use this dataset, please cite the associated publication: Vercellino et al., “Measurement of Generative AI Workload Power Profiles for Whole-Facility Data Center Infrastructure Planning,” arXiv:2604.07345 (2026).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

GPU Lifetimes on Titan Supercomputer: Survival Analysis and Reliability

The Cray XK7 Titan was the top supercomputer system in the world for a long time and remained critically important throughout its nearly seven year life. It was an interesting machine from a reliability viewpoint as most of its power came from 18,688 GPUs whose operation was forced to execute three rework cycles, two on the GPU mechanical assembly and one on the GPU circuitboards. We write about the last rework cycle and a reliability analysis of over 100,000 years of GPU lifetimes during Titan’s 6-year-long productive period. Using time between failures analysis and statistical survival analysis techniques, we find that GPU reliability is dependent on heat dissipation to an extent that strongly correlates with detailed nuances of the cooling architecture and job scheduling. We describe the history, data collection, cleaning, and analysis and give recommendations for future supercomputing systems. We make the data and our analysis codes publicly available.

Ostrouchov, George↗

DOE OSTI2021

Optimal calibration of gates in trapped-ion quantum computers

Abstract To harness the power of quantum computing, it is essential that a quantum computer provide maximal possible fidelity for a quantum circuit. To this end, much work has been done in the context of qubit routing or embedding, i.e., mapping circuit qubits to physical qubits based on gate performance metrics to optimize the fidelity of execution. Here, we take an alternative approach that leverages a unique capability of a trapped-ion quantum computer, i.e., the all-to-all qubit connectivity. We develop a method to determine a fixed number (budget) of quantum gates that, when calibrated, will maximize the fidelity of a batch of input quantum programs. This dynamic allocation of calibration resources on randomly accessible gates, determined using our heuristics, increases, for a wide range of calibration budget, the average fidelity from 70% or lower to 90% or higher for a typical batch of jobs on an 11-qubit device, in which the fidelity of calibrated and uncalibrated gates are taken to be 99% and 90%, respectively. Our heuristics are scalable, more than 2.5 orders of magnitude faster than a randomized method for synthetic benchmark circuits generated based on real-world use cases.

Maksymov, Andrii (ORCID:0000000282442851)↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 1. Introduction

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter introduces the study.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 2. Study Approach

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter describes the LA100 study approach, including the modeling framework and scenarios.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 3. Electricity Demand Projections

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter explores how electricity is consumed by customers now, how that might change through 2045, and potential opportunities to better align electricity demand and supply.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 4. Customer-Adopted Rooftop Solar and Storage

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter explores the technical and economic potential for rooftop solar in LA, and how much solar and storage might be adopted by customers.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 5. Utility Options for Local Solar and Storage

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter identifies and ranks locations for utility-scale solar (ground-mount, parking canopy, and floating) and storage, and associated costs for integrating these assets into the distribution system.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 6. Renewable Energy Investments and Operations

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter explores pathways to 100% renewable electricity, describing the types of generation resources added, their costs, and how the systems maintain sufficient resources to serve customer demand, including resource adequacy and transmission reliability.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 7. Distribution System Analysis

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter summarizes the growth in distribution-connected energy resources and provides a detailed review of impacts to the distribution grid of growth in customer electricity demand, solar, and storage, as well as required distribution grid upgrades and associated costs.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 8. Greenhouse Gas Emissions

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter summarizes greenhouse gas emissions from power, buildings, and transportation sectors, along with the potential costs of those emissions.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 9. Air Quality and Public Health

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter summarizes changes to air quality (fine particulate matter and ozone) and public health (premature mortality, emergency room visits due to asthma, and hospital admissions due to cardiovascular diseases), and the potential economic value of public health benefits.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 10. Environmental Justice

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter explores implications for environmental justice, including procedural and distributional justice, with an in-depth review of how projections for customer rooftop solar and health benefits vary by census tract.

100% Renewable↗

DOE OSTI2021

The Los Angeles 100% Renewable Energy Study (LA100): Chapter 11. Economic Impacts and Jobs

The City of Los Angeles has set ambitious goals to transform its electricity supply, aiming to achieve a 100% renewable energy power system by 2045, along with aggressive electrification targets for buildings and vehicles. To reach these goals, and assess the implications for jobs, electricity rates, the environment, and environmental justice, the Los Angeles City Council passed a series of motions directing the Los Angeles Department of Water and Power (LADWP) to determine the technical feasibility and investment pathways of a 100% renewable energy portfolio standard. The Los Angeles 100% Renewable Energy Study (LA100) is a first-of-its-kind objective, rigorous, and science-based power systems analysis to determine what investments could be made to achieve these goals. The LA100 final report is presented as a collection of 12 chapters and an executive summary, each of which is available as an individual download. This chapter reviews economic impacts, including local net economic impacts and gross workforce impacts.

100% Renewable↗