Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “HPC Account”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Development of a High Performance Computing Accounts Administration Web Application

INL's High Performance Computing (HPC) accounts administration tool is a web application that that allows an administrator to change user information, create an HPC account, create or modify an LDAP group or project, and modify the members in a group. An administrator can search for a user, obtain the relevant information about the user, and update their account information. To create or modify an LDAP group or project, the administrator can filter a list of groups or projects according to the name, owner, or administrator that created the project or group, choose a name accordingly, and specify the group or project owner.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Effect of Processing on Precipitation Kinetics in Nano Polycrystalline 7075 Aluminum Alloy

Fairmount Technologies (FT) seeks to optimize the thermomechanical processing of Aluminum Alloy 7075. The target yield strength is 20% higher than commercially available 7075-T6 temper sheet which will enable structural light weighting by a similar magnitude and reduce the carbon footprint of the transportation sector. To reduce processing cost and make this commercially viable, it is important to understand physical mechanisms driving microstructural changes. A mesoscale phase-field model seeded with experimentally measured microstructures and assessed thermodynamic and kinetic properties will be developed and exercised using HPC at PNNL. It will account for the effect of deformation, temperature, and chemistry on precipitation kinetics in nanocrystalline microstructures. Our objective is to understand this “stress-aging” coupling by performing parametric simulations with mesoscale models for a better understanding of GP zone and η' formation in nanocrystalline 7075 Al. This will help FT optimize processing schedules to achieve desired microstructures that increase both strength and ductility.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Investigating User Experiences with Data Abstractions on High Performance Computing Systems

Scientific exploration generates expanding volumes of data that commonly require High Performance Computing (HPC) systems to facilitate research. HPC systems are complex ecosystems of hardware and software that frequently are not user friendly. The Usable Data Abstractions (UDA) project set out to build usable software for scientific workflows in HPC environments by undertaking multiple rounds of qualitative user research. Qualitative research investigates how individuals accomplish their work and our interview-based study surfaced a variety of insights about the experiences of working in and with HPC ecosystems. This report examines multiple facets to the experiences of scientists and developers using and supporting HPC systems. We discuss how stakeholders grasp the design and configuration of these systems, the impacts of abstraction layers on their ability to successfully do work, and the varied perceptions of time that shape this work. Examining the adoption of the Cori HPC at NERSC we explore the anticipations and lived experiences of users interacting with this system’s novel storage feature, the Burst Buffer. We present lessons learned from across these insights to illustrate just some of the challenges HPC facilities and their stakeholders need to account for when procuring and supporting these essential scientific resources to ensure their usability and utility to a variety of scientific practices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

HPC ODA Commons [SWR-26-003]

HPC ODA Commons is a community-driven platform for standardizing HPC operational data analytics. HPC sites generate enormous volumes of operational data - scheduler logs, accounting records, monitoring streams - but turning that data into actionable insight is needlessly hard. Each site builds bespoke parsers, schemas, and evaluation pipelines. Results can't be compared across institutions. Promising analytics ideas stay siloed because there's no shared language for describing the data, the experiments, or the outcomes. HPC ODA Commons fixes this by establishing community-governed contracts - versioned schemas, canonical artifacts, and benchmark recipes - that make ODA workflows discoverable, reproducible, and comparable. It pairs these standards with a practical, CLI-first toolkit that lets operators and researchers go from raw logs to standardized results without sending data off-cluster.

Menear, Kevin [National Laboratory of the Rockies ↗
DOE OSTI2022

Exploring Tradeoffs in Federated Learning on Serverless Computing Architectures

Federated learning is driving the development of new techniques to efficiently and securely use data across multiple sites while using diverse resources. One of these techniques is the use of the serverless computing paradigm to abstract away resource specific configurations, allowing federated learning across heterogeneous environments. However, deploying federated learning across edge resources, the cloud, and traditional HPC sites will require specialized approaches in order to best account for the weaknesses and strengths of each resource. In this work, we explore the new tradeoffs presented by managing a federated learning task across heterogeneous resources and demonstrate these tradeoffs with experiments using a serverless federated learning framework.

Baughman, Matt↗
DOE OSTI2020

Final Reports of the 2020 Los Alamos National Laboratory Computational Physics Student Summer Workshop

For the past ten years, the workshop has been bringing a highly talented and diverse group of student every summer. Students work in teams of two, alongside typically two mentors, on research projects reflecting a broad range of topics within computational physics. In addition, students attend a series of lectures on topics within computational physics, facility tours, and networking events. The program lasts ten weeks, with this year’s workshop running from June 8 to August 14. At the end of the summer, students give a final presentation, along with a written report. Those reports are what make up the remaining sections of this document. Admission to the workshop is by a competitive process, with the mentors forming the selection committee. One of the important accomplishments of the workshop has been to create a student pipeline from diverse schools that sometimes are not normally tapped by LANL recruiting. Many workshop students maintain a continuing relationship with LANL, returning as students interns, post-doctoral researchers, and staff members. Additionally, workshop alumni act as ambassadors for LANL. The result is a wider awareness both of LANL as a potential employer, and of the technical work that happens at LANL. This year, the workshop format was changed in several ways, in order to accommodate the off-site, virtual format. Students worked on LANL virtual desktop systems remotely, also accessing LANL HPC resources. In order to facilitate communication, student were given accounts on both Webex, a video teleconferencing platform, and Mattermost, an online team collaboration and chat platform, similar to Slack. Daily communication between students and mentors was primarily on Mattermost, with Webex conferencing as needed. The lectures were all done on Webex. Given the difficulty of the virtual format, and a concern that students might have video teleconferencing burn-out after an academic semester largely moved to that format, all lectures were optional this year. In spite of this, the attendance was generally high. Lecturers were asked to try to move to a more high-level, ”What is it?,” format. Once again, the students did a tremendous job. Over the course of ten weeks, they did important research across a staggering array of disciplines. The following pages contain the final report for each team’s research efforts. We hope you will find reading them as exciting as it was for us to produce them.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Final Reports of the 2021 Los Alamos National Laboratory Computational Physics Student Summer Workshop

Since 2011, the Los Alamos National Laboratory Computational Physics Student Summer Workshop has been bringing together a highly talented and diverse group of students every summer. Students work in teams of two, alongside typically two mentors, on research projects reflecting a broad range of topics within computational physics. In addition, students attend a series of lectures on topics within computational physics, facility tours, and networking events. The program lasts ten weeks, with this year’s workshop running from June 7 to August 13. At the end of the summer, students give a final presentation, along with a written report. Those reports are what make up the remaining sections of this document. Admission to the workshop is by a competitive process, with the mentors forming the selection committee. One of the important accomplishments of the workshop has been to create a student pipeline from diverse schools that sometimes are not normally tapped by LANL recruiting. Many workshop students maintain a continuing relationship with LANL, returning as student interns, post-doctoral researchers, and staff members. Additionally, workshop alumni act as ambassadors for LANL. The result is a wider awareness both of LANL as a potential employer, and of the technical work that happens at LANL. This year, the workshop was once again in an off-site, virtual format. Students worked on LANL virtual desktop systems remotely, also accessing LANL HPC resources. In order to facilitate communication, student were given accounts on both Webex, a video teleconferencing platform, and Mattermost, an online team collaboration and chat platform, similar to Slack. Daily communication between students and mentors was primarily on Mattermost, with Webex conferencing as needed. The lectures were all done on Webex. Given the difficulty of the virtual format, and a concern that students might have video teleconferencing burn-out, all lectures were optional this year. In spite of this, the attendance was generally high. Lecturers were asked to try to move to a more high-level, ”What is it?,” format. Once again, the students did a tremendous job. Over the course of ten weeks, they did important research across a staggering array of disciplines. The following pages contain the final report for each team’s research efforts. Enjoy!

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2020

Early Application Results on Pre-exascale Architecture with Analysis of Performance Challenges and Projections (Milestone PM-AD-1080 WBS 2.2)

This Exascale Computing Project (ECP) Milestone Report summarizes the status of all 30 ECP Applications Development (AD) sub-projects at the end of FY19. In August and September of 2019, a comprehensive assessment of AD projects was conducted jointly by the ECP leadership and a team of external subject matter experts. Reviews took place in person over five days-two at the National Renewable Energy Laboratory and three at Argonne National Laboratory and the University of Chicago. The review committees were tasked with evaluating each sub-project's progress in porting their code(s) to current multi-GPU architectures considered precursors to planned exascale machines. This includes characterizing which modules have been ported to multi-accelerator nodes, initial performance analyses, the status of software integration, and a current vision of successes, obstacles, and next steps. As such this report contains not only an accurate snapshot of each sub-project's current status, but also represents an unprecedentedly broad account of experiences porting large scientific applications to next-generation HPC architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

High-Fidelity and High-Performance Computational Simulations for Rapid Design Optimization of Sulfur Thermal Energy Storage (CRADA Report)

NREL and Element 16 collaborated on sulfur thermal energy storage modeling using NREL’s high performance computing (HPC) resources to assist its application in industrial processes. Industrial process heat (IPH) accounts for ~70% of US manufacturing energy use and is primarily produced by fossil fuel combustion. Approximately, 1500 TWht (~60% Terawatt hour thermal) of IPH demand is in the temperature range of 100-300°C. Industrial applications in this temperature range include drying, hydrothermal processing, thermal enhanced oil recovery, food and beverage, bioethanol production, etc. Cost-effective thermal energy storage (TES) that increases the utilization of waste and renewable heat (solar, geothermal, etc.) could provide significant energy savings and reliable heat sources, decrease emissions, and increase US manufacturing competitiveness through reductions in fuel consumption. This HPC4EI project facilitated Element 16’s development of low-cost and high-impact molten sulfur TES for dispatchable IPH. The development of a high-fidelity model validated by experimental data and HPC simulations enabled the successful resolution of the complex interplay between fluid dynamics and heat transfer processes during transient operation of sulfur TES, overcoming the numerical challenges posed by the non-linear temperature-dependent physical properties of sulfur. The project helped accelerate Element 16’s molten sulfur TES product design and support its broad applications.

25 ENERGY STORAGE↗
DOE OSTI2025

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗
DOE OSTI2023

Developing a Vorticity-Velocity-Based Off-Body Solver to Perform Multifidelity Simulations of Wind Farms

Wind power has become a key player in satisfying the global energy needs. With increased market penetration, unanticipated unsteady loading induced failures, installation related reductions in power generation, and significant maintenance costs have underscored the need to predict the unsteady fluid-structure interactions related to turbine layout and off-design wind conditions. Contemporary turbine design tools are incapable of accounting for such loadings. As a result, researchers have started utilizing high-Performance-Computing (HPC) based Computational Fluid Dynamics (CFD) solvers, such as the U.S. Department of Energy sponsored ExaWind software package, to investigate these phenomena. Unfortunately, such HPC tools are computationally expensive for routine industrial use, often because of the sheer number of cells required to resolve the wake flowfield. This paper describes a preliminary effort to address this issue by developing a vorticity-velocity based CFD off-body solver, VorTran-M2-AMReX, that integrates directly with DOE's ExaWind wind turbine analysis system to perform accurate and reliable simulations of wind turbine/farm at a lower computational cost than ExaWind alone. This article summarizes work undertaken to date concerning the assembly of the proposed analysis tool, and provides preliminary validation and verification of the VorTran-M2-AMReX off-body solver.

adaptive mesh refinement↗
DOE OSTI2022

Best Practices for NERSC Training

The National Energy Research Supercomputing Center (NERSC) at Lawrence Berkeley National Laboratory (LBNL) organizes approximately 20 training events per year for its 8,000 users from 800 projects, who have varying levels of High Performance Computing (HPC) knowledge and familiarity with NERSC's HPC resources. Due to the novel circumstances of the pandemic, NERSC began transforming our traditional smaller-scale, on-site training events to larger-scale, fully virtual sessions in March 2020. We treated this as an opportunity to try new approaches and improve our training best practices. This paper describes the key practices we have developed since the start of this transformation, including considerations for organizing events; collaboration with other HPC centers and the DOE ECP Program to increase reach and impact of events; targeted emails to users to increase attendance; efficient management of user accounts for computational resource access; strategies for preventing Zoombombing; streamlining the publication of professional-quality, closed-captioned videos on the NERSC YouTube channel for accessibility; effective communication channels for Q&A; tailoring training contents to NERSC user needs via close collaboration with vendors and presenters; standardized training procedures and publishing of training materials; and considerations for planning HPC training topics. Additionally, most of these practices will be continued after the pandemic as effective norms for training.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

High-Accuracy Simulations to Model Pyrometallurgical Processes in a Secondary Lead Reverberatory Furnace

The US manufacturing industry produces about 1.3 million tons of refined lead each year using secondary sources consisting mainly of lead batteries. ORNL is partnering with Gopher resource, the second largest lead recycling company in the United States, and GTI, to develop a high-fidelity CFD model of a directly fired, reverberatory-style, secondary lead furnace. These High Performance Computing (HPC) simulations are aimed to use first principles modeling for combustion and melting processes of the secondary lead feed while accounting for complex interphase interactions between the gas, solid charge (lead) material, slag, and metal phases. Through validation against operating plant data, this effort will enable significant improvements in design, operational parameters, and energy efficiency, thus improving productivity and refractory lifetime of secondary lead melting furnaces. Estimated savings/reduction of, at least, 1 trillion BTU, 1 million ton/year of greenhouse gas emissions, and $\$50$ million/year to the US lead industry can be expected. ORNL resources and expertise in high-performance computing and multicomponent, multiphase flows were utilized to realize this goal while advancing the understanding of the smelting and melting processes occurring within the furnace.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Workflow Provenance in the Computing Continuum for Responsible, Trustworthy, and Energy-Efficient AI

As Artificial Intelligence (AI) becomes more pervasive in our society, it is crucial to develop, deploy, and assess Responsible and Trustworthy AI (RTAI) models, i.e., those that consider not only accuracy but also other aspects, such as explainability, fairness, and energy efficiency. Workflow provenance data have historically enabled critical capabilities towards RTAI. Provenance data derivation paths contribute to responsible workflows through transparency in tracking artifacts and resource consumption. Provenance data are well-known for their trustworthiness helping explainability, reproducibility, and accountability. However, there are complex challenges to achieve RTAI, which are further complicated by the heterogeneous infrastructure in the computing continuum (Edge-Cloud-HPC) used to develop and deploy models. As a result, a significant research and development gap remains between workflow provenance data management and RTAI. In this paper, we present a vision of the pivotal role of workflow provenance in supporting RTAI and discuss related challenges. We present a schematic view between RTAI and provenance, and highlight open research directions.

Santos Souza, Renan↗
DOE OSTI2024

AutoCheck: Automatically Identifying Variables for Checkpointing by Data Dependency Analysis

Checkpoint/Restart (C/R) has been widely deployed in numerous HPC systems, Clouds, and industrial data centers, which are typically operated by system engineers. Nevertheless, there is no existing approach that helps system engineers without domain expertise and domain scientists without system fault tolerance knowledge identify those critical variables accounted for correct application execution restoration in a failure for C/R. To address this problem, we propose an analytical model and a tool (AutoCheck) that can automatically identify critical variables to checkpoint for C/R. AutoCheck relies on first, analytically tracking and optimizing data dependency between variables and other application execution state, and second, a set of heuristics that identify critical variables for checkpointing from the refined data dependency graph (DDG). AutoCheck allows programmers to pinpoint critical variables to checkpoint quickly within a few minutes. We evaluate AutoCheck on 13 representative HPC benchmarks, demonstrating that AutoCheck can efficiently identify correct critical variables to checkpoint.

HPC↗
DOE OSTI2025

NLR HPC Kestrel Jobs Data

Overview: Anonymized job-level records from the Kestrel HPC system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, utilization, energy estimates, and efficiency metrics. Sensitive fields (user, account, job name, submit line, working directory, submit script, and job type) are replaced with 7-character cryptographic hashes. System & Timeframe: Kestrel is located at the NLR campus. Standard compute nodes have 104 cores and 256 GB RAM; bigmem nodes have 2,000 GB. GPU nodes (gpu-h100 partition) use NVIDIA H100 GPUs. Data covers jobs submitted August 2023 through December 2025. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.kestrel.job-anon.zip — Anonymized job records (Hive-partitioned Parquet) datacard.md — Full dataset documentation ~11 million rows, 50 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct with timezone-aware export (SLURM_TIME_FORMAT="%Y-%m-%dT%H:%M:%S%z"), loaded into PostgreSQL. Calculated columns updated via database triggers and batch functions. All timestamps use timestamptz and correctly handle DST transitions. Preprocessing: Anonymization of name, user, account, submit_line, work_dir, submit_script, and job_type via 7-char hex hashes Derived columns: queue_wait, cpu_eff, max/min/avg_mem_eff, energy estimates Simplified job state mapping (e.g., "CANCELLED by 132357" → "CANCELLED") Boolean flags: python_job, reframe_job Temporal decomposition: year, month, day, day_of_week, hour, minute from submit_time Shared node tracking: shared_job_count, nodes_shared, jobs_shared Key Variables: Scheduling: job_id, partition, state_simple, submit_time, start_time, end_time, queue_wait Resources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max/min/avg_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, consumed_energy_raw_joules, consumed_energy_raw_watt_hours Sharing: shared_job_count, nodes_shared, jobs_shared Partitions: short, standard, debug, gpu-h100 Job States: CANCELLED, COMPLETED, FAILED, PENDING, RUNNING QoS Levels: normal, high Important Notes: Timestamps include timezone offsets; DST transitions are handled correctly, though adding intervals across DST boundaries requires offset adjustment shared_job_count reflects physical node co-residency, not use of the shared partition Job step records and raw Slurm JSONB fields are excluded Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗