Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “system partition”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelism

Graph neural networks (GNNs), an emerging class of machine learning models for graphs, have gained popularity for their superior performance in various graph analytical tasks. Mini-batch training is commonly used to train GNNs on large graphs, and data parallelism is the standard approach to scale mini-batch training across multiple GPUs. Data parallel approaches contain redundant work as subgraphs sampled by different GPUs contain significant overlap. To address this issue, we introduce a hybrid parallel mini-batch training paradigm called Split parallelism. Split parallelism avoids redundant work by splitting the sampling, loading, and training of each mini-batch across multiple GPUs. Split parallelism, however, introduces communication overheads that can be more than the savings from removing redundant work. We further present a lightweight partitioning algorithm that probabilistically minimizes these overheads. We implement spllit parllelism in GSplit and show that it outperforms state-of-the-art mini-batch training systems like DGL, Quiver, and P3.

Lim, Seung-Hwan [ORNL] (ORCID:0000000194616866)↗
DOE OSTI2020

A micromirror array with annular partitioning for high-speed random-access axial focusing

Dynamic axial focusing functionality has recently experienced widespread incorporation in microscopy, augmented/virtual reality (AR/VR), adaptive optics and material processing. However, the limitations of existing varifocal tools continue to beset the performance capabilities and operating overhead of the optical systems that mobilize such functionality. The varifocal tools that are the least burdensome to operate (e.g. liquid crystal, elastomeric or optofluidic lenses) suffer from low (≈100 Hz) refresh rates. Conversely, the fastest devices sacrifice either critical capabilities such as their dwelling capacity (e.g. acoustic gradient lenses or monolithic micromechanical mirrors) or low operating overhead (e.g. deformable mirrors). Here, we present a general-purpose random-access axial focusing device that bridges these previously conflicting features of high speed, dwelling capacity and lightweight drive by employing low-rigidity micromirrors that exploit the robustness of defocusing phase profiles. Geometrically, the device consists of an 8.2 mm diameter array of piston-motion and 48-μm-pitch micromirror pixels that provide 2π phase shifting for wavelengths shorter than 1100 nm with 10–90% settling in 64.8 μs (i.e., 15.44 kHz refresh rate). The pixels are electrically partitioned into 32 rings for a driving scheme that enables phase-wrapped operation with circular symmetry and requires <30 V per channel. Optical experiments demonstrated the array’s wide focusing range with a measured ability to target 29 distinct resolvable depth planes. Overall, the features of the proposed array offer the potential for compact, straightforward methods of tackling bottlenecked applications, including high-throughput single-cell targeting in neurobiology and the delivery of dense 3D visual information in AR/VR.

42 ENGINEERING↗
DOE OSTI2022

Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations

Computational Fluid Dynamics - Discrete Element Method is used to model gas-solid systems in several applications in energy, pharmaceutical and petrochemical industries. Computational performance bottlenecks often limit the problem sizes that can be simulated at industrial scale. The data structures used to store several millions of particles in such large-scale simulations have a large memory footprint that does not fit into the processor cache hierarchies on current high-performance-computing platforms, leading to reduced computational performance. This paper specifically addresses this aspect of memory access bottlenecks in industrial scale simulations. The use of space-filling curves to improve memory access patterns is described and their impact on computational performance is quantified in both shared and distributed memory parallelization paradigms. The Morton space filling curve applied to uniform grids and k-dimensional tree partitions are used to reorder the particle data-structure thus improving spatial and temporal locality in memory. The performance impact of these techniques when applied to two benchmark problems, namely the homogeneous-cooling-system and a fluidized-bed, are presented. We report these optimization techniques lead to approximately two-fold performance improvement in particle focused operations such as neighbor-list creation and data-exchange, with ~ 1.5 times overall improvement in a fluidization simulation with 1.27 million particles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Modular and Transferable Reinforcement Learning Framework for the Fleet Rebalancing Problem

Mobility on demand (MoD) systems show great promise in realizing flexible and efficient urban transportation. However, significant technical challenges arise from operational decision making associated with MoD vehicle dispatch and fleet rebalancing. For this reason, operators tend to employ simplified algorithms that have been demonstrated to work well in a particular setting. To help bridge the gap between novel and existing methods, we propose a modular framework for fleet rebalancing based on model-free reinforcement learning (RL) that can leverage an existing dispatch method to minimize system cost. In particular, by treating dispatch as part of the environment dynamics, a centralized agent can learn to intermittently direct the dispatcher to reposition free vehicles and mitigate against fleet imbalance. We formulate RL state and action spaces as distributions over a grid partitioning of the operating area, making the framework scalable and avoiding the complexities associated with multiagent RL. Numerical experiments, using real-world trip and network data, demonstrate that RL reduces waiting time by 28% to 38% for the same-day evaluation, 17% to 44% for cross-day evaluation, and 22% to 25% for cross-season evaluation compared with no rebalancing scenarios. This approach has several distinct advantages over baseline methods including: improved system cost; high degree of adaptability to the selected dispatch method; and the ability to perform scale-invariant transfer learning between problem instances with similar vehicle and request distributions.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Elucidating the speciation of extracted lanthanides by diglycolamides

Many studies over the past few decades have been devoted to addressing the application of diglycolamides (DGAs) for hydrometallurgical-based, f-element separations. Work to date has shown the molecular structure of a DGA derivative can have a significant impact on intra-lanthanide partitioning patterns. More recent studies have pushed towards probing the structure function relationship of the lanthanide-DGA complex to enable the design of more efficient lanthanide separation systems. Spectroscopic techniques, such as UV–Visible, Fourier Transformed Infrared Spectroscopy (FT-IR), Nuclear Magnetic Resonance (NMR), and Extended X-ray Absorption Fine Structure (EXAFS), provide information regarding the inner-sphere coordination of a given lanthanide-DGA complex. Scattering techniques, such as Dynamic Light Scattering (DLS), Small-Angle X-ray Scattering (SAXS), and Small-Angle Neutron Scattering (SANS), address nanoscale structures including aggregate sizes and morphology. Here, this review assesses the current state-of-knowledge regarding lanthanide-DGA hydrometallurgical (i.e., solvent extraction systems) interrogated using spectroscopic and scattering techniques to characterize the extracted Ln 3+ DGA species. Of particular interest to this review is the impact of varied diluents, inclusion and variation of phase modifiers, and DGA derivatization on system characteristics. While there has been extensive literature on the application of DGAs for f-element separations, the literature lacks a collective assessment of the speciation of Ln 3+ in the organic phase. This review provides new insights into the field of DGA separations, explicitly with an application to intra-lanthanide separations. Specifically, this review illustrates the importance of both the co-extraction anion (Cl - , NO 3 - , or ClO 4 - ) as it pertains to both the aggregate size and Ln 3+ distribution coefficient. It is evident the ability of the anion to disrupt the hydrogen bonding network limits both aggregate size and distribution coefficients according to the Hoffmeister series. This suggests the importance of the large, softer anions with a low charge-to-surface area ratio on encouraging hydrogen bond interactions. In addition, the co-extracted cation (H + vs Na + ) is important for mitigating transfer of Ln 3+ from the aqueous to the organic phase through extensive hydrogen bonding networks. These networks are responsible for forming supramolecular aggregates where a change in morphology is observed with increasing concentrations of H + and/or Ln 3+ in the organic phase.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

The effect of relative humidity on eddy covariance latent heat flux measurements and its implication for partitioning into transpiration and evaporation

While the eddy covariance (EC) technique is a well-established method for measuring water fluxes (i.e., evaporation or 'evapotranspiration’, ET), the measurement is susceptible to many uncertainties. One such issue is the potential underestimation of ET when relative humidity (RH) is high (> 70%), due to low-pass filtering with some EC systems. Yet, this underestimation for different types of EC systems (e.g. open-path or closed-path sensors) has not been characterized for synthesis datasets such as the widely used FLUXNET2015 dataset. Here, we assess the RH-associated underestimation of latent heat fluxes (LE, or ET) from different EC systems for 163 sites in the FLUXNET2015 dataset. We found that the LE underestimation is most apparent during hours when RH is higher than 70%, predominantly observed at sites using closed-path EC systems, but the extent of the LE underestimation is highly site-specific. We then propose a machine learning based method to correct for this underestimation, and compare it to two energy balance closure based LE correction approaches (Bowen ratio correction, BRC, and attributing all errors to LE). Our correction increases LE by 189% for closed-path sites at high RH (> 90%), while BRC increases LE by around 30% for all RH conditions. Additionally, we assess the influence of these corrections on ET-based transpiration (T) estimates using two different ET partitioning methods. Results show opposite responses (increasing vs. slightly decreasing T-to-ET ratios, T/ET) between the two methods when comparing T based on corrected and uncorrected LE. Overall, our results demonstrate the existence of a high RH bias in water fluxes in the FLUXNET2015 dataset and suggest that this bias is a pronounced source of uncertainty in ET measurements to be considered when estimating ecosystem T/ET and WUE.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Coherency-Aware Learning Control of Inverter-Dominated Grids: A Distributed Risk-Constrained Approach

Here, this letter investigates the importance of integrating the coherency knowledge for designing controllers to dampen sustained oscillations in wide-area power networks with significant penetration of inverter-interfaced resources. Coherency is a fundamental property of power systems, where time-scale separation in frequency dynamics leads to clustered behavior among generators of different groups. Large-scale penetration of inverter-driven low inertia resources replacing conventional synchronous generators (SGs) can lead to perturbation in the coherent partitioning; hence, integrating such information is of utmost importance for oscillation control designs. We present the coherency-aware design of a distributed output feedback-based reinforcement learning method that additionally incorporates risk constraints to capture the uncertainties related to net-load fluctuations. The use of domain-aware coherency information has produced improved training and oscillation performance than the coherency-agnostic control design, hence proving to be effective in controller design. Finally, we validated the proposed method with numerical experiments on the benchmark IEEE 68-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Dynamic Oscillation and Motion of Oil‐in‐Water Emulsion Droplets

Abstract The interplay of interfacial tensions on droplets results in a range of self‐powered motions that mimic those of living systems and serve as a tunable model to understand their complex non‐equilibrium behavior. Spontaneous shape deformations and oscillations are crucial features observed in nature but difficult to incorporate in synthetic artificial systems. Here, we report sessile oil‐in‐water emulsions that exhibit rapid oscillating behavior. The oscillations depend on the nature and concentration of the surfactant, the chemical composition of the oil, and the wettability of the solid substrate. The rapid changes in the contact angle per oscillation are observed using side‐view optical microscopy. We propose that the changes in the interfacial tension of the oil droplets is due to the partitioning of the surfactant into the oil phase and the movement of self‐emulsified oil out of the parent droplets giving rise to the rhythmic variation in droplet contact‐line. The ability to control and understand droplet oscillation can help model similar oscillations in out‐of‐equilibrium systems in nature and reproduce biomimetic behavior in artificial systems for various applications, such as microfluidic lab‐on‐a‐chip and adaptive materials.

Krishna Mani, Sanjana↗
DOE OSTI2023

Dynamic Oscillation and Motion of Oil‐in‐Water Emulsion Droplets

The interplay of interfacial tensions on droplets results in a range of self‐powered motions that mimic those of living systems and serve as a tunable model to understand their complex non‐equilibrium behavior. Spontaneous shape deformations and oscillations are crucial features observed in nature but difficult to incorporate in synthetic artificial systems. Here, we report sessile oil‐in‐water emulsions that exhibit rapid oscillating behavior. The oscillations depend on the nature and concentration of the surfactant, the chemical composition of the oil, and the wettability of the solid substrate. The rapid changes in the contact angle per oscillation are observed using side‐view optical microscopy. We propose that the changes in the interfacial tension of the oil droplets is due to the partitioning of the surfactant into the oil phase and the movement of self‐emulsified oil out of the parent droplets giving rise to the rhythmic variation in droplet contact‐line. The ability to control and understand droplet oscillation can help model similar oscillations in out‐of‐equilibrium systems in nature and reproduce biomimetic behavior in artificial systems for various applications, such as microfluidic lab‐on‐a‐chip and adaptive materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Fast and Scalable Sparse Triangular Solver for Multi-GPU Based HPC Architectures

Designing efficient and scalable sparse linear algebra kernels on modern multi-GPU based HPC systems is a daunting task due to significant irregular memory references and workload imbalance across the GPUs. This is particularly the case for \textit{Sparse Triangular Solver (SpTRSV)} which introduces additional two-dimensional computation dependencies among subsequent computation steps. Dependency information is exchanged and shared among GPUs, thus warrant for efficient memory allocation, data partitioning, and workload distribution as well as fine-grained communication and synchronization support. In this work, we demonstrate that directly adopting unified memory can adversely affect the performance of SpTRSV on multi-GPU architectures, despite linking via fast interconnect like NVLinks and NVSwitches. Alternatively, we employ the latest NVSHMEM technology based on Partitioned Global Address Space programming model to enable efficient fine-grained communication and drastic synchronization overhead reduction. Furthermore, to handle workload imbalance, we propose a malleable task-pool execution model which can further enhance the utilization of GPUs. By applying these techniques, our experiments on the NVIDIA multi-GPU supernode V100-DGX-1 and DGX-2 systems demonstrate that our design can achieve on average 3.53x (up to 9.86x) speedup on a DGX-1 system and 3.66x (up to 9.64x) speedup on a DGX-2 system with 4-GPUs over the Unified-Memory design. The comprehensive sensitivity and scalability studies also show that the proposed zero-copy SpTRSV is able to fully utilize the computing and communication resources of the multi-GPU system.

Xie, Chenhao↗
DOE OSTI2021

Importance of charge self-consistency in first-principles description of strongly correlated systems

First-principles approaches have been successful in solving many-body Hamiltonians for real materials to an extent when correlations are weak or moderate. As the electronic correlations become stronger often embedding methods based on first-principles approaches are used to better treat the correlations by solving a suitably chosen many-body Hamiltonian with a higher level theory. The success of such embedding theories, often referred to as second-principles, is commonly measured by the quality of self-energy Σ which is either a function of energy or momentum or both. However, Σ should, in principle, also modify the electronic eigenfunctions and thus change the real space charge distribution. While such practices are not prevalent, some works that use embedding techniques do take into account these effects. In such cases, choice of partitioning, of the parameters defining the correlated Hamiltonian, of double-counting corrections, and the adequacy of low-level Hamiltonian hosting the correlated subspace hinder a systematic and unambiguous understanding of such effects. Further, for a large variety of correlated systems, strong correlations are largely confined to the charge sector. Then an adequate nonlocal low-order theory is important, and the high-order local correlations embedding contributes become redundant. Here we study the impact of charge self-consistency within two example cases, TiSe 2 and CrBr 3 , and show how real space charge re-distribution due to correlation effects taken into account within a first-principles Green’s function-based many-body perturbative approach is key in driving qualitative changes to the final electronic structure of these materials.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Advances and challenges in CO2 foam technologies for enhanced oil recovery in carbonate reservoirs

Utilization of surfactants for generating foam has the potential for fluid-mobility control during CO2 flooding processes, leading to improved oil recovery. This manuscript is a review of the most important aspects for the design of CO2-foams with mobility control in carbonate reservoirs, including recent advances in novel surfactants and analytical techniques for analysis of surfactant adsorption on carbonate minerals and their thermal stability. Several key parameters and properties regarding foam transport in porous media are reviewed such as, the minimum pressure gradient required for foam generation, the effects of the partition coefficient of a CO2-soluble surfactant on foam transport in rock cores, visualization of foam flow by either microfluidic or PET/CT imaging processes in heterogeneous or fractured systems and the effect of oil on foam transport under reservoir conditions. Also are discussed the most recent advances of two major foam modeling methods, including the semi-empirical STARS foam model and the population balance model, as well as CO2 foam pilot tests and the factors governing their successes.

Jian, Guoqing↗
DOE OSTI2024

PPI DataHub Project Data Package: S. elongatus PCC 7942 Circadian Control Bioproduction Transcriptomics (PB-DP3)

The purpose of this experiment was to evaluate how circadian clock regulation impacts carbon partitioning between storage, growth, and product synthesis in Synechococcus elongatus PCC 7942 in providing insights to strategies for enhanced bioproduction. Sample data was acquired using a Illumina HiSeq sequencer system and processed for RNA sequencing (RNA-Seq) expression analysis. Transcriptomic differential expression analysis revealed coordinated circadian clock-driven adjustment of the cell cycle and rewiring of energy and carbon metabolism. Processed RNA-Seq datasets are openly accessible from the PNNL DataHub project dataset download page and contain secondary processed RNA-seq results files and supporting metadata materials linked to relevant source code information supporting data transparency and reuse.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Partitioning of Radionuclides in Various Streams of the Mo-99 Purification Process

The Universal Neutron Irradiator (UNI) was designed to irradiate a small volume of uranium solution and to achieve high fission power without creating a large radionuclide (RN) inventory. The UNI system is equipped with a fully enclosed off-gas system with online hydrogen gas monitoring to prevent the production of an explosive atmosphere, as well as storage cylinders for capturing fission gases. It also includes a catalytic recombiner in the target capsule (to immediately recombine radiolytic hydrogen and oxygen gas in the system) and access ports for sampling the target solution and capsule headspace. In UNI Phase I experiments, 50 mL of uranyl sulfate solution was irradiated at 0.3–0.5 W/mL fission power using a tantalum target assembly to produce photoneutrons via irradiation with an electron beam. The purposes of the Phase I experiments were to test the efficacy of adding 200 ppm Fe 2+ to the target solution to prevent uranyl peroxide precipitation under representative irradiation conditions, and to monitor iodine speciation in the irradiated uranyl sulfate solution. The focus of the UNI Phase II experiments was to use an irradiated uranyl sulfate solution containing a mixture of fission products and monitor their distribution in various streams during the primary recovery column (titania-based column) that separates Mo-99 from uranium. Irradiations were designed to create sufficient activity for tracking their distribution in various processing streams using a combination of gamma counting and inductively coupled plasma mass spectrometry (ICP-MS) analyses. Obtained data will be used to better understand the composition of waste streams, accumulation of RN on the titania column, and identify what RNs accumulate in the uranyl sulfate solution. Furthermore, this knowledge will allow us to determine waste classifications for various streams produced during the Mo-99 purification process and develop an appropriate waste clean-up strategy.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Decomposing a renewable energy design and dispatch model

We address a mixed-integer linear programming model which selects a cost-minimizing set of available technologies with which to design a renewable energy system and prescribe their associated dispatch decisions. Realistically sized instances of such models pose computational challenges. To this end, we develop a Lagrangian heuristic based on a decomposition methodology which partitions the model into blocks and optimizes these more manageable, smaller subproblems. It also provides a lower bound to assess solution quality. In conclusion, we apply this methodology to the National Renewable Energy Laboratory's Renewable Energy Integration and Optimization (REopt TM ) model to generate near-optimal solutions to realistic instances containing, on average, approximately 300,000 variables and at least as many constraints, with a mean 30% optimality gap improvement using a five-minute solution time limit, compared to directly solving the original monolith.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Entanglement Effect and Angular Momentum Conservation in a Nonseparable Tunneling Treatment

The important, and often dominant, role of tunneling in low temperature kinetics has resulted in numerous theoretical explorations into the methodology for predicting it. Nevertheless, there are still key aspects of the derivations that are lacking, particularly for nonseparable systems in the low temperature regime, and further explorations of the physical factors affecting the tunneling rate are warranted. In this work we obtain a closed-form rate expression for the tunneling rate constant that is a direct analog of the rigid-rotor-harmonic-oscillator expression. This expression introduces a novel "entanglement factor" that modulates the reaction rate. Furthermore, we are able to extend this expression, which is valid for nonseparable systems at low temperatures, to properly account for the conservation of angular momentum. In contrast, previous calculations have considered only vibrational transverse modes and so effectively employ a decoupled rotational partition function for the orientational modes. We also suggest a simple theoretical model to describe the tunneling effects in the vicinity of the crossover temperature (the temperature where tunneling becomes the dominating mechanism). This model allows one to naturally classify, interpret, and predict experimental data. Among other things, it quantitatively explains in simple terms the so-called "quantum bobsled" effect, also known as the negative centrifugal effect, which is related to curvature of the reaction path. Taken together, the expressions obtained here allow one to predict the thermal and E-resolved rate constants over broad ranges of temperatures and energies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

NLR HPC Kestrel Jobs Data

Overview: Anonymized job-level records from the Kestrel HPC system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, utilization, energy estimates, and efficiency metrics. Sensitive fields (user, account, job name, submit line, working directory, submit script, and job type) are replaced with 7-character cryptographic hashes. System & Timeframe: Kestrel is located at the NLR campus. Standard compute nodes have 104 cores and 256 GB RAM; bigmem nodes have 2,000 GB. GPU nodes (gpu-h100 partition) use NVIDIA H100 GPUs. Data covers jobs submitted August 2023 through December 2025. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.kestrel.job-anon.zip — Anonymized job records (Hive-partitioned Parquet) datacard.md — Full dataset documentation ~11 million rows, 50 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct with timezone-aware export (SLURM_TIME_FORMAT="%Y-%m-%dT%H:%M:%S%z"), loaded into PostgreSQL. Calculated columns updated via database triggers and batch functions. All timestamps use timestamptz and correctly handle DST transitions. Preprocessing: Anonymization of name, user, account, submit_line, work_dir, submit_script, and job_type via 7-char hex hashes Derived columns: queue_wait, cpu_eff, max/min/avg_mem_eff, energy estimates Simplified job state mapping (e.g., "CANCELLED by 132357" → "CANCELLED") Boolean flags: python_job, reframe_job Temporal decomposition: year, month, day, day_of_week, hour, minute from submit_time Shared node tracking: shared_job_count, nodes_shared, jobs_shared Key Variables: Scheduling: job_id, partition, state_simple, submit_time, start_time, end_time, queue_wait Resources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max/min/avg_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, consumed_energy_raw_joules, consumed_energy_raw_watt_hours Sharing: shared_job_count, nodes_shared, jobs_shared Partitions: short, standard, debug, gpu-h100 Job States: CANCELLED, COMPLETED, FAILED, PENDING, RUNNING QoS Levels: normal, high Important Notes: Timestamps include timezone offsets; DST transitions are handled correctly, though adding intervals across DST boundaries requires offset adjustment shared_job_count reflects physical node co-residency, not use of the shared partition Job step records and raw Slurm JSONB fields are excluded Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗