Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “power system monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

TEAMER – Field Demonstration of MarineSitu’s Marine Energy Monitoring (Abstract)

In order to effectively monitor for marine life around marine energy devices and thus minimize the risk of collision, multiple sensors working in coordination and augmented with around-the-clock automated monitoring algorithms need to be installed in challenging high-energy tidal and wave environments. Such systems are often too expensive for widespread adoption, or lack sufficient sensors or smarts to enable around-the-clock, real-time monitoring without human involvement. MarineSitu has been working to tackle this problem by developing a low-cost, combined sonar and stereo camera sensor array with connected real-time AI-based algorithms for automatically detecting marine life in these marine energy suitable environments. In this TEAMER project with Pacific Northwest National Lab (PNNL), MarineSitu will be testing this novel sensor system for the first time in the high-energy tidal channel environment at PNNL’s Marine and Coastal Research Lab. Throughout this deployment, MarineSitu will be monitoring their system and running analytics on the sensor’s data in real-time. Meanwhile, PNNL Data Scientists and Ocean Engineers, will be evaluating the system’s effectiveness and ease of use both as a tool for plug-and-play environmental monitoring and novel environmental monitoring research. In doing so, the team will improve MarineSitu’s system and software, produce insightful data products, and develop novel visualizations and AI algorithms for combining and analyzing the data produced by systems like MarineSitu’s.

16 TIDAL AND WAVE POWER↗
DOE OSTI2022

Characterizing Plug Load Energy Use and Savings Potential in Army Buildings

The Assistant Secretary of the Army (Installations, Energy and Environment) tasked the Pacific Northwest National Laboratory to examine plug loads in typical Army buildings. Plug loads (also known as miscellaneous electric loads (MELs)) represent the electricity used by appliances and devices that are plugged in or hardwired and serve functions outside of a building’s core end uses. Common plug loads include computers, printers, copiers, networking devices, refrigerators, and vending machines. They also include personal electronic devices such as televisions, smart phones, tablets, and gaming systems. Examples of hardwired MELs include elevators, air compressors, and fire and security systems. The findings from this study confirm that significant energy is consumed within Army buildings by plug load devices and hardwired MEL equipment. A number of opportunities are identified for reducing unnecessary energy use that could save the Army over $5 million per year when broadly applied. Army regulations clearly spell out expectations for the purchase and operation of information technology equipment (computers, laptops, monitors, printers, and multi-function devices). However, the policies regarding the shutdown or activation of sleep and other lower power modes after 30 minutes of inactivity (15 minutes for monitors) do not appear to be consistently followed. There are many effective approaches and pathways for impacting change as it relates to improving awareness, implementing measures, and adjusting behaviors to identify and reduce plug load energy use. The Army should prioritize and consider deploying all of these to better understand and manage plug load equipment to save energy and enhance resilience across their facilities. Engaging the building occupants who use these devices daily via outreach and education should be a strong component of the strategy. The focus should be on reducing waste without sacrificing productivity or the benefits that many of these devices provide. Continued evaluation of plug loads beyond that performed here is important to gather lessons from additional building and equipment types, and to stay aware of evolving device technology and management options. This will highlight additional needs for policies, best practices, control technologies, and education of personnel to achieve real reductions in energy waste from plug load equipment. It is recommended that this study may serve as the foundation for a broader and sustained focus on plug loads and MELs, towards simultaneously enhancing the productivity, readiness, and resilience of the Army while reducing energy use and demand, and freeing up resources to better support the mission.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Voltage Estimation in Low-Voltage Distribution Grids with Distributed Energy Resources

Present distribution grids generally have limited sensing capabilities and are therefore characterized by low observability. Improved observability is a prerequisite for increasing the hosting capacity of distributed energy resources such as solar photovoltaics (PV) in distribution grids. In this context, this paper presents learning-aided low-voltage estimation using untapped but readily available and widely distributed sensors from cable television (CATV) networks. The cable broadband sensors offer timely local voltage magnitude sensing with 5-minute resolution and can provide an order of magnitude more data on the time-varying state of a secondary distribution system than currently deployed utility sensors. The proposed solution incorporates voltage readings from neighboring CATV sensors, taking into account spatio-temporal aspects of the observations, and estimates single-phase voltage magnitudes at all non-monitored low-voltage buses using random forests. The effectiveness of the proposed approach was demonstrated using a multi-phase 1572-bus feeder from the SMART-DS data set for two case studies passive distribution feeder (without PV) and active distribution feeder (with PV). The analysis was conducted on simulated data, and the results show voltage estimates with a high degree of accuracy, even at extremely low percentages of observable nodes.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Liquid Chromatography Coupled to Refractive Index or Mass Spectrometric Detection for Metabolite Profiling in Lysate-based Cell-free Systems

Engineering cellular metabolism for targeted biosynthesis can require extensive design-build-test-learn (DBTL) cycles as the engineer works around the cell's survival requirements. Alternatively, carrying out DBTL cycles in cell-free environments can accelerate this process and alleviate concerns with host compatibility. A promising approach to cell-free metabolic engineering (CFME) leverages metabolically active crude cell extracts as platforms for biomanufacturing and for rapidly discovering and prototyping modified proteins and pathways. Realizing these capabilities and optimizing CFME performance requires methods to characterize the metabolome of lysate-based cell-free platforms. That is, analytical tools are necessary for monitoring improvements in targeted metabolite conversions and in elucidating alterations to metabolite flux when manipulating lysate metabolism. In this work, metabolite analyses using high-performance liquid chromatography (HPLC) coupled with either optical or mass spectrometric detection were applied to characterize metabolite production and flux in E. coli S30 lysates. Specifically, this report describes the preparation of samples from CFME lysates for HPLC analyses using refractive index detection (RID) to quantify the generation of central metabolic intermediates and by-products in the conversion of low-cost substrates (i.e., glucose) to various high-value products. The analysis of metabolite conversion in CFME reactions fed with 13 C-labeled glucose through reversed-phase liquid chromatography coupled to tandem mass spectrometry (MS/MS), a powerful tool for characterizing specific metabolite yields and lysate metabolic flux from starting materials, is also presented. Altogether, applying these analytical methods to CFME lysate metabolism enables the advancement of these systems as alternative platforms for executing faster or novel metabolic engineering tasks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Process intensification approach to enhancing heat and mass transfer: Radio frequency (RF) assisted drying of paper and board

Conventional multi-cylinder drying of paper and board typically relies on both conductive drying from steam-heated cylinders and convective drying, where heated air flows over the paper web surface. Conduction primarily contributes to heat transfer, while convection is the main driver of mass transfer. However, conventional drying systems are heavily dependent on steam, usually powered by fossil fuels, and are often energy-inefficient with high levels of waste. Additionally, these surface-driven processes result in a lower percentage of energy absorption compared to the energy supplied, leading to significant energy losses. To improve this long-standing process, an experimental system was developed to investigate a process intensification approach involving the integration of Radio Frequency (RF) heating, a volumetric electromagnetic technology, alongside traditional conduction and convection drying methods. This study also emphasizes the use of sensors to continuously monitor key parameters such as moisture content, supply system temperatures, sample temperatures, air flows, and drying rates in real-time. The effect of RF as an auxiliary energy source in localized environments at varying moisture levels was explored to optimize industrial drying systems, quantify potential improvements, and provide insights for future studies. Experimental results from trials combining RF with convection and with the base case alternating conduction-convection drying processes are presented. It is shown that RF is a viable process intensification approach for paper drying improving the drying rate and energy intensity at higher moisture contents. These findings offer valuable insights for process intensification and contribute to the process development, modeling, and simulation of advanced paper drying techniques.

42 ENGINEERING↗
DOE OSTI2021

Degradation mechanisms and partial shading of glass-backsheet and double-glass photovoltaic modules in three climate zones determined by remote monitoring of time-series current–voltage and power datastreams

Degradation and partial shading impact the long-term reliability and power production of photovoltaic (PV) modules and power plants. Time-series power (P mp ) and current–voltage (I-V) curve datastreams from PV modules enable a remote diagnostic approach to quantify active degradation mechanisms and identify partial shading. We study three to nine years of these datastreams, including 3.6 million P mp I-V curves and 36 million values, from eight PV modules, four each of double-glass and glass-backsheet module architectures, located in three distinctly different Köppen-Geiger climate zones, to determine the module’s performance loss rates (PLR), identify active degradation mechanisms and power loss modes, along with partial shading by local objects. Considering both module architectures, PLR results indicate that the BSh climate zone is the most aggressive for module degradation, while the Alpine ET zone is the mildest climate. PLR of double-glass modules located in BWh and BSh climate zones are different due to the significantly greater uniform current loss (ΔP Isc ) for double-glass modules in BSh, at a 5% significance level. Power loss for four out of five modules located in the BWh and BSh climates are dominated by uniform current degradation. Statistical analysis of multistep I-V curves detects partial shading experienced by three studied modules with details of the shading profile, the shading Poynting vector diagram for the obstacle’s relative position, shading scenarios, and duration. Lastly, this work demonstrates how remote monitoring and diagnosis of P mp & I-V time-series of modules can provide quantitative operations and maintenance insights into system performance, degradation mechanisms, and shading.

14 SOLAR ENERGY↗
DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗
DOE OSTI2021

Online Monitoring System for Concrete Structures Affected by Alkali-Silica Reaction

This report presents a comprehensive study of developing ultrasonic wave and acoustic emission (AE) techniques for long-term monitoring of alkali-silica reaction (ASR) development in concrete. Small, medium, and full-scale concrete specimens were cast, conditioned, and monitored for various periods, from at least one year to 2.4 years. The reactive concrete specimens contain coarse or fine reactive aggregates to study the effects of different types of reactive aggregates. Confinements were also designed to simulate the 2-dimensional confinement effects of reinforcement in the shield building of nuclear power plants. The concrete specimens were stored in a environmental chamber with high humidity and high temperature to accelerate the ASR development. The ultrasonic monitoring data shows high sensitivity to ASR development and could detect cracking initiation well before visible surface cracks occurred. However, the linear ultrasonic analysis based on wave velocity is strongly affected by temperature variation. Therefore, a nonlinear ultrasonic method was proposed to measure thermally induced nonlinear acoustic responses of concrete (thermal modulation of ultrasonic wave). The measured nonlinear acoustic parameter shows a high correlation with ASR expansion across specimens with different reactive aggregates and confinement conditions. The same conclusion was obtained from nonlinear resonance tests on small concrete prisms. Compared to the linear acoustic methods, the nonlinear tests show high sensitivities to ASR damage from internal microcracking initiation at the early stage to visible cracks at the late stage of ASR. The attributes of the thermal modulation of nonlinear ultrasonic method include high sensitivity, immunity to temperature effects, and strong correlation with ASR expansion, which demonstrate great potentials of the nonlinear ultrasonic method for diagnosis of ASR damage and prediction of concrete deterioration process. Acoustic emission is a passive sensing technique for damage assessment, and access to only one surface is needed even for thick and heavily reinforced elements such as the walls utilized for nuclear shield building. Additionally, relatively few sensors are required to monitor the progression of the damage process. Results indicate that the AE data can be related to the damage rating index, which is a petrography-based means of assessing damage due to ASR in reinforced concrete. Furthermore, AE is capable of detecting ASR damage long before surface cracking is visually noticeable. Boundary conditions play in important role in the progression of ASR damage and differences in boundary conditions are reflected in the AE data. Entropy based data assessment methods provide a means to assess the damage state, and convolutional neural network based data assessment provided a means to assess the damage state in real-time. Results indicate that artificial neural network models may be used as a means to predict volumetric expansion based on AE data.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Recent Changes to U.S. Coal Plant Operations and Current Compensation Practices

Over the last decade, the U.S. electric power sector has gone through one of the most dramatic changes in its existence. The combination of low natural gas prices as a result of the shale gas revolution and significant reduction in construction and operating costs for renewable resources, in part due to federal tax credits such as the production tax credit and investment tax credit mainly benefitting wind and solar, respectively, has resulted in a significant shift away from coal-fired generation, and instead towards natural gas and renewable generation. Furthermore, the operating profile of existing coal-fired electric generating units has changed significantly. As new natural gas combined cycle plants have become increasingly more efficient and cheaper to operate than older existing coal-fired power plants, coal units continue to lose baseload generation share and more frequently operate as load-following, or cycling, resources. These trends are of particular importance to state public utility commissions. Functioning as economic regulators, commissions oversee investments in a reliable, efficient system while balancing emissions goals, customer demands, and other policy objectives. Changes to coal plant operations as a result of increased competition from other fuel sources may have a bearing on system reliability and economics, and therefore constitute an important area for commissions to monitor.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2022

IMS observations of infrasound and acoustic-gravity waves produced by the January 2022 volcanic eruption of Hunga, Tonga: A global analysis

The 15 January 2022 Hunga, Tonga, volcano's explosive eruption produced the most powerful blast recorded in the last century, with an estimated equivalent TNT yield of 100–200 megatons. The blast energy was propagated through the atmosphere as various wave types. The most prominent wave was a long-period (>2000 s) surface-guided Lamb wave with energy comparable to that of the 1883 Krakatoa Lamb wave; both were clearly observed by pressure sensors (barometers) worldwide. Internal gravity, acoustic-gravity, and infrasound waves were captured in great detail by the entire infrasound component of the International Monitoring System (IMS). For instance, infrasound waves (<300 s period) were seen to circumnavigate Earth up to eight times. Atmospheric waves captured by the IMS infrasound network and selected barometers near the source provide insight on Earth's impulse response at planetary scales.

58 GEOSCIENCES↗
DOE OSTI2021

Equalization of Intensity-Modulated Fiber-Optic Voltage Sensors for Power Distribution Systems

We test fiber-optic voltage sensors based on optical reflection from a piezoelectric transducer. Our specific devices possess a 2 kHz fundamental resonance, and we verify a readily usable frequency band from approximately 10 Hz to 3 kHz, with a dynamic range of 60 dB for a detection system integrating over this entire band. Additionally, we demonstrate a digital signal processing approach to equalize the measured frequency response, enabling accurate retrieval of short-pulse inputs. These results suggest the value and applicability of intensity-modulated fiber-optic voltage sensors for measuring both steady-state waveforms and broadband transients which, coupled with the straightforward and compact design of the sensors, should make them effective tools in electric grid monitoring.

42 ENGINEERING↗
DOE OSTI2022

High Temperature Thick Film Sensor Development Based on Refractory Oxide Semiconductors

Solid-state, high-temperature sensing devices are required for accurate temperature, strain/stress, and failure monitoring for a range of advanced manufacturing, transportation, and military applications. High temperature conditions limit sensing strategies, where typically traditional metal and semiconductor materials are unstable, and the sensing options are limited to optical spectroscopy methods. In addition, strategies to embed the sensors directly within active components and protective thermal refractory are of interest that permit near-environment sensing which is not possible with the traditional electronic materials and components. Rare-earth chromite semiconductors exhibit some properties of interest for high temperature sensing technologies, such as: high microstructure and chemical stability, relatively high electronic conductivity at high temperatures, and relative thermal expansion coefficient matching to refractory thermal protective insulation (such as high-zirconia and -alumina ceramics). In this work, n-type and p-type doped lanthanum chromites compositions were computationally modelled and prepared, and the electrical, thermoelectric, and thermomechanical properties were measured for the obtained material systems. A few compositions were then processed into thick film health and temperature sensors and tested at high temperatures. To establish relative stability under harsh environment conditions, post- mortem microstructure and chemical composition was characterized after extended and cycle testing at high temperatures.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2021

Design and evaluation of a cyber‐physical testbed for improving attack resilience of power systems

Abstract A power system is a complex cyber‐physical system whose security is critical to its function. A major challenge is to model, analyse and visualise the communication backbone of the power systems concerning cyber threats. To achieve this, the design and evaluation of a cyber‐physical power system (CPPS) testbed called Resilient Energy Systems Lab (RESLab) are presented to capture realistic cyber, physical, and protection system features. RESLab is architected to be a fundamental platform for studying and improving the resilience of complex CPPS to cyber threats. The cyber network is emulated using Common Open Research Emulator (CORE), which acts as a gateway for the physical and protection devices to communicate. The physical grid is simulated in the dynamic time frame using Power World Dynamic Studio (PWDS). The protection components are modelled with both PWDS and physical devices including the SEL Real‐Time Automation Controller (RTAC). Distributed Network Protocol 3 (DNP3) is used to monitor and control the grid. Then, the design is exemplified and the tools are validated. This work presents four case studies on cyberattack and defence using RESLab, where we demonstrate false data and command injection using Man‐in‐the‐Middle and Denial of Service attacks and validate them on a large‐scale synthetic electric grid.

Sahu, Abhijeet↗
DOE OSTI2020

Data Quality Analysis and Solutions for Distribution-Level PMUs

Synchrophasors provide promising enhancement to the distribution system monitoring. However, different from transmission systems, distribution systems are usually with higher noise, phase unbalance, and distributed energy resources. All of these could impair the data quality of distribution-level phasor measurement units (D-PMUs) and thus deteriorate the performance of D-PMU based applications. Therefore, the data quality of D-PMU should be evaluated to ensure the effectiveness of D-PMU based applications. In this paper, factors in distribution systems that could jeopardize D-PMU data quality are evaluated. The impact of data quality on D-PMU applications is analyzed. Finally, solutions are proposed to improve D-PMU data quality to satisfy these requirements. This work provides effective methods to improve the performance of distribution-level grid monitoring using synchrophasor measurements.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Machine-Protection System for the PIP-II Linear Accelerator: Architecture and Data Processing Schemes

PIP-II at Fermilab features a brand-new, 800 MeV leading-edge Linear Accelerator (Linac) that will enable the Fermilab complex to deliver more than a megawatt of beam power to the Long Base-line Neutrino Facility (LBNF), requiring a robust Machine Protection System (MPS) to prevent beam-induced damage to delicate cryomodules and vacuum components. Signals in various accelerator components, in both analog and digital format, are collected and processed to assest the operating conditions of the accelerator. In the situation a potential damage is foreseen, the beam permit signal will be drop. A critical element of this system is the fast Analog Machine Protection System, a high-bandwidth platform designed for real-time beam loss monitoring. The system utilizes a modular, FPGA-based architecture to digitize signals from beam-sensing devices, such as AC Current Transformers (ACCTs), non-invasive Ring Pickups (RPUs) and beam scrapers. To eliminate both high frequency as well as 60 Hz noise that might be picked up in the industrial environment, data processing schemes beyond regular digital filtering are planned and implemented.

Wu, Jinyuan [Fermilab] (ORCID:0000000344329521)↗
DOE OSTI2024

Improved Beam Loss Accounting with Fast Data Acquisition (DAQ) Chassis

Identifying the source of beam loss events in the CE-BAF accelerator can be a challenging task. However, with our new prototype system, this task becomes more effi-cient. The system, developed in the fall of 2022, utilizes a dispersive beam position monitor (BPM) and the exist-ing switched electrode electronics BPM hardware. Previ-ously a commercial off-the-shelf data acquisition (DAQ) system was employed to capture BPM wire signals at a sample rate of 20 kS/s. The fast shutdown signal triggered the system, which disables the beam at the injector. Analysis of beam position and energy variation before a beam loss event was used to determine if the beam loss event was associated with an energy transient. The proto-type system, implemented using National Instruments hardware and LabVIEW® software, relied on a software trigger. Manual post-processing was required to ascertain whether the fault was due to an un-tripped cavity with a gradient or phase transient. This work focuses on deploying a Fast DAQ Chassis to monitor BPM hardware in real time and during beam loss events. This system was originally developed and in-stalled in CEBAF to monitor the time-domain RF control signals in the legacy analog RF systems. This technology was leveraged to also monitor BPM signals. As the new system employs a hardware trigger, developing tools to automatically identify faults linked to energy transients unrelated to cavity faults will be straightforward. This paper will discuss the project's initial updates, underlin-ing the crucial role of each member of our team in this achievement

Tiskumara, J.↗
DOE OSTI2024

Vegetation Warming Experiment: Landscape-scale digital camera imagery for vegetation phenology, Utqiagvik (Barrow), Alaska, 2017

Images captured using a StarDot NetCam SC phenocamera looking east from the top of the Barrow Environmental Observatory (BEO) Sled Shed, Barrow (now Utqiagvik), Alaska. The camera was installed to remotely monitor plant phenology and operation of the BNL TEST group's ZPW (Zero Power Warming) chambers during the growing season of 2017. Images were captured from early spring (24 April) through to late fall (15 November). Snowmelt, vegetation growth and senescence, and snow accumulation were captured. Images were uploaded to the BNL FTP server every 10 minutes. jpg images have been compressed in *.tar.gz format (12.1 GB) and additional information in *.pdf and *.txt files. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗