Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Metadata Normalization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

62 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Data from: "Towards CONUS-Wide ML-Augmented Conceptually-Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics"

This data package was generated to support the manuscript “Towards CONUS-Wide Machine Learning-Augmented Conceptually Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics.” It provides input files, model outputs, plotting data, scripts, notebooks, and documentation used to develop, evaluate, and reproduce Mass-Conserving Perceptron (MCP)-based hydrologic modeling experiments across 513 selected Catchment Attributes and Meteorology for Large-sample Studies in the United States (CAMELS-US) basins. The files are organized by modeling component and analysis purpose, including rainfall–runoff experiments, snow module experiments, coupled hydrologic-snow experiments, Long Short-Term Memory (LSTM) benchmark results, model skill metrics, initialization and epoch records, cell-state normalization files, Akaike Information Criterion (AIC)-based model comparison files, and data used to generate manuscript figures. Tabular files can be opened using standard spreadsheet software or Python/R data-analysis tools. Python scripts, Jupyter notebooks, and selected MATLAB scripts are included for model execution, postprocessing, plotting, and statistical analysis. Quality assurance and quality control were conducted through the source-data selection and modeling workflow. Meteorological forcing, streamflow, and static catchment attributes were derived from the CAMELS-US dataset, and snow water equivalent data were derived from the University of Arizona (UA) Snow Water Equivalent dataset. Selected basins and time periods were screened during the associated research workflow to avoid missing observations or poor-quality cases. Static geospatial features were processed primarily using Quantum Geographic Information System (QGIS) and Geospatial Data Abstraction Library (GDAL) workflows. Additional details are provided in the associated manuscript and documentation.

ESS-DIVE CSV File Formatting Guidelines Reporting ↗
DOE OSTI2025

Data for Roebuck et al. (2025), "Differences in dissolved organic matter composition between rivers and estuaries is conserved across freshwater and saltwater coastal regions"

Dissolved organic matter (DOM) in coastal surface waters influences local water quality and is an important component of biogeochemical cycling in coastal systems, but the processes that alter DOM composition along lower reaches of rivers and estuarine waters are poorly understood. Roebuck et al. (2025) leveraged a spatially distributed community sampling effort in coastal ecosystems across two regions to identify broad spatial drivers of surface water DOM composition and identify transferable trends between saltwater and freshwater coastal systems. Samples were collected by community members from 47 locations within the mid-Atlantic and Great Lakes coastal regions.This dataset includes:* A selection of commonly reported absorbance and fluorescence peaks normalized to dissolved organic carbon concentrations* Parallel factor output from the EC1 fluorescence datasets* A selection of commonly reported absorbance and fluorescence peaks * Spectral indices output from matlab script for absorbance and fluorescence datasets* CO2sys calculations of pH changes under varying temperatures and a constant salinity, DIC, and alkalinity concentrationAll data files are plain-text CSV (comma separated value) and no special software is required to read them.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

The importance of accounting for landscape position when investigating grasslands: A multidisciplinary characterisation of a Californian coastal grassland

Data from the characterisation of the Point Reyes Field Site, published in AGU Earth's Future under the title: The importance of accounting for landscape position when investigating grasslands: A multidisciplinary characterisation of a Californian coastal grassland. This paper explored the effect of landscape position on the response of a Californian grassland to seasonal changes. All files are csv files. The EMI data contains 8 csv files with a metadata csv explaining the columns. The dataset also includes soil variables including total concentrations calculated from fused samples, then dissolved and measured on ICP-AES for whole-rock elements and ICP-MS for trace elements. Mineral composition was attained using X-ray diffraction at BL 11-3 at SSRL . Data was then run through the High Score database to characterise different mineral phases. total It also includes a table with bulk soil characteristics such as soil pH, cation exchange capacity, and soil textural data. Data from Teros 12 Meter soil moisture, electrical conductivity and temperature sensors are presented in SMS Csv file. While the WL bottom and top files contain data from Piezometers measuring the ground water table. We have included a csv file that contains soil CO2 efflux data from Feb 2021-Oct 2021 in the Point Reyes Grassland Experiment We have included the spatially orientated (easting northing) remotely sensed datasets that were used in the K-means clustering analysis conducted on our site with electrical conductivity, normalised difference vegetation index, elevation, slope, solar radiation, topographic position and wetness index, and a clustering score. Finally there is a list of all the identified grassland species at the site.For more information on flux data, please contact the corresponding author.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

iButton and Tinytag snow/ground interface temperature measurements at Teller 27 and Kougarok 64 from 2022-2023, Seward Peninsula, Alaska

Snow/ground interface temperature measurements were collected at the NGEE Arctic Teller Road Site at mile marker 27 (TL_MM27) and at the Kougarok Road Site at mile marker 64 (KG_MM64) on the Seward Peninsula, Alaska. Data were collected between October 1, 2022 to September 18, 2023 using iButton Link DS1921G-F5# Thermochron miniature temperature sensors (https://www.ibuttonlink.com/products/ds1921g) and Tinytag TGP-4017 internal sensors (https://www.micronmeters.com/product/tgp-4017-internal-sensor-40-to-85-c-40-f-to-185-f) deployed across the Kougarok and Teller sites. These sensors are a cost-efficient way to collect snowpack temperatures at a higher spatial resolution than what is normally achieved. iButton data were collected every 4 hours, while Tinytag data were collected every 30 minutes. In total, data were collected from 196 iButtons and 26 Tinytags. This dataset contains four *.csv files of near-ground surface temperatures at various locations throughout each study site and two *.kml files of sensor locations. Data were collected throughout the snow cover season so that snowpack characteristics could be derived using the temperature data. Sensors were placed both inside and outside of vegetation to better capture the spatial variability of snow properties across each domain.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

iButton snow temperature measurements at Teller 27 and Kougarok 64, Seward Peninsula, Alaska, 2019-2020

Snow temperature measurements were collected at the NGEE Arctic Teller Road Site at mile marker 27 (TL_MM27) and at the Kougarok Road Site at mile marker 64 (KG_MM64) on the Seward Peninsula. Data were collected from October 1, 2021 to August 16, 2022 using iButton Link DS1926-F5# Thermochron miniature temperature sensors (https://www.ibuttonlink.com/products/ds1921g) and Tinytag TGP-4017 internal sensors (https://www.micronmeters.com/product/tgp-4017-internal-sensor-40-to-85-c-40-f-to-185-f) deployed across the Kougarok and Teller sites. These sensors are a cost-efficient way to collect snowpack temperatures at a higher spatial resolution than what is normally achieved. iButton data were collected every 4 hours beginning on October 1, 2021. Tinytag data collection began between October 9, 2021 and October 12, 2021 depending on the specific Tinytag and data were collected every 30 minutes. In total, data were collected from 236 iButtons and 30 Tinytags. This dataset contains four *.csv files of near-ground surface temperatures at various locations throughout each study site and four *.shp files of sensor locations. Data were collected throughout the snow cover season so that snowpack characteristics could be derived using the temperature data. Sensors were placed both inside and outside of vegetation to better capture the spatial variability of snow properties across each domain.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

iButton and Tinytag snow temperature measurements at Teller 27 and Kougarok 64, Seward Peninsula, Alaska, 2021-2022

Snow temperature measurements were collected at the NGEE Arctic Teller Road Site at mile marker 27 (TL_MM27) and at the Kougarok Road Site at mile marker 64 (KG_MM64) on the Seward Peninsula. Data were collected from October 1, 2021 to August 16, 2022 using iButton Link DS1926-F5# Thermochron miniature temperature sensors (https://www.ibuttonlink.com/products/ds1921g) and Tinytag TGP-4017 internal sensors (https://www.micronmeters.com/product/tgp-4017-internal-sensor-40-to-85-c-40-f-to-185-f) deployed across the Kougarok and Teller sites. These sensors are a cost-efficient way to collect snowpack temperatures at a higher spatial resolution than what is normally achieved. iButton data were collected every 4 hours beginning on October 1, 2021. Tinytag data collection began between October 9 and October 12, 2021 depending on sensor installation date. Tinytag data were collected every 30 minutes. In total, data were collected from 236 iButtons and 30 Tinytags. This dataset contains four *.csv files of near-ground surface temperatures at various locations throughout each study site and four *.shp files of sensor locations. Data were collected throughout the snow cover season so that snowpack characteristics could be derived using the temperature data. Sensors were placed both inside and outside of vegetation to better capture the spatial variability of snow properties across each domain. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

iButton snow-ground interface temperature measurements in Los Alamos, New Mexico from 2023-2024

Snow/ground interface temperature measurements were collected at two sites in Los Alamos, New Mexico. Data were collected from November 29, 2023 to April 8, 2024 using iButton Link DS1921G-F5# Thermochron miniature temperature sensors (https://www.ibuttonlink.com/products/ds1921g). These sensors are a cost-efficient way to collect snowpack temperatures at a higher spatial resolution than what is normally achieved. iButton data were collected every 3 hours from a total of 19 iButtons. iButtons were placed in pairs, with one iButton placed at the ground surface and another buried 1 - 5 cm below the ground surface. One buried iButton did not successfully collect data, and therefore was excluded from this dataset. Data were collected throughout the snow cover season so that snowpack characteristics could be derived using the temperature data. Specifically, this dataset was used as a validation source for a novel machine learning approach to estimating snow depth (see related publication). Sensors were placed in areas with bare ground or minimal grass coverage, located away from any large vegetation. At Site A (TA51), manual snow depths were collected as validation data. These measurements were taken next to iButtons periodically throughout the winter, and notes on other precipitation types were also recorded. At Site B (TA6 Meteorological Station), a nearby sensor collected snow depths throughout the winter. This dataset contains one *.csv file of snow/ground interface temperatures at two sites, one *.csv file of manually collected snow depths, and one *.kml file of sensor locations. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗