Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “system metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Predicting the heat release variability of Li-ion cells under thermal runaway with few or no calorimetry data

Accurate measurement of the variability of thermal runaway behavior of lithium-ion cells is critical for designing safe battery systems. However, experimentally determining such variability is challenging, expensive, and time-consuming. Here, we utilize a transfer learning approach to accurately estimate the variability of heat output during thermal runaway using only ejected mass measurements and cell metadata, leveraging 139 calorimetry measurements on commercial lithium-ion cells available from the open-access Battery Failure Databank. We show that the distribution of heat output, including outliers, can be predicted accurately and with high confidence for new cell types using just 0 to 5 calorimetry measurements by leveraging behaviors learned from the Battery Failure Databank. Fractional heat ejection from the positive vent, cell body, and negative vent are also accurately predicted. We demonstrate that by using low cost and fast measurements, we can predict the variability in thermal behaviors of cells, thus accelerating critical safety characterization efforts.

25 ENERGY STORAGE↗
DOE OSTI2023

Fermionic quantum processing with programmable neutral atom arrays

Simulating the properties of many-body fermionic systems is an outstanding computational challenge relevant to material science, quantum chemistry, and particle physics.-5.4pc]Please note that the spelling of the following author names in the manuscript differs from the spelling provided in the article metadata: D. González-Cuadra, D. Bluvstein, M. Kalinowski, R. Kaubruegger, N. Maskara, P. Naldesi, T. V. Zache, A. M. Kaufman, M. D. Lukin, H. Pichler, B. Vermersch, Jun Ye, and P. Zoller. The spelling provided in the manuscript has been retained; please confirm. Although qubit-based quantum computers can potentially tackle this problem more efficiently than classical devices, encoding nonlocal fermionic statistics introduces an overhead in the required resources, limiting their applicability on near-term architectures. In this work, we present a fermionic quantum processor, where fermionic models are locally encoded in a fermionic register and simulated in a hardware-efficient manner using fermionic gates. We consider in particular fermionic atoms in programmable tweezer arrays and develop different protocols to implement nonlocal gates, guaranteeing Fermi statistics at the hardware level. We use this gate set, together with Rydberg-mediated interaction gates, to find efficient circuit decompositions for digital and variational quantum simulation algorithms, illustrated here for molecular energy estimation. Finally, we consider a combined fermion-qubit architecture, where both the motional and internal degrees of freedom of the atoms are harnessed to efficiently implement quantum phase estimation as well as to simulate lattice gauge theory dynamics.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2026

Hybrid-BPR (Bayesian Personalized Ranking with Feature Embeddings and Explicit Negative Sampling) [SWR-26-039]

Hybrid-BPR is a Python library for Bayesian Personalized Ranking (BPR) with two key capabilities that go beyond standard BPR implementations: 1. User and item feature embeddings - incorporate content-based signals (genres, tags, metadata) alongside collaborative filtering. 2. Implicit negative interactions - use observed non-interactions (e.g. viewed-but-not-clicked) as negative training signal instead of random sampling from the full item space. The software is built for recommender systems research with MLflow experiment tracking, parallel hyperparameter sweeps, and standard ranking metrics.

Sandhu, Rimple [National Laboratory of the Rockies↗
DOE OSTI2021

Maize specialized metabolome networks reveal organ-preferential mixed glycosides

Despite the scientific and economic importance of maize, little is known about its specialized metabolism. Here, five maize organs were profiled using different reversed-phase liquid chromatography-mass spectrometry methods. The resulting spectral metadata, combined with candidate substrate-product pair (CSPP) networks, allowed the structural characterization of 427 of the 5,420 profiled compounds, including phenylpropanoids, flavonoids, benzoxazinoids, and auxin-related compounds, among others. Only 75 of the 427 compounds were already described in maize. Analysis of the CSPP networks showed that phenylpropanoids are present in all organs, whereas other metabolic classes are rather organ-enriched. Frequently occurring CSPP mass differences often corresponded with glycosyl- and acyltransferase reactions. The interplay of glycosylations and acylations yields a wide variety of mixed glycosides, bearing substructures corresponding to the different biochemical classes. For example, in the tassel, many phenylpropanoid and flavonoid-bearing glycosides also contain auxin-derived moieties. The characterized compounds and mass differences are an important step forward in metabolic pathway discovery and systems biology research.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Evaluation of Data Catalog Software for Hanford Site Environmental Datasets

Environmental information and data underpin achievement of the U.S. Department of Energy (DOE) Office of Environmental Management (EM) mission at the Hanford Site. The Hanford Environmental Data Management (HEDM) Program is the DOE Richland Operations Office (RL) approach to develop and implement a formal program for managing environmental data and the associated records, materials, and systems at the Hanford Site. The current project, contract, organization, and contractor-specific efforts at managing environmental data sets are insufficient to provide orderly, long-term, site-wide access. A vital element to be created within the HEDM program plan is a catalog of data sources, called the Hanford Environmental Information and Data Index (HEIDI), that will enable long-term access and retrievability for the multiple independent sources of data that might otherwise be difficult to discover. This report compares leading open source and commercial data catalog platforms using criteria to assess the functionality needed to develop the HEIDI catalog of Hanford data sources that connects and exchanges data with established Hanford Local Area Network (HLAN) enterprise information technology systems. Proprietary platforms evaluated included ArcGIS Enterprise Sites, Junar, OpenDataSoft, and Socrata, and non-proprietary platforms included Energy Data eXchange (EDX), Comprehensive Knowledge Archive Network (CKAN), and DKAN (a Drupal-based open data portal based on CKAN). Capabilities supporting data discoverability, retrieval, and archival, as well as metadata standard requirements and integration into the HLAN were rated as either failing to meet requirements (F), meeting requirements (M), or exceeding requirements by delivering additional desired features (E). The lowest rating for any capability area was assigned as the overall rating for the platform. These findings enable DOE-RL and the contractors implementing the HEDM plan to focus on candidate tools likely to meet the requirements for implementing HEIDI. All of the platforms receiving an overall rating of ‘F’ were unable to be deployed on Hanford infrastructure or within dedicated cloud resources. A propriety software-as-a-service (SaaS) model of delivering a data catalog (e.g., found in software such as Junar and OpenDataSoft) favors consistency across customers at the expense of customization and configurable roles that are needed for Hanford work. Hosting data on a shared commercial platform places limits on dataset size (maximum of 240 Mb for OpenDataSoft), a significant limitation for HEIDI implementation. EDX, a government data catalog based on CKAN, received the ‘F’ rating due to an inability to incorporate authentication from HLAN into the system. Among platforms rated ‘M’ or ‘E’, only the Socrata platform had a SaaS delivery model. In contrast to other SaaS platforms, Socrata provided custom roles and gateways that allow local datasets to be incorporated into an online catalog. Socrata also complies with the Federal Risk and Authorization Management Program, a significant benefit for cloud-based management of Hanford data. The other platforms rated ‘M’ or ‘E’, ArcGIS Enterprise Sites, CKAN, and DKAN, provide fully self-hosted options, allowing for greater control and flexibility with the HEIDI catalog. These widely used tools have supportive communities of practice, extensive customization options, and demonstrated deployments that provide evidence that they can meet requirements, often deliver additional desired features, and work well with federal government systems. Completely customized alternatives built on a collection of applications were not evaluated because achieving similar performance to CKAN or DKAN requires substantial resources, especially in the absence of the active communities that have grown to support these tools. ArcGIS Enterprise Sites, Socrata, CKAN, and DKAN were evaluated as strong candidates for successful implementation with HEIDI.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Smart Semi-Supervised Accumulation of Large Repositories for Industrial Control Systems Device Information

Industrial Control Systems device manufacturers frequently add new features to improve their product performance. Oftentimes, these changes are mainly vendor-driven initiatives, and customers may not be aware of the full impact of these new capabilities on their cybersecurity posture. In the energy sector, this can lead to considerable dissonance between vendor-provided cybersecurity claims and a customer’s responsibility for Operation Technology cybersecurity compliance. Thus, the resulting dynamic verification burden is shifted towards the customer and may pose a significant cybersecurity risk to the energy sector landscape. We found that there is very limited research into cybersecurity auditing for Operational Technology. However, a solution is needed for vetting the vendor-supplied feature claims and their adherence to cybersecurity requirements and standards. We are presently engaged in an effort to develop such a system. This paper demonstrates one vital aspect of this effort in proposing an end-to-end framework to accumulate a large repository of ICS device information for this vetting system, curate the dataset, and conduct extensive processing. This framework is designed to use web scraping, data analytics and Natural Language Processing (NLP) techniques to identify vendor websites, automate the collection of website-accessible documents and automatically derive metadata from them for identification of product documents relevant to the repository. We have found that this automated approach to vendor identification, document extraction into a product repository, and NLP pre-processing is unique and has not been previously presented in the literature. The preliminary work shows that this is feasible and can produce reliable results with minimum supervision. Future work will be built upon this foundation in order to achieve semi-supervised vetting of device technical information – a vital capability for ensuring that vendor-claimed device cybersecurity capabilities match industry requirements.

Ameri, Kimia↗
DOE OSTI2024

Vegetation Warming Experiment: Plant Physiology, Utqiagvik (Barrow), Alaska, 2021

Leaf gas exchange measurements on Carex aquatilis Wahlenb. following a single season warming treatment. Data were collected in 2021 from 5 treatment warming chambers and paired control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska. Data include CO2 response (ACi) curves, light response (AQ) curves, and dark-adapted respiration (Rdark) logged data measured at controlled leaf temperatures from 5–25 °C. The data package includes 4 data files in .csv format, 10 metadata files and the complete instrument output for all measurements. These data were collected as part of an experiment using Zero Power Warming (ZPW) chambers that delivered a single season warming treatment of ~4 °C above ambient air temperature. Four different plant species were targeted over four experimental years from 2017–2021. See related data packages for processed gas exchange data, leaf trait data (leaf mass per area, leaf nitrogen concentration), ambient and chamber environmental conditions, phenocamera images, thaw depth and GPS locations of chambers. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Rapid Characterization and Statistical Analysis of High-Volume Field-Harvested Photovoltaic Connectors

Photovoltaic (PV) installations heavily depend on connectors for efficient module and string interconnections without requiring skilled labor. Yet this seemingly innocuous component of PV systems is a leading cause of module failures, multiple high-profile fires, and lawsuits in the PV industry. This work aims to answer critical questions regarding why connectors fail and the contributing factors to their failure. The study involves collecting and analyzing more than 17,000 field-harvested connectors from various solar installations across the United States. The vast dataset, which includes connector metadata, visual inspections, and resistance measurements, provides unprecedented insight into the state of health of PV connectors across the US, including the geographic locations, connector types, and installation practices most prone to failures. The work presented here describes a novel rapid characterization method for processing large numbers of connectors and is supported by parallel forensic analysis to discern the root causes of failures as well as a levelized cost of lifetime model to determine the economic ramifications of connector failure. Ultimately, the findings may inform PV developers about the best practices to extend connector longevity and lead to more resilient and reliable PV systems.

connectors↗
DOE OSTI2021

RITE Gen5 KHPS Performance - Period B

Includes Kinetic Hydropower System (KHPS) Turbine performance data from the RITE Gen5 KHPS turbine at the 5 meter size. This data was collected over 16 days in May 2021 during Period B operation. Along with performance data, the data includes setup data, project metadata, and characteristic data for RITE Gen5 KHPS (5m) turbine.

16 TIDAL AND WAVE POWER↗
DOE OSTI2026

Data and scripts associated with a manuscript analyzing ELM-FATES parameter sensitivity under pre-fire and postfire scenarios using machine learning

NOTE: The manuscript associated with this data package is currently in review. The data may be revised based on reviewer feedback. Upon manuscript acceptance, this data package will be updated with the final dataset and additional metadata. This data package is associated with the manuscript “Fire Severity-Dependent Shifts in Vegetation Parameter Sensitivity: A Pre- and Post-Fire Analysis Using ELM-FATES and Explainable AI” submitted to Journal of Advances in Modeling Earth Systems (Zahura et al. 2026). The study examines vegetation physiological parameters controlling pre-fire and post-fire vegetation dynamics. To support this analysis, 73 vegetation parameters in Functionally Assembled Terrestrial Ecosystem Simulator (FATES) (Fisher et al., 2018) , which is coupled with E3SM (Energy Exascale Earth System Model) land model (ELM, ELM-FATES), were perturbed using a Sobol sequence to generate 1,024 ensemble members for two plant functional types: needleleaf evergreen extratropical trees (NEET) and C3 grass. Simulations were conducted for the pre-fire period (2016) and post-fire period (2018–2023). Burn severity was represented by modifying the Nesterov index in FATES to 75,000, 150,000, and 300,000 for low, moderate, and high severity, respectively. A no-fire scenario was also included. Simulations were performed for 16 grid cells in the American River Watershed across different burn severities and plant functional types. XGBoost (eXtreme Gradient Boosting) models were trained using the parameter ensembles and ELM-FATES-simulated outputs, including leaf area index (LAI), gross primary productivity (GPP), aboveground biomass, vegetation evaporation, transpiration, and soil evaporation. Models were trained separately for each year and burn severity, followed by SHAP (SHapley Additive exPlanations) analysis to identify changes in dominant parameters after fire disturbance. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. The data package contains the ELM-FATES simulation data. The scripts and data related to the analysis will be added later. The inputs and outputs from ELM-FATES are inside the “FATES” folder. “FATES_domain_surface” contains the domain and surface netcdfs that were used to run ELM-FATES in the study area. “FATES_parameters” contains the 1024 ensembles that were generated using Sobol sequence. “FATES_outputs” folder contains ELM-FATES simulated variables. All files are .csv and .nc (NetCDF).

Aboveground biomass↗
DOE OSTI2025

RhizoGrid Indexed Sorghum Rhizosphere Multi-Omics

PerCon SFA project data dentification of spatially resolved biomarkers of drought in Sorghum bicolor rhizosphere molecular-microbe interactions using a novel root cartography "RhizoGrid" system for sampling plants under drought and control conditions across 10 equally sized root zone environments (4 quadrants each). Each quadrant was sampled and processed for 16S amplicon, metabolomics, and X-ray computed tomography (XCT). Data download includes experimental metadata and results files for 16S rRNA sequence analysis of microbial community assembly (processed data files), liquid chromatography mass spectrometry (LC-MS) metabolomics analysis of microbial community root exudates (processed data files), X-ray computed tomography (XCT) spatial gradient analysis (raw and processed data files) of microbial community composition, and related computational modeling outputs.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Leaf Nitrogen and Carbon Content, and Leaf Mass Per Area, Kougarok Road, Seward Peninsula, Alaska, 2018

Nitrogen and carbon content, leaf mass per area (LMA) and leaf water content (LWC) of leaves sampled from locations on the Kougarok mile marker 64 NGEE Arctic site, Seward Peninsula, Alaska. Samples were collected in July 2018 from Alnus viridis, Betula nana, Betula glandulosa, Arctostaphylos alpina and Salix pulchra. This data package includes leaf sample information and trait data (*.csv). Metadata files include data descriptions (_dd.csv) for tabular data and a key to species symbols used in data files. All included files are listed and described in NGA207_flmd.csv. See data package NGA208 "Full spectrum 350-2500 nm canopy spectral reflectance, Seward Peninsula, Alaska, 2018" for linked canopy spectral reflectance data, dGPS locations and sample photographs. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Plant Physiology, Utqiagvik (Barrow), Alaska, 2018

Leaf gas exchange measurements on Arctagrostis latifolia (R. Br.) Griseb. following a single season warming treatment. Data were collected in 2018 from 5 treatment warming chambers and paired control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska. Data include CO2 response (ACi) curves, light response (AQ) curves, and dark-adapted respiration (Rdark) logged data measured at controlled leaf temperatures from 5–25 °C. The data package includes 4 data files in .csv format, 10 metadata files and the complete instrument output for all measurements. These data were collected as part of an experiment using Zero Power Warming (ZPW) chambers that delivered a single season warming treatment of ~4 °C above ambient air temperature. Four different plant species were targeted over four experimental years from 2017–2021. See related data packages for processed gas exchange data, leaf trait data (leaf mass per area, leaf nitrogen concentration), ambient and chamber environmental conditions, phenocamera images, thaw depth and GPS locations of chambers. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Vegetation Warming Experiment: Plant Physiology, Utqiagvik (Barrow), Alaska, 2019

Leaf gas exchange measurements on Eriophorum angustifolium Honck. following a single season warming treatment. Data were collected in 2019 from 5 treatment warming chambers and paired control plots located on the Barrow Environmental Observatory (BEO), Utqiagvik, Alaska. Data include CO2 response (ACi) curves, light response (AQ) curves, and dark-adapted respiration (Rdark) logged data measured at controlled leaf temperatures from 5–25 °C. The data package includes 4 data files in .csv format,, and include 4 data files, 10 metadata files and the complete instrument output for all measurements. These data were collected as part of an experiment using Zero Power Warming (ZPW) chambers that delivered a single season warming treatment of ~4 °C above ambient air temperature. Four different plant species were targeted over four experimental years from 2017–2021. See related data packages for processed gas exchange data, leaf trait data (leaf mass per area, leaf nitrogen concentration), ambient and chamber environmental conditions, phenocamera images, thaw depth and GPS locations of chambers. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Continuous snow depth, ground interface temperature and shallow soil temperature measurements from 2021-10-1 to 2022-6-14, Seward Peninsula, Alaska

The dataset contains co-located snow depth, ground interface temperature, and shallow soil temperature measured at 98 discrete locations in a watershed located along the Nome-Teller road at mile marker 27 (referred to as T27), and at 53 discrete locations on a hillslope located along the Kougarok road at mile marker 64 (referred to as K64), in Seward Peninsula, Alaska. The dataset aims to understand the local heterogeneity of snow depth, snow temperature, and soil temperature dynamics and their interactions in a discontinuous permafrost region. The dataset is also valuable to train and evaluate machine learning and physical models to predict snow depth or the impact of snow depth on ground surface temperature. At each location, temperatures above and below the ground surface were measured by a pair of vertically deployed distributed temperature profiling probes designed based on Dafflon et al (2022). The probes have high precision temperature sensors spaced at 5 or 10 cm. The mean daily snow depth was estimated by identifying the pair of consecutive sensors with maximum drop of daily temperature high-frequency fluctuations. The ground interface temperature was measured by the sensor located 1-5 cm above the ground surface at 15-minute intervals. The shallow soil temperature was measured by the sensor located 1-5 cm below the ground surface at 15-minute intervals. The dataset includes a description of the probe locations in the "Probe_locations_*.csv" file and the 3 data files (Snow_depths_*.csv, Ground_interface_temperatures_*.csv, Shallow_soil_temperatures_*.csv). * is either T27 or K64, which are the two study sites. In each data file, each column corresponds to a measurement location. Metadata files include data descriptions (_dd.csv) for tabular data. All included files are listed and described in xxxx_flmd.csv. A more detailed description of data processing, along with an updated dataset incorporating multiple seasons and improved snow depth estimation is available at https://doi.org/10.15485/2480365 (Wang et al., 2025a, Wang et al., 2025b).The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Leveraging Pre-Built Catalogs and Object-Level Scheduling to Eliminate I/O Bottlenecks in HPC Environments

Modern High-Performance Computing (HPC) environments face mounting challenges due to the shift from large to small file datasets, along with an increasing number of users and parallelized applications. As HPC systems rely on Parallel File Systems (PFS), such as Lustre for data processing, performance bottlenecks stemming from Object Storage Target (OST) contention have become a significant concern. Existing solutions, such as LADS with its object-level scheduling approach, fall short in large-scale HPC environments due to their inability to effectively address metadata I/O bottlenecks and the growing number of I/O processes. This study highlights the pressing need for a comprehensive solution that tackles both OST contention and metadata I/O challenges in diverse HPC workloads. To address these challenges, we propose SwiftLoad, an object-level I/O scheduling framework that leverages a metadata catalog to enhance the performance and efficiency of parallel HPC utilities. The adoption of the metadata catalog mitigates the metadata I/O bottlenecks that commonly occur in HPC utilities, a challenge that is particularly pronounced in object-level I/O scheduling. SwiftLoad addresses OST contention and the uneven distribution of I/O processes across different OSTs through mathematical modeling and incorporates a Loader Configuration Module to regulate the number of I/O processes. Evaluated with two representative utilities—data deduplication profiling and data augmentation—SwiftLoad achieved performance improvements of up to 5.63x and 11.0x, respectively, on a production supercomputer.

HPC↗
DOE OSTI2021

Adaptive Spatially Aware I/O for Multiresolution Particle Data Layouts

Large-scale simulations on nonuniform particle distributions that evolve over time are widely used in cosmology, molecular dynamics, and engineering. Such data are often saved in an unstructured format that neither preserves spatial locality nor provides metadata for accelerating spatial or attribute subset queries, leading to poor performance of visualization tasks. Furthermore, the parallel I/O strategy used typically writes a file per process or a single shared file, neither of which is portable or scalable across different HPC systems. We present a portable technique for scalable, spatially aware adaptive aggregation that preserves spatial locality in the output. We evaluate our approach on two supercomputers, Stampede2 and Summit, and demonstrate that it outperforms prior approaches at scale, achieving up to 2.5× faster writes and reads for nonuniform distributions. Furthermore, the layout written by our method is directly suitable for visual analytics, supporting low-latency reads and attribute-based filtering with little overhead.

Usher, Will↗
DOE OSTI2020

ESS-DIVE reporting format for leaf-level gas exchange data and metadata

Here we present documentation of the ESS-DIVE reporting format for leaf-level gas exchange data and metadata. This reporting format provides guidance to data contributors on how to store data to maximize their discoverability, facilitate their efficient reuse, and add value to individual datasets. For data users, the reporting format will better allow data repositories to optimize data search and extraction, and more readily integrate similar data into harmonized synthesis products. The reporting format specifies data table variable naming and unit conventions, as well as metadata characterizing experimental conditions and protocols. For common data types that were the focus of this initial version of the reporting format, i.e., survey measurements, dark respiration, carbon dioxide and light response curves, and parameters derived from those measurements, we took a further step of defining required additional data that would maximize the potential reuse of those data types. To aid data contributors and the development of data ingest tools by data repositories we provided a translation table comparing the outputs of common gas exchange instruments. The reporting format presented here is intended to form a foundation for future development that will incorporate additional data types and variables as gas exchange systems and measurement approaches advance in the future. The reporting format documentation is maintained and updated on the ESS-DIVE Community Space GitHub. This data package is the first published version of this reporting format, and comprises a zip file of the complete content of https://github.com/ess-dive-community/essdive-leaf-gas-exchange v1.0. The zip contains the reporting format description, guidelines, variable tables and instructions in GitHub markdown language (*.md) and 2 metadata templates as spreadsheets with drop down options (*.xlsx files, also function in GoogleSheets).

54 ENVIRONMENTAL SCIENCES↗