Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NetCDF”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

CROCUS Tipping Bucket Rain Gauge Data from Argonne Deployable Mast Deployed at Argonne National Laboratory During Urban Flooding Campaign

The Tipping Bucket Rain Gauge (TBRG) dataset contains data from a non-heated Met One 12-inch tipping bucket rain gauge that was mounted on the Argonne Deployable Mast (ADM). The ADM is a rapid deployable meteorological trailer that can be outfitted with instrumentation to measure urban heat island effects, urban flooding or urban flux measurements. During the urban flooding field campaign, the ADM was outfitted with multiple precipitation measurement systems, including the TBRG. This dataset contains one minute measurements for precipitation accumulation during the ADM's deployment at the Argonne Testbed for Multiscale Observational Science (ATMOS) site. These data are helpful for identifying periods of precipitation, leading to potential flooding. TBRGs can be used to validate optical rain gauge data and disdrometer data collected during the CROCUS urban flooding campaign. Data were collected at ATMOS, a 20-acre prairie site at Argonne National Laboratory in Lemont, Illinois. The data is presented as daily NetCDF (.nc) files, each containing approximately 24 hours of observations. Files follow the naming convention of: the project (CROCUS), location (ADM-atmos), instrument name (tbrg), data level (raw, a1), and date (year, month, day). The NetCDF format can be accessed using common scientific software such as Python using xarray, netCDF4 or act-doe.

1-min Precipitation Accumulation↗
DOE OSTI2026

CROCUS Tipping Bucket Rain Gauge Data from Argonne Deployable Mast Deployed at NEIU Carruthers Center for Inner City Studies (CCICS)

The Tipping Bucket Rain Gauge (TBRG) dataset contains data from a non-heated Met One 12-inch tipping bucket rain gauge that was mounted on the Argonne Deployable Mast (ADM). The ADM is a rapid deployable meteorological trailer that can be outfitted with instrumentation to measure urban heat island effects, urban flooding or urban flux measurements. During the urban flooding field campaign, the ADM was outfitted with multiple precipitation measurement systems, including the TBRG. This dataset contains one minute measurements for precipitation accumulation during the ADM's deployment at the Northeastern Illinois University (NEIU) Carruthers Center for Inner City Studies (CCICS) campus. These data are helpful for identifying periods of precipitation, leading to potential flooding. TBRGs can be used to validate optical rain gauge data and disdrometer data collected during the CROCUS urban flooding campaign. Data were collected at the CCICS building parking lot, located in the south side of Chicago, IL. The data is presented as daily NetCDF (.nc) files, each containing approximately 24 hours of observations. Files follow the naming convention of: the project (CROCUS), location (ADM-ccics), instrument name (tbrg), data level (raw, a1), and date (year, month, day). The NetCDF format can be accessed using common scientific software such as Python using xarray, netCDF4 or act-doe.

1-minute Precipitation Accumulation↗
DOE OSTI2026

Data and scripts associated with a manuscript analyzing ELM-FATES parameter sensitivity under pre-fire and postfire scenarios using machine learning

NOTE: The manuscript associated with this data package is currently in review. The data may be revised based on reviewer feedback. Upon manuscript acceptance, this data package will be updated with the final dataset and additional metadata. This data package is associated with the manuscript “Fire Severity-Dependent Shifts in Vegetation Parameter Sensitivity: A Pre- and Post-Fire Analysis Using ELM-FATES and Explainable AI” submitted to Journal of Advances in Modeling Earth Systems (Zahura et al. 2026). The study examines vegetation physiological parameters controlling pre-fire and post-fire vegetation dynamics. To support this analysis, 73 vegetation parameters in Functionally Assembled Terrestrial Ecosystem Simulator (FATES) (Fisher et al., 2018) , which is coupled with E3SM (Energy Exascale Earth System Model) land model (ELM, ELM-FATES), were perturbed using a Sobol sequence to generate 1,024 ensemble members for two plant functional types: needleleaf evergreen extratropical trees (NEET) and C3 grass. Simulations were conducted for the pre-fire period (2016) and post-fire period (2018–2023). Burn severity was represented by modifying the Nesterov index in FATES to 75,000, 150,000, and 300,000 for low, moderate, and high severity, respectively. A no-fire scenario was also included. Simulations were performed for 16 grid cells in the American River Watershed across different burn severities and plant functional types. XGBoost (eXtreme Gradient Boosting) models were trained using the parameter ensembles and ELM-FATES-simulated outputs, including leaf area index (LAI), gross primary productivity (GPP), aboveground biomass, vegetation evaporation, transpiration, and soil evaporation. Models were trained separately for each year and burn severity, followed by SHAP (SHapley Additive exPlanations) analysis to identify changes in dominant parameters after fire disturbance. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. The data package contains the ELM-FATES simulation data. The scripts and data related to the analysis will be added later. The inputs and outputs from ELM-FATES are inside the “FATES” folder. “FATES_domain_surface” contains the domain and surface netcdfs that were used to run ELM-FATES in the study area. “FATES_parameters” contains the 1024 ensembles that were generated using Sobol sequence. “FATES_outputs” folder contains ELM-FATES simulated variables. All files are .csv and .nc (NetCDF).

Aboveground biomass↗
DOE OSTI2026

Data and Scripts Associated with "Modeling Ecohydrological Responses of Vegetation to Urban Microclimates Using the E3SM Land Model"

This dataset supports the study of vegetation ecohydrological responses to urban microclimates using the land component of the Energy Exascale Earth System Model (ELM) at four urban sites in Knoxville, Tennessee, USA. It includes the model inputs, simulation outputs, and associated scripts for running ELM simulations and analyzing the resulting data. The Model_Inputs folder includes static surface data, satellite-derived phenology (i.e., leaf area index), and atmospheric forcing data used to drive ELM simulations. Detailed descriptions of these datasets are provided in Section 2.3.2 of the associated manuscript. The Model_Outputs folder contains simulation results for the baseline, treatment, and ensemble experiments. Outputs from the baseline and treatment simulations are provided as raw ELM NetCDF files. Because the raw outputs from the 4,000-member ensemble are prohibitively large, the ensemble results are provided as summarized CSV files, which also serve as the source data for Figure 5 of the associated manuscript. The Scripts folder contains three components: E3SM, the core codebase of the Energy Exascale Earth System Model (E3SM); elm-olmt, the Offline Land Model Testbed (OLMT) used to perform the simulations; and knoxville_elm, which contains the analysis scripts used to process model outputs and generate the figures and results presented in the associated manuscript. Additional information is provided in Scripts_readme.txt within the Scripts directory.

Lu, Xiaoman [ORNL] (ORCID:0000000306698780)↗
DOE OSTI2025

Files and scripts to support manuscript Needham et al. Canopy Gradients of Respiration

This dataset includes the parameter files, relevant output files, and scripts to perform analysis with Jupyter notebooks that support the manuscript Needham et al 2025 “Canopy Gradients of Respiration Drive Plant Carbon Budgets and Leaf Area Index.” We add functionality to the Functionally Assembled Terrestrial Ecosystem Simulator (FATES) to allow flexible vertical gradients of leaf maintenance respiration (Rdark) and maximum carboxylation rate (Vcmax) through the canopy. We test the sensitivity of FATES to canopy gradients in Rdark, both in global simulations to assess broad scale impacts on leaf area index (LAI) and vegetation carbon, and in single site simulations where we assess impacts on plant functional type (PFT) competitive dynamics. Parameter files are netcdf files that can be converted to human readable .cdl files using NCO tools. Analysis scripts are Jupyter notebook files. These can be opened and run using the open source Jupyter notebook software. Model outputs are netcdf files.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Meteorological and Hydrological Drought Analysis Datasets for Livneh, ClimRR, and TGW

The current dataset contains data upload links to the following SPI & SRI drought analysis data over CONUS: Climate Forcing and Simulation Scenario: Livneh (https://www.nature.com/articles/sdata201542): Historical ClimRR (https://climrr.anl.gov/climrrdata): Historical, Mid-Century, End-Century TGW (https://tgw-data.msdlive.org/): Historical, Mid-Century, End-Century Analysis Period: Vary by Climate Forcing Drought Type: Meteorological (Precipitation) and Hydrological (Runoff) Drought (3-Month SRI) Output Format: NetCDF and CSV Spatial Resolution: 1/16th Degree for NetCDF and HUC4 for CSV Temporal Resolution: Monthly

Tidwell, Vincent C [Pacific Northwest National Lab↗
DOE OSTI2020

GCAM-Demeter LU dataset

The dataset includes the projected global gridded land cover (excluding the Antarctic) for the period of 2015-2100 at 0.05-degree resolution and 5-year time step under fifteen SSP-RCP scenarios driven by five GCMs (i.e., gfdl, hadgem, ipsl, miroc, and noresm), using the Global Change Analysis Model (GCAM) and a geospatial downscaling model (Demeter). The data has been stored in self-describing NetCDF format. The dataset also includes the mean and standard deviation of the results driven by the five GCMs. More specifically, the data in each year includes grid-explicit fraction (in percent) of each of the 32 plant functional types that are widely used in current Earth system models. The NetCDF files are named as “GCAM_Demeter_LU_SSP_RCP_Model_Year.nc”, where “SSP” and “RCP” denote the SSP and RCP scenarios, including 'ssp1_rcp26', 'ssp1_rcp45', 'ssp1_rcp60', 'ssp2_rcp26', 'ssp2_rcp45', 'ssp2_rcp60', 'ssp3_rcp45', 'ssp3_rcp60', 'ssp4_rcp26', 'ssp4_rcp45', 'ssp4_rcp60', 'ssp5_rcp26', 'ssp5_rcp45', 'ssp5_rcp60', and 'ssp5_rcp85'. “Year” denotes the year of the land use data, and “GCM” denotes the source driving forcing data from five global climate models (gfdl, hadgem, ipsl, miroc, and noresm), or the mean (“modelmean”) and the standard deviation (“modelstd”) of the results from the five GCMs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

GCAM-Demeter-LU

This GCAM-Demeter-LU is made available under the Open Data Commons Attribution License:http://opendatacommons.org/licenses/by/1.0/. The dataset includes the projected global gridded land cover (excluding the Antarctic) for the period of 2015-2100 at 0.05-degree resolution and 5-year time step under fifteen SSP-RCP scenarios driven by five GCMs (i.e., gfdl, hadgem, ipsl, miroc, and noresm), using the Global Change Analysis Model (GCAM) and a geospatial downscaling model (Demeter). The data has been stored in self-describing NetCDF format. The dataset also includes the mean and standard deviation of the results driven by the five GCMs. More specifically, the data in each year includes grid-explicit fraction (in percent) of each of the 32 plant functional types that are widely used in current Earth system models. The NetCDF files are named as “GCAM_Demeter_LU_SSP_RCP_Model_Year.nc”, where “SSP” and “RCP” denote the SSP and RCP scenarios, including 'ssp1_rcp26', 'ssp1_rcp45', 'ssp1_rcp60', 'ssp2_rcp26', 'ssp2_rcp45', 'ssp2_rcp60', 'ssp3_rcp45', 'ssp3_rcp60', 'ssp4_rcp26', 'ssp4_rcp45', 'ssp4_rcp60', 'ssp5_rcp26', 'ssp5_rcp45', 'ssp5_rcp60', and 'ssp5_rcp85'. “Year” denotes the year of the land use data, and “GCM” denotes the source driving forcing data from five global climate models (gfdl, hadgem, ipsl, miroc, and noresm), or the mean (“modelmean”) and the standard deviation (“modelstd”) of the results from the five GCMs.See https://github.com/JGCRI/chen_et_al_2020a for details on how to reproduce this data. Rights and permissions Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver http://creativecommons.org/publicdomain/zero/1.0/ applies to the metadata files associated with this article

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2015

Turbulence Properties at the ENA site

The data set includes the turbulence properties as derived from the data collected by the Doppler lidar (DL) at the ARM Eastern North Atlantic (ENA) site. Among the reported turbulence properties included are profiles of variance of vertical air motion, skewness of vertical air motion, updraft strength, downdraft strength, updraft fraction, and downdraft fraction. Due to the persistent presence of drizzle at the location, the DL echoes were classified as those corresponding to aerosols and drizzle using the backscatter cross-section threshold. This daily backscatter cross-section threshold is also reported. The data set begins on January 1, 2015 and ends January 1, 2021. The gaps in the data set correspond to the periods when the instrument was not operational. The header of the NetCDF file is as below. netcdf dl.turb.stats { dimensions: time = 52609; height = 241; threshold_time = 1828; variables: double time(time); time:Long_Name = "seconds since 1970-1-1"; time:units = "seconds"; double height(height); height:Long_Name = "height of the center of the range gate above ground"; height:units = "meters"; double echo_fraction(time, height); echo_fraction:Long_Name = "Fraction of hour with valid Doppler lidar echoes" ; echo_fraction:units = "N/A"; double wvar(time, height); wvar:Long_Name = "Variance of vertical air motion"; wvar:units = "m^2/s^2"; double wskew(time, height); wskew:Long_Name = "Skewness of vertical air motion"; wskew:units = "N/A"; double updraft_velocity(time, height); updraft_velocity:Long_Name = "Hourly averaged updraft velocity"; updraft_velocity:units = "m/s"; double downdraft_velocity(time, height); downdraft_velocity:Long_Name = "Hourly averaged downdraft velocity"; downdraft_velocity:units = "m/s"; double updraft_fraction(time, height); updraft_fraction:Long_Name = "Hourly updraft fraction"; updraft_fraction:units = "N/A"; double downdraft_fraction(time, height); downdraft_fraction:Long_Name = "Hourly downdraft fraction"; downdraft_fraction:units = "N/A"; double threshold_time(threshold_time); threshold_time:Long_Name = "Seconds since 1970-1-1 for the day"; threshold_time:units = "seconds"; double bscat_threshold(threshold_time); bscat_threshold:Long_Name = "Backscatter threshold used for discerning lidar returns from hydrometeors and aerosols."; bscat_threshold:units = "1/(m*Sr)"; }

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Stride Search v. 1.0

Stride Search provides a flexible tool for detecting storms or other extreme climate events in high-resolution climate data sets saved on uniform latitude-longitude grids in standard NetCDF format. Users provide the software a quantitative description of a meteorological event they are interested in; the software searches a data set for locations in space and time that meet the user’s description. In its first stage, Stride Search performs a spatial search of the data set at each timestep by dividing a search domain into circular sectors of constant geodesic radius. Data from a netCDF file is read into memory for each circular search sector. If the data meet or exceed a set of storm identification criteria (defined by the user), a storm is recorded to a linked list. Finally, the linked list is examined and duplicate detections of the same storm are removed and the results are written to an output file. The first stage’s output file is read by a second program that builds storm. Additional identification criteria may be applied at this stage to further classify storms. Storm tracks are the software’s ultimate output and routines are provided for formatting that output for various external software libraries for plotting and tabulating data.

Bosler, Peter↗
DOE OSTI2019

Accelerated Climate Model for Energy (Final Report)

The E3SM (formerly ACME) project utilizes the netCDF Operator (NCO) toolkit to generate, regrid, and split climate datasets for pre- and post-processing analysis. NCO tools are themselves embedded in the E3SM-unified, E3SM-diags, A-Prime, MPAS-Analysis, Processflow, and LIVVkit diagnostic suites, and so improve and leverage the analysis and diagnostics capabilities of the E3SM Science and Infrastructure teams. In this project, the UCI team extended and customized NCO features used standalone and by the diagnostic suites to accelerate and ease analysis and distribution of E3SM experiments. With supplementary funding, the UCI team also contributed physical improvements to the treatment of snow in the E3SM land, land-ice, and sea-ice model components.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Parallel membership queries on very large scientific data sets using bitmap indexes

© 2019 John Wiley & Sons, Ltd. Many scientific applications produce very large amounts of data as advances in hardware fuel computing and experimental facilities. Managing and analyzing massive quantities of scientific data is challenging as data are often stored in specific formatted files, such as HDF5 and NetCDF, which do not offer appropriate search capabilities. In this research, we investigated a special class of search capability, called membership query, to identify whether queried elements of a set are members of an attribute. Attributes that naturally have classification values appear frequently in scientific domains such as category and object type as well as in daily life such as zip code and occupation. Because classification attribute values are discrete and require random data access, performing a membership query on a large scientific data set creates challenges. We applied bitmap indexing and parallelization to membership queries to overcome these challenges. Bitmap indexing provides high performance not only for low cardinality attributes but also for high cardinality attributes, such as floating-point variables, electric charge, or momentum in a particle physics data set, due to compression algorithms such as Word-Aligned Hybrid. We conducted experiments, in a highly parallelized environment, on data obtained from a particle accelerator model and a synthetic data set.

Yildiz, Beytullah↗
DOE OSTI2022

Domain-Specific Type-Safe APIs for Hierarchical Scientific Data with Modern C++

General-purpose library application programming interfaces (APIs) for self-describing hierarchical scientific data storage, such as the HDF5 and NetCDF libraries, are traditionally of runtime nature. Runtime errors for entry existence and data types are typically caught later in the development process of higher-level application-specific APIs. In this paper, we propose exploiting modern C++ metaprogramming features to add compile-time type-safety to improve the interaction with a well-defined metadata-rich scientific schema in domain-specific hierarchical datasets. We tackle two aspects of common use: (i) direct data access, (ii) flexible “in-memory” index models for efficient search and data processing. The proposed APIs use C++17’s template type auto deduction features, C++11’s enum class for type-safety and C-style preprocessor macros for generative templated code. We showcase the pros and cons of our initial work on the standard NeXus schema used for annotating and storing experimental neutron scattering data at several facilities around the world on top of HDF5. Extendable compile-time type-safe APIs are a desirable feature that could be indexed by any modern integrated development environment (IDE). Hence, such APIs can help ease the learning curve for domain scientists using a less error-prone software interaction to enhance the findability of their data without resorting to a domain-specific language (DSL).

Godoy, William↗
DOE OSTI2024

Enhancing Discoverability and Management of Atmospheric Data at Scale: Solutions from the ARM Data Center

The Atmospheric Radiation Measurement (ARM) is a multi-laboratory and multi-institutional U.S. Department of Energy (DOE) Office of Science National User Facility. The ARM Data Center (ADC), located at Oak Ridge National Laboratory, collects, archives, and shares vast atmospheric data crucial for climate research. The ADC manages over 7 PB of data from 460 instruments worldwide, processing it into more than 11,000 diverse data products using the Network Common Data Form (NetCDF) for machine-independent accessibility. The primary challenge addressed in this paper is the efficient management and distribution of vast and diverse datasets essential for the climate research community, enhancing accessibility through advanced tools like Data Discovery. The ADC has developed advanced infrastructure and software architecture to handle the continuous influx of heterogeneous data to enhance data discoverability, resulting in increased scientific collaboration. In 2023, users from over 34 countries downloaded and utilized ARM data, resulting in 1,455 publications. The ADC’s efforts have significantly improved the discoverability and usability of atmospheric data, fostering extensive scientific research and collaboration. This paper details the solutions implemented by the ADC team for efficient data discovery and distribution, and it demonstrates ARM’s capability of staging processed data for scientific analysis.

Shah, Chirag [ORNL] (ORCID:0000000203145737)↗
DOE OSTI2024

DaYu: Optimizing Distributed Scientific Workflows by Decoding Dataflow Semantics and Dynamics

The combination of ever-growing scientific datasets and distributed workflow complexity creates I/O performance bottlenecks due to data volume, velocity, and variety. Although the increasing use of descriptive data formats (e.g., HDF5, netCDF) helps organize these datasets, it also creates obscure bottlenecks due to the need to translate high level operations into file addresses and then into low-level I/O operations. To address this challenge, we introduce DaYu, a method and toolset for analyzing (a) semantic relationships between logical datasets and file addresses, (b) how dataset operations translate into I/O, and (c) the combination across entire workflows. DaYu's analysis and visualization enables identification of critical bottlenecks and reasoning about remediation. We describe our methodology and propose optimization guidelines. Evaluation on scientific workflows demonstrates up to 3.7x performance improvements in I/O time for obscure bottlenecks. The time and storage overhead for DaYu's time-ordered data is typically under 0.2% of runtime and 0.25% of data volume, respectively.

Tang, Meng↗
DOE OSTI2020

Quick n Dirty

QnD is a pure Python package for working with binary files. It has built-in support for HDF5, NetCDF3, and PDB formats. It provides a Pythonic, dict-like interface, and allows easy movement among time-like "records". Both reading and writing are enabled. The HDF and NetCDF capabilities rely on external libraries, while the PDB support is part of QnD itself and entirely in Python. QnD works under Python 2 and 3. For interactive use, much less typing is involved than for some other packages.

MUNRO, DavidH.↗
DOE OSTI2026

Adapt: A Weather Radar Data Analysis and Nowcasting Platform for Informed Adaptive Scanning

SF-26-021 Adapt is a data processing platform for real-time data analysis, short term prediction of targets convective cells and tracking for archived data. It provides tools for downloading, processing, segmenting, projecting, analyzing, and visualizing storm cell data from weather radar. The pipeline includes cell detection, motion estimation using optical flow, cell property extraction, and persistence to NetCDF and SQLite/Parquet for guiding adaptive scanning.

Raut, Bhupendra Ashokrao [Argonne National Laborat↗