SEARCH · Engineering Papers

Results for “netCDF-4”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

6 records

All sources NASA NTRS DOE OSTI

NASA NTRS2017

Moving from HDF4 to HDF5/netCFD-4

In this presentation, we will go over the major differences between two file formats and libraries, and will talk about the HDF5 features that users should consider when designing new products in HDF5netCDF4. We will also discuss the h4h5tools toolkit that can facilitate conversion of data in the existing HDF4 files to HDF5 and netCDF-4, and we will engage the participants in the discussion of how The HDF Group can help with the transition and adoption of HDF5 and netCDF-4.

netCDF-4↗

NASA NTRS2016

National Climate Assessment - Land Data Assimilation System (NCA-LDAS) Data and Services at NASA GES DISC

The National Climate Assessment-Land Data Assimilation System (NCA-LDAS) is an Integrated Terrestrial Water Analysis, and is one of NASAs contributions to the NCA of the United States. The NCA-LDAS has undergone extensive development, including multi-variate assimilation of remotely-sensed water states and anomalies as well as evaluation and verification studies, led by the Goddard Space Flight Centers Hydrological Sciences Laboratory (HSL). The resulting NCA-LDAS data have recently been released to the general public and include those from the Noah land-surface model (LSM) version 3.3 (Noah-3.3) and the Catchment LSM version Fortuna-2.5 (CLSM-F2.5). Standard LSM output variables including soil moistures temperatures, surface fluxes, snow cover depth, groundwater, and runoff are provided, as well as streamflow using a river routing system. The NCA-LDAS data are archived at and distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). The data can be accessed via HTTP, OPeNDAP, Mirador search and download, and NASA Earth data Search. To further facilitate access and use, the NCA-LDAS data are integrated into the NASA Giovanni, for quick visualization and analysis, and into the Data Rods system, for retrieval of time series of long time periods. The temporal and spatial resolutions of the NCA-LDAS data are, respectively, daily-averages and 0.125x0.125 degree, covering North America (25N 53N; 125W 67W) and the period January 1979 to December 2015. The data files are in self-describing, machine-independent, CF-compliant netCDF-4 format.

data assimiliation↗

NASA NTRS2019

OPeNDAP and HDF5 in the Cloud: Techniques and Best Practices for Serving HDF5 Data

In our talk we will discuss considerations and best practices for organizing data and metadata in HDF5 files when serving ESDIS products in S3 with OPeNDAP. The first part of the talk will be devoted to the best practices for organizing data and user-defined metadata including CF conventions in HDF5. We will also address interoperability with netCDF-4 and its role in accessing data in HDF5. In the second part of the talk we will go over OPeNDAP considerations for S3 access to HDF5 data.

HDF5↗

NASA NTRSDate not supplied

Interoperability and Other Aspects of Guiding Data Producers for the Benefit of End Users

The purpose of this paper is to discuss how the Climate and Forecast (CF) Metadata Conventions and netCDF standard have influenced the recommendations and guidance provided to producers of data products based on NASA’s Earth observations. It has been long-recognized that interoperable datasets and use of standards and conventions are beneficial to the users of these datasets, especially those who make use of multiple datasets for their research and applications. The Dataset Interoperability Working Group (DIWG), one of NASA’s Earth Science Data System Working Groups (ESDSWGs), was established in 2013, and has developed and published many recommendations. The Data Product Development Guide (DPDG) Working Group, established in 2018 as another of the ESDSWGs, has published a DPDG for Data Producers and a Quick Start Guide, incorporating guidance from many sources, including the recommendations from the DIWG. The DPDG includes recommendations regarding data formats (prominently netCDF-4) and metadata based primarily on the CF Metadata Conventions and the Attribute Convention for Data Discovery (ACDD). In early 2023, it was decided that the Resource Center for Data Producers (RCDP) Working Group be established as another ESDSWG, with the goals of providing all the information relevant and helpful for data producers via an easily accessible website, and of recommending how the DPDG and QSG could be maintained as living documents, given the rapidly changing technologies, and the need for incorporating the experience and feedback from the users of these documents.

Data product development↗

NASA NTRSDate not supplied

Enabling Cloud Services and Enhanced Data Discovery With Earthdata-Varinfo

NASA’s Earth Observing System Data and Information System (EOSDIS) contains thousands of Earth science datasets from satellites, models, and field campaigns. Each of these collections can contain hundreds of variables that describe each measurement within the dataset, therefore an automated method for generating UMM-Var records is necessary. The Unified Metadata Model for Variables (UMM-Var) provides a framework for variable metadata records in NASA’s Common Metadata Repository (CMR). The Python tool, earthdata-varinfo, was developed to solve this problem of automating the curation of UMM-Var records. Given either a collection DMR file or a netCDF-4 file, earthdata-varinfo can scrape variable metadata and return a CMR compliant UMM-Var record. Earthdata-varinfo can generate thousands of UMM-Var records in a matter of seconds, thus enabling subsetting capabilities and enhancing data discovery.

Eni Awowale↗

NASA NTRSDate not supplied

Simplifying Analysis of Hierarchical HDF5 and NetCDF4 Files with Xarray-Datatree

NASA’s Earth Observing System Data and Information System (EOSDIS) contains thousands of Earth science datasets from satellites, models, and field campaigns. EOSDIS data are stored in formats that are well supported by the Earth Science community. These formats include the Hierarchical Data Format (HDF), with derivative flavors such as HDF-5 and the Network Common Data Format (NetCDF-4). The HDF specification allows for a directory-like hierarchy within a single file, known as "groups". Observational data and associated metadata within a single file can be distributed amongst multiple internal groups, which can also be nested to multiple levels. Working with datasets that have a group hierarchical structure can be difficult because of the nested structure of groups. Widely used packages, such as xarray, have data models that do not accommodate the hierarchical structure within HDF files, requiring users to traverse the file and open different HDF groups as separate, unrelated objects. Xarray-datatree is a Python package developed to solve the difficulty of traversing HDFs with a hierarchical group structure by creating a tree-like hierarchical data structure in xarray. The tree-like structure allows each group to be accessed once a DataTree object is instantiated. The migration of xarray-datatree into the xarray core library will reduce barriers to accessing Earth science data by eliminating the need to understand and traverse the specific hierarchy of a grouped HDF file.

Eni Awowale↗