Engineering Papers⌕ Search

Engineering topics

Owen Littlejohns

Publications and source records attributed to Owen Littlejohns.

7 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Zarr stores in NASA's EOSDIS

Data hosted by the National Aeronautics and Space Administration (NASA) are expected to increase to over 600 PB by the end of the decade (see Figure 1). This rapid increase is, in part, driven by the launch of new, high-data-volume Earth observing missions, such as the NASA-Indian Space Research Organisation (ISRO) Synthetic Aperture Radar (NISAR), Surface Water and Ocean Topography (SWOT), Tropospheric Emissions: Monitoring of Pollution (TEMPO) and Plankton, Aerosol, Cloud ocean Ecosystem (PACE) missions.

Owen Littlejohns↗
NASA NTRSDate not supplied

Enabling Cloud Services and Enhanced Data Discovery With Earthdata-Varinfo

NASA’s Earth Observing System Data and Information System (EOSDIS) contains thousands of Earth science datasets from satellites, models, and field campaigns. Each of these collections can contain hundreds of variables that describe each measurement within the dataset, therefore an automated method for generating UMM-Var records is necessary. The Unified Metadata Model for Variables (UMM-Var) provides a framework for variable metadata records in NASA’s Common Metadata Repository (CMR). The Python tool, earthdata-varinfo, was developed to solve this problem of automating the curation of UMM-Var records. Given either a collection DMR file or a netCDF-4 file, earthdata-varinfo can scrape variable metadata and return a CMR compliant UMM-Var record. Earthdata-varinfo can generate thousands of UMM-Var records in a matter of seconds, thus enabling subsetting capabilities and enhancing data discovery.

Eni Awowale↗
NASA NTRSDate not supplied

Collaborating With Xarray to Enable Reading Hierarchical Data Files

NASA has a lot of expertise, but doesn’t need to write every single piece of code. Pangeo is a fantastic open-source community of tools for geoscience research. Xarray, a Python package, is a widely used part of this ecosystem for accessing and analyzing geoscience data.

Owen Littlejohns↗
NASA NTRSDate not supplied

Simplifying Analysis of Hierarchical HDF5 and NetCDF4 Files with Xarray-Datatree

NASA’s Earth Observing System Data and Information System (EOSDIS) contains thousands of Earth science datasets from satellites, models, and field campaigns. EOSDIS data are stored in formats that are well supported by the Earth Science community. These formats include the Hierarchical Data Format (HDF), with derivative flavors such as HDF-5 and the Network Common Data Format (NetCDF-4). The HDF specification allows for a directory-like hierarchy within a single file, known as "groups". Observational data and associated metadata within a single file can be distributed amongst multiple internal groups, which can also be nested to multiple levels. Working with datasets that have a group hierarchical structure can be difficult because of the nested structure of groups. Widely used packages, such as xarray, have data models that do not accommodate the hierarchical structure within HDF files, requiring users to traverse the file and open different HDF groups as separate, unrelated objects. Xarray-datatree is a Python package developed to solve the difficulty of traversing HDFs with a hierarchical group structure by creating a tree-like hierarchical data structure in xarray. The tree-like structure allows each group to be accessed once a DataTree object is instantiated. The migration of xarray-datatree into the xarray core library will reduce barriers to accessing Earth science data by eliminating the need to understand and traverse the specific hierarchy of a grouped HDF file.

Eni Awowale↗