Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Catalog”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

EOSDIS CMR: Shifting Data Discovery & Use into a Higher Gear

Earth observation data comes in many forms, formats, and from a multitude of sources; to make the best of a very large and diverse data catalog (data from a dozen different national Distributed Active Archive Centers (DAACs) as well as international sources), NASA has created a one-stop-shop for earth data consumers to view, find, and get the data they need regardless of its original source or format, which is powered by a Common Metadata Repository (CMR). CMR is the underpinning that allows for the visualization, search, discovery, manipulation, and acquisition of a variety of datasets, and as such it is constantly evolving to do more and serve our communities better; CMR has embraced community ownership by making itself an open-source API, being compatible with Catalog Services for the Web (CSW) and OpenSearch APIs, and by encouraging the user community to make and share improvements.

CMR↗

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗

Solar-terrestrial data access distribution and archiving

It is recommended that a central data catalog and data access network (CDC/DAN) for solar-terrestrial research be established, initially as a NASA pilot program. The system is envisioned to be flexible and to evolve as funds permit, starting from a catalog to an access network for high-resolution data. The report describes the various functional requirements for the CDC/DAN, but does not specify the hardware and software architectures as these are constantly evolving. The importance of a steering committee, working with the CDC/DAN organization, to provide scientific guidelines for the data catalog and for data storage, access, and distribution is also stressed.

Source record↗

Mapping Inquiry Tool (MapIT) Database

The Mapping Inquiry Tool (MapIT) database consists of a geodatabase and data catalog of geologic, geophysical, structural, hydrologic, and contextual data, based on the data types to support geologic carbon storage activities and other subsurface energy systems resource assessments. The database was aggregated from publicly available data across the USA from state and federal entities. The database is structured by categories including rock unit geology, boundaries, national CS datasets, geophysical data, faults and structural data, infrastructure, surface hydrology, groundwater, and more. The data described in the data catalog is also available in the Mapping Inquiry Tool (https://edx.netl.doe.gov/dataset/mapping-inquiry-tool). Version 3 of the geodatabase and data catalog have been updated as of 5/17/2024. The database was published with a limited number of layers. The Catalog V3 contains many more resources than the geodatabase, documenting all layers that will be included in MapIT, and includes links to the original sources of the data. Within the catalog, in the final column, there is information about if the file is included in the geodatabase or not. Use the links provided in the catalog to download data directly from the original source if not included in the geodatabase. Four resources are included in this submission: 1. Geodatabase 2. ReadMe file 3. Catalog of data layers and additional data resources 4. Web link to a resource describing the motivation and reviewing the content of the geodatabase - DOE NETL Carbon Storage Site Mapping Inquiry Tool Database

carbon storage↗

Cloud - Aerosol LIDAR Infrared Pathfinder Satellite Observations (CALIPSO) - Data Management System: Data Products Catalog V4.95

The CALIPSO V4.51 Lidar Level 1 and Level 2 data product is an updated version of an already order-able dataset. The changes were signed off by the CALIPSO Configuration Control Board, versioned, and the code uploaded to a code repository. There is no ITAR/SBU data or code associated with this product. Data will be publicly order-able at the NASA LaRC Atmospheric Sciences Data Center (ASDC). All documentation and web sites will be made public once the data product is released. The data is in HDF4 format and will be generated for majority of the mission (June 2006 - August 2023). The attached Data Products Catalog (v4.95) describes the content of these new data products.

Mark Vaughan↗

The Sloan Digital Sky Survey Quasar Catalog: Sixteenth Data Release

We present the final Sloan Digital Sky Survey IV (SDSS-IV) quasar catalog from Data Release 16 of the extended Baryon Oscillation Spectroscopic Survey (eBOSS). This catalog comprises the largest selection of spectroscopically confirmed quasars to date. The full catalog includes two subcatalogs (the current versions are DR16Q_v4 and DR16Q_Superset_v3 at https://data.sdss.org/sas/dr16/eboss/qso/DR16Q/): a "superset" of all SDSS-IV/eBOSS objects targeted as quasars containing 1,440,615 observations and a quasar-only catalog containing 750,414 quasars, including 225,082 new quasars appearing in an SDSS data release for the first time, as well as known quasars from SDSS-I/II/III. We present automated identification and redshift information for these quasars alongside data from visual inspections for 320,161 spectra. Here, the quasar-only catalog is estimated to be 99.8% complete with 0.3%-1.3% contamination. Automated and visual inspection redshifts are supplemented by redshifts derived via principal component analysis and emission lines. We include emission-line redshifts for Hα, Hβ, Mg II, C III], C IV, and Lyα. Identification and key characteristics generated by automated algorithms are presented for 99,856 broad absorption-line quasars and 35,686 damped Lyman alpha quasars. In addition to SDSS photometric data, we also present multiwavelength data for quasars from the Galaxy Evolution Explorer, UKIDSS, the Wide-field Infrared Survey Explorer, FIRST, ROSAT/2RXS, XMM-Newton, and Gaia. Calibrated digital optical spectra for these quasars can be obtained from the SDSS Science Archive Server.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Cloud Aerosol LIDAR Infrared Pathfinder Satellite Observations (CALIPSO) - Data Management - Data Products Catalog V4.94

The CALIPSO V2.00 Lidar Level 2 Polar Stratospheric Cloud data product is an updated version of an already order-able dat set. The changes were signed off by the CALIPSO Configuration Control Board, versioned, and the code uploaded to a code repository. There is no ITAR/SBU data or code associated with this product. Data will be publicly order-able at the NASA LaRC Atmospheric Sciences Data Center (ASDC). All documentation and web sites will be made public once the data product is released. The data is in HDF4 format and will be generated for majority of the mission (June 2006 - March 2021). The attached Data Products Catalog (v4.94) describes this new product in section 2.14, pp 115- 119.

Mark Vaughan↗

The magnetosphere, ionosphere, and atmosphere as a system - Dynamics Explorer 5 years later

The Dynamics Explorer (DE) program summarizes its accomplishments during the first 5 years since the launch of the two DE satellites. This introduction to six review articles provides background information and a brief history of the program, especially citing the contributions of many people to its development. The principal investigators who had primary responsibility to implement the program are listed, together with the instruments they provided. Orbital information and approaches and constraints to data acquisition are explained. The brief description of the ground data processing and analysis system provides information on access to data catalogs and data sets. Each review article is then placed in the context of the categories of scientific objectives of the program.

Hoffman, R. A.↗

Prospective Seal Unit Spatial Extent Database for U.S. Sedimentary Basins

The Prospective Seal Unit Spatial Extent Database for U.S. Sedimentary Basins contains a series of spatial datasets representing spatial extents of publicly available data for caprock and seal rock units within the Appalachian Basin, Denver-Julesburg Basin, Great Valley Basin (Sacramento and San Joaquin Basins), Illinois Basin, Michigan Basin, San Juan Basin, U.S. Gulf Coast Basin, and Williston Basin. The database is designed to support carbon storage feasibility and resources assessment for carbon transport and storage (CTS) projects while displaying the spatial extent of prospective seal units and provide a guide to the original data source. This database leverages publicly available data resources from authoritative sources (e.g. U.S. Geological Survey, State Geologic Surveys, and published reports), and aims to help guide users to understand the seal unit's spatial coverage and data gaps from the regional to sub-basin/field scale. The database is organized by seal unit/formation, including the spatial extent for data found to be available for the seal unit. The various datasets represented include spatial extents of the lithologic formation, depth to top structural contour maps, and thickness/isopach maps. Included in this submission are the following resources: 1. Geodatabase/Dataset: “prospective-seal-unit-extents-2025.gdb” 2. ReadMe: “readme-prospective-seal-unit-spatial-extent-dataset-2025.pdf” 3. Data Catalog: “prospective-seal-unit-spatial-extents-data-catalog-2025.xlsx” 4. Data Sources Key: “data-source.csv” Please see NETL disclaimers here: https://netl.doe.gov/home/disclaimer

Basin↗

Assessing Low-Temperature Geothermal Play Types: Relevant Data and Play Fairway Analysis Methods

This data catalog contains information on low temperature geothermal play types. The U.S. Department of Energy (DOE) Geothermal Technologies Office (GTO) supports the Geothermal Heating and Cooling Geospatial Datasets and Analysis project, conducted by the National Renewable Energy Laboratory (NREL). This project is part of a broader effort to demonstrate the multifaceted value of integrating geothermal power and geothermal heating and cooling technologies into national decarbonization strategies and community energy plans. There is a need to establish baseline low-temperature geothermal resource data sets and evaluate methods for deploying these technologies. This project aims to reduce exploration risk of low temperature geothermal systems by collecting baseline datasets that can be used for Play Fairway Analysis methodologies. This data catalog contains links to publicly available datasets from different sources that can be relevant for the low temperature geothermal systems. This submission contains data catalogs for Alaska, Hawaii, and the Conterminous United States, as well as a technical report on the methods used to classify and asses the geothermal play types.

15 GEOTHERMAL ENERGY↗

Livewire Data Platform: A Catalog of Transportation and Mobility Data

A two-page fact sheet on the capabilities and core services of the Livewire Data Platform, a growing catalog of transportation and mobility-related data that empowers researchers and community planners to easily and securely share and preserve data that support projects and decision making.

47 OTHER INSTRUMENTATION↗

Livewire Data Platform: A Catalog of Transportation and Mobility Data

A two-page fact sheet on the capabilities and core services of the Livewire Data Platform, a growing catalog of transportation and mobility-related data that empowers researchers and community planners to easily and securely share and preserve data that support projects and decision making.

Livewire, EEMS, energy-efficient mobility systems,↗

NASA Environmental Justice Data Search Interface Overview

NASA’s Earth Science Division (ESD) is committed to empower Environmental Justice (EJ) communities by expanding awareness, accessibility, and use of Earth science data to enable contributions to Earth science research and applications. To that end, the NASA Earth Science Data Systems (ESDS) Program developed an EJ Data Catalog, a simple guide to NASA datasets and socioeconomic datasets that may be useful in EJ research. The EJ Data Catalog is divided by topics—such as disasters, urban flooding, extreme heat, food availability, water availability, climate, and health and air quality—and possible use cases for each dataset. The new version of the EJ Data Catalog is now integrated into NASA’s Science Discovery Engine (SDE), an open-source science infrastructure to enable collaborative and interdisciplinary science. In this workshop you will learn about NASA’s Equity and Environmental Justice (EEJ) activities and opportunities as well as participate on an interactive live demo of the new Science Discovery Engine for Environmental Justice.

environmental justice↗

AgRISTARS: Interim catalog ground data summary, data acquisition year 1979

To honor numerous requests for information about data holdings, and to facilitate the requirements specifications process, a series of interim catalogs are being developed. The 1979 data acquisition year is covered in this volume with subsequent years to follow under different covers. This catalog lists by state those sample segment numbers for which aircraft data has been acquired and/or field inventory products produced.

Doyle, H. M.↗

Catalog of Viking mission data

This catalog announces the present/expected availability of scientific data acquired by the Viking missions and contains descriptions of the Viking spacecraft, experiments, and data sets. An index is included listing the team leaders and team members for the experiments. Information on NSSDC facilities and ordering procedures, and a list of acronyms and abbreviations are included in the appendices.

Vostreys, R. W.↗