SEARCH · Engineering Papers

Results for “Active Archive Center”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

All sources NASA NTRS DOE OSTI

NASA NTRS2019

EARTHDATA PUB: A Data Publication Workflow Solution for NASA’s EOSDIS

Each NASA Distributed Active Archive Center (DAAC) faces the challenge of dealing with an increasingly diverse number of publishable data products from diverse data producers. Data producers, on the other hand, may experience pain points when interacting with the EOSDIS for the first time or when publishing different data at different DAACs. As a result, there has been a growing need to develop a common software framework that serves as a common interface for data producers, rigorously defines the data publication procedure for DAAC staff, facilitates the management of various data publication processes, and tracks the progress of data publication. This software should also account for the different configurations at different DAACs. Currently, two primary data publication workflow and tracking tools exist in operation at the EOSDIS: Semi-Automated ingest System (SAuS) and Data Publication workflow Portal (DAPPeR). However, neither tool is cloud-ready. Automated data processing could be managed by Cumulus, an EOSDIS cloud-based data ingest, archive and management system. However, Cumulus does not support manual tasks or on-premise implementations. We propose to develop the Earthdata Publication Minimum Viable Product (Earthdata Pub MVP) -- a cloud-hosted solution that works with both cloud and on-premise systems and implements the communications and exchange requirements generated by the Earthdata Pub information architecture team.

Rice, Justin L.↗

NASA NTRS2019

Design and Construction of a NASA Airborne and Field Investigation Inventory

NASA conducts airborne and field investigations that produce a wealth of valuable research data. Unfortunately, this data is often scattered across individual scientist hard drives or NASA Distributed Active Archive Centers and it can be difficult to locate and retrieve. Although satellite data has been successfully consolidated by tools such as EarthData Search, airborne and field investigation data present unique challenges stemming from the variability of temporal, spatial, platform, and instrument metadata. To address these difficulties with data retrieval and metadata variability, the Interagency Implementation and Concepts Team established an Airborne Data Management Group to improve airborne data search, understanding, access, and use. Surveys have been conducted of end users in order to build query lists that will drive the augmentation and standardization of existing metadata. Detailed metadata was then laboriously compiled from present and historic airborne and field investigations to build a database that will enable intelligent data search and retrieval. The inventory structure and function will be described and demonstrated. The purpose of this presentation is to bring awareness to this effort, to highlight and describe the issues and complications in development, and to increase user interest prior to public release in 2020.

Davis, Carson↗

NASA NTRS2019

Collecting and Processing Earth Science Data Metrics at NASA ESDIS

Since the launch of Terra satellite in 1999, the number of Earth Science remote sensing data products created and distributed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) has increased from a few hundred to nearly ten thousand. NASA's Earth Science Data and Information System (ESDIS) Metrics System (EMS) collects metrics on data ingest, archive, and distribution by its Distributed Active Archive Centers (DAACs) and the Science Investigator-led Systems (SIPS), known as Data Providers. These metrics are critical in helping NASA management as well as data producers in resource planning and gaining a wide range of knowledge of data users and data usage.EMS receives flat files, or log files of data archive, ingest, and distribution either in their raw format, such as Apache web logs, or text files of log records formatted by the Data Providers. Tens of millions of records are processed each day to extract metrics on data products, user information, distribution protocols and services, and so on. The metrics are then made available to designated parties.This presentation provides an overview of the EMS processing workflow and improvement efforts made in recent years to handle ever-increasing number of data records and new metrics requirements, discusses several key steps including mapping log records to data products and identifying user communities along with geo-distribution, and demonstrates typical metrics capabilities produced by the EMS system. Challenges and potential approaches to improve the system are also discussed.

Pan, Jianfu↗

NASA NTRS2019

Reusability in NASA's Earth Observation System Data and Information System (EOSDIS)

NASA's Earth Observation System Data and Information System (EOSDIS) has been operational since 1994. The term FAIR is relatively recent compared to the operational life of EOSDIS. However, given the evolutionary nature of EOSDIS in response to both the state of the art and community expectations, it is useful to assess where EOSDIS stands relative to the FAIR data management. In this presentation we evaluate EOSDIS against the Reusability principle, which calls for a clear and accessible data usage license, rich description of data, detailed provenance, and compliance with domain-relevant community standards.Achieving consistent compliance for datasets is particularly challenging with the diverse community of missions and scientists providing data to EOSDIS. To address this, EOSDIS's constituent Distributed Active Archive Centers work with data producers to utilize the applicable standards and conventions. In addition, a guidebook is under development to provide comprehensive and understandable guidance on how to construct compliant, and thus more reusable, data products.

Hampapuram (Rama) Ramapriyan↗

NASA NTRS2013

TPSAS-NF1676L-17783-DND

The ASDC, in its role as an EOS-DIS (Earth Observing System Data and Information System) DAAC (Distributed Active Archive Center) has made substantial improvements to the way in which data is delivered. The architecture has been developed, in response to emerging customer needs to support multiple paths for access.

Tiffany Mathews↗

NASA NTRS2020

Satellite-based remote sensing data set of global surface water storage change from 1992 to 2018

The recent availability of freely and openly available satellite remote sensing products has enabled the implementation of global surface water monitoring at a level not previously possible. Here we present a global set of satellite-derived time series of surface water storage variations for lakes and reservoirs for a period that covers the satellite altimetry era. Our goals are to promote the use of satellite-derived products for the study of large inland water bodies and to set the stage for the expected availability of products from the Surface Water and Ocean Topography (SWOT) mission, which will vastly expand the spatial coverage of such products,expected from 2021 on. Our general strategy is to estimate global surface water storage changes (1V) in large lakes and reservoirs using a combination of paired water surface elevation (WSE) and water surface area (WSA)extent products. Specifically, we use data produced by multiple satellite altimetry missions (TOPEX/Poseidon,Jason-1, Jason-2, Jason-3, and Envisat) from 1992 on, with surface extent estimated from Terra/Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) from 2000 on. We leverage relationships between elevation and surface area (i.e., hypsometry) to produce estimates of 1 Veven during periods when either of the variables was not available. This approach is successful provided that there are strong relationships between the two variables during an overlapping period. Our target is to produce time series of1Vas well as of WSE and WSA fora set of 347 lakes and reservoirs globally for the 1992–2018 period. The data sets presented and their respective algorithm theoretical basis documents are publicly available and distributed via the Physical Oceanography Dis-tributed Active Archive Center (PO DAAC; https://podaac.jpl.nasa.gov/, last access: 13 May 2020) of NASA’sJet Propulsion Laboratory. Specifically, the WSE data set is available at https://doi.org/10.5067/UCLRS-GREV2(Birkett et al., 2019), the WSA data set is available at https://doi.org/10.5067/UCLRS-AREV2 (Khandelwal and Kumar, 2019), and the 1V data set is available at https://doi.org/10.5067/UCLRS-STOV2 (Tortini et al., 2019).The records we describe represent the most complete global surface water time series available from the launch of TOPEX/Poseidon in 1992 (beginning of the satellite altimetry era) to the near present. The production of long-term, consistent, and calibrated records of surface water cycle variables such as in the data set presented here is of fundamental importance to baseline future SWOT products

Riccardo Tortini↗

NASA NTRS2019

An Integrated Data Analytics Platform

An Integrated Science Data Analytics Platform is an environment that enables the confluence of resources for scientific investigation. It harmonizes data, tools and computational resources which subsequently enable the research community to focus on the investigation rather than spending time on security, data preparation, management, etc. OceanWorks is a NASA technology integration project to establish a cloud-based Integrated Ocean Science Data Analytics Platform at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) for big ocean science. It focuses on advancement and maturity by bringing together several NASA open-source, big data projects for parallel analytics, anomaly detection, in-situ to satellite data matchup, quality-screened data subsetting, search relevancy, and data discovery. Our communities are relying on data distributed through data centers such as the PO.DAAC, COAPS, NCAR, and many others to conduct their research. In typical investigations, scientists would engage in: search for data, evaluate the relevance of that data, download it, and then apply algorithms to identify trends. Such workflow cannot scale if the research involves a massive amount of data or multi-variate measurements. NASA’s Surface Water and Ocean Topography (SWOT) mission is expected to produce massive amount of observational data during its 3-year nominal mission. Collections like SWOT challenges all existing Earth Science data archival, distribution and analysis paradigms. In this paper, we will discuss how OceanWorks enhances the analysis of physical ocean data where the computation is done on an elastic cloud platform next to the archive to deliver fast, web-accessible services for working with oceanographic measurements.

Yang, Chaowei↗

NASA NTRSDate not supplied

Application of ML/AI for Identifying Earth Science Datasets in Research Publications

NASA Data Active Archive Centers, or DAACs, ingest, store and distribute data acquired from satellites, ground systems as well as modelling data. These data are organized by the datasets, each presenting collection of files usually associated with the certain mission, instrument, processing level, parameter(s), algorithm and/or model. The number of datasets offered by a single DAAC to the public varies. GES DISC, for example, currently offers for public use approximately ~1,300 datasets. While each publicly offered dataset comes with supporting documentation, it is challenging for novice and even experienced scientists to navigate among the datasets that offer similar parameters to find the datasets for their particular research application. Supplying dataset documentation with the scientific paper citations that refer to that dataset provides means for the dataset users to educate themselves with the application research that dataset is being used in. Collecting citations of the papers that use the datasets for their research yield valuable insights into application areas of those datasets, information about usage of the dataset groups for specific applications and those application topics. It also gives insights into the “deep metrics” of the dataset usage, as opposed to the common metrics of the dataset usage such as number of users who downloaded the dataset files and volumes of downloaded data. Association of a certain scientific paper with the dataset(s) presents a challenge because most of the paper authors do not properly cite the datasets, datasets usually have cryptic names and Digital Object Identifiers (DOIs) that are used for dataset identification were assigned to the datasets only few years ago. Simple Google or online library search do not provide even meaningful fraction of the results when performed by the dataset name or DOI, however they provide too many results when the search is done by more broader terms such as mission and instrument names. Attempts to create an AI system capable to identify dataset in the scientific papers have already been made using neural networks classifiers on the basis of the dataset mission, instrument and variable name. This method was applied to NASA SEDAC, which has 41 datasets in total. In GES DISC there can be as many as ~100 datasets per mission/instrument with some of the datasets consisting of multiple variables so there is a need for more differentiating parameters for dataset identification in the paper. The approach we are currently investigating is creating AI classifiers that are based on multiple dataset features, or keywords, extracted from the NASA Earthdata Common Dataset Repository (CMR). The features are weighted based on how precisely they can identify a dataset. The classifier uses preprocessed paper text as input and searches for the CMR datasets whose feature sets are the closest to the feature sets contained in the paper. The challenges of dataset identification include variety of ways the paper authors describe the datasets in their papers and incomplete tagging of the CMR dataset description (DIFs).

Irina Gerasimov↗

NASA NTRSDate not supplied

An Automated Approach to Labelling Datasets in Earth Science Publications

NASA Data Active Archive Centers, orDAACs, ingest, store, and distribute dataacquired from satellites, ground systems as well asreanalysis models. Many authors use this datain their research. However, most of the datasets usedin Earth Science Publications are not citedcorrectly or not cited at all. Thus, there is no directlink between the datasets used and thescientific publications which reference them. Thisleads to issues with reproducibility of theresults, attribution of the research results, anddiscovery of new datasets. This project began byexploring various methods of automatically labellingGoddard Earth Sciences Data andInformation Services Center (GES DISC) datasets usingSupervised Machine Learning and EarthData Search Common Metadata Repository (CMR) queries.The ultimate goal was to create alibrary of citations that utilized automated citationlabeling to directly link the researchpublications to the data they use. Supervised MachineLearning approaches struggled due to thelimited amount of labelled training data to learnfrom. Increasing the volume of training data isdifficult as it requires subject matter experts todevote time to manually reviewing journalarticles and determining the datasets used. The CMRqueries were inconsistent because theunderlying metadata is continuously being updated.Thus, it is hard to generalize theeffectiveness of the CMR results as they are dependenton the internal state of CMR. Theseapproaches helped inform the decision to transitionthe project into using a Knowledge Graph.Another key aspect of this project focused on theautomated extraction of features (platform,instrument, variables, etc) and explicit citationsfrom within Earth Science Publications. Theseautomated extractions were used to classify researchpapers based on their platform/instrumentcouples. This information was input into the CitationManagement System for GES DISC. Theseplatform/instrument couples also provide an additionalfacet that can be searched on the GESDISC website.

Edward Jahoda↗

NASA NTRSDate not supplied

Earth Observation Data Provenance for Future Climate Research - Requirements and Challenges

Observations and measurements of the Earth’s environment have been collected from space since the 1960’s. Flight Projects, airborne and field campaigns have developed and operated multi-year missions with global observing instruments, and Principle Investigator Science Teams have developed algorithms, calibrated, and derived a wide variety of Earth system environmental parameters. These data are archived and distributed for research purposes by NASA’s ESDIS Project and Distributed Active Archive Centers (DAACs). They are expected to be an important basis for Earth science and climate change research extending well beyond their observation times, the Principle Investigators/Science Teams research projects and life of the Flight Projects.

John Moses↗

NASA NTRSDate not supplied

NASA's Earthdata Forum: Bringing the Science Community Together

The Earthdata Forum is an actively maintained and operational user forum for all participating NASA EOSDIS Distributed Active Archive Centers (DAACs) and GCMD. The Forum serves as a cross-DAAC platform from which user communities can obtain authoritative information relating to NASA Earth Science. The Atmospheric Science Data Center (ASDC) DAAC takes on the role of both Project Manager and Hosting Organization. As Project Manager, the ASDC leads and facilitates the project's efforts. The Forum launched February 2020 with five participating DAACs: Atmospheric Science Data Center (ASDC), Crustal Dynamics Data Information System (CDDIS), Goddard Earth Science Data and Information Services Center (GES DISC), Global Hydrology Resource Center (GHRC), and Land Processes DAAC (LP DAAC). Since February 2020, the of Level 1 and Atmosphere Archive and Distribution System DAAC (LAADS), the Ocean Biology DAAC (OBDAAC) and Global Change Master Directory (GCMD) have also joined the Forum. The Forum assumes both a proactive and reactive approach to NASA products scientific queries, allowing the scientific community to ask questions about NASA's products, tools, and services and receive announcements and news regarding products, new data releases, and such. In this presentation, we will describe the Earthdata Forum approach to support the scientific community in detail and discuss the successes and challenges of this approach.

Ingrid Garcia-Solera↗

NASA NTRSDate not supplied

Stewardship Best Practices for Improved Discovery and Reuse of Heterogeneous and Cross-Disciplinary Earth System Data

Some of the Earth system data products such as those from NASA airborne and field investigations (a.k.a. campaigns), are highly heterogeneous and cross-disciplinary, making the data extremely challenging to manage. For example, airborne and field campaign measurements tend to be sporadic over a period of time, with large gaps. Data products generated are of various processing levels and utilized for a wide range of inter- and cross-disciplinary research and applications. Data and derived products have been historically stored in a variety of domain-specific standard (and some non-standard) formats and in various locations such as NASA Distributed Active Archive Centers (DAACs), NASA airborne science facilities, field archives, or even individual scientists’ computer hard drives. As a result, airborne and field campaign data products have often been managed and represented differently, making it onerous for data users to find, access, and utilize campaign data. Some difficulties in discovering and accessing the campaign data originate from the incomplete data product and contextual metadata that may contain details relevant to the campaign (e.g. campaign acronym and instrument deployment locations), but tend to lack other significant information needed to understand conditions surrounding the data. Such details can be burdensome to locate after the conclusion of a campaign. Utilizing consistent terminology, essential for improved discovery and reuse, is also challenging due to the variety of involved disciplines. To help address the aforementioned challenges faced by many repositories and data managers handling airborne and field data, this presentation will describe stewardship practices developed by the Airborne Data Management Group (ADMG) within the Interagency Implementation and Advanced Concepts Team (IMPACT) under the NASA’s Earth Science Data systems (ESDS) Program.

best practices↗

NASA NTRSDate not supplied

pyQuARC: Open Source Library for Earth Observation Metadata Quality Assessment

Metadata quality is essential to effective data discovery and has become increasingly vital as more Earth Science data sets become available. The Common Metadata Repository (CMR) hosts metadata describing NASA’s Earth Observation data products, which are archived across 12 Distributed Active Archive Centers (DAACs). The Analysis and Review of CMR (ARC) Team, located at Marshall Space Flight Center, conducts metadata quality assessments to ensure that these data products are discoverable, accessible, and usable. To achieve these goals, the ARC team has developed a metadata quality assessment framework to evaluate metadata completeness, correctness, and consistency. ARC uses a combination of manual and automated methods to assess these three components and identify areas of improvement; the team then collaborates with the DAACs to resolve any findings. To streamline this process, ARC is currently developing a host of scripts, known as pyQuARC, to automate metadata quality assessments as much as possible. pyQuARC is an open source library for Earth Observation Metadata Quality Assessment, and the tool utilizes ARC’s metadata quality assessment framework to make basic validation checks, pinpoint inconsistencies between dataset-level (i.e. collection) and file-level (i.e. granule) metadata, and identify opportunities for more descriptive and robust information. Since pyQuARC is also customizable, other users can make modifications as needed, and future metadata standards can also be implemented. Once pyQuARC is fully developed, it will support multiple schema types to serve the broader EOSDIS metadata community. This presentation will provide an overview of pyQuARC and its process of development while showcasing the tool’s valuable features and uses.

Jenny Wood↗

NASA NTRSDate not supplied

New Ways of Facilitating Improved Data Discovery and Access for NASA's Suborbital Earth Science Observations

NASA conducts field research in various Earth Science disciplines utilizing airborne and other non-satellite platforms to acquire in situ and remotely sensed observations indicative of physical processes across a range of scales. Field efforts are key in the development and validation of instruments and satellite algorithm refinements. The heterogeneous data, with a range of file formats, scales, and acquisition methods, support research in several science areas. NASA’s archive process assigns data products to discipline-oriented Distributed Active Archive Centers (DAACs) for stewardship. Over time, individual DAACs have developed tools for data browsing and serving disparate user bases. As science becomes more interdisciplinary, researchers need to incorporate observations from multiple campaigns, and multiple DAACs, into their work. Motivated in part by this shifting paradigm of needs, the Catalog of Archived Suborbital Earth Science Investigations (CASEI) was created. CASEI provides a single starting point to browse, search, and discover airborne and field data. Contextual metadata are organized and inter-linked allowing intuitive, integrated exploration across all NASA DAACs. Campaign science objectives, platform and instrument configurations, geographical details, geophysical concepts, and more are tracked in CASEI’s database, facilitating multi-parameter search, browse, and discovery of relevant data products. Researchers are able to directly access associated data products, via DOI links, regardless of the DAAC where they reside. Significant events, key time periods of high science interest within the longer-duration campaign effort, are also indicated and allow for a more efficient identification of critical data subsets. This presentation describes CASEI’s development, intensive metadata curation process, and demonstrates the web interface experience. Initial content metrics and plans for continued maintenance will also be discussed.

metadata↗

NASA NTRSDate not supplied

Maximizing Earth Science Observations with Data Harmonization: Harmonized Landsat/Sentinel-2

In August 2021, NASA released science quality harmonized Landsat/Sentinel-2 products for both cloud-based access and direct download from the Land Processes Distributed Active Archive Center (LP DAAC). These 30-meter products, HLSS30 (Sentinel-2 component) and HLSL30 (Landsat component) are placed on the same grid and are generated and distributed fully in the cloud. Similarly, ESA is prototyping harmonized Landsat Sentinel-2 products at 10-meter resolution. The data production and science teams from NASA and ESA will present technical details on the data production system, data product status and availability, and the benefit of harmonizing instruments with similar sensing characteristics between the agencies.

Earth Science↗

NASA NTRSDate not supplied

FAIR-ness Assessment of NASA’s Earth Observation System Data and Information System (EOSDIS)

This presentation addresses the challenge of evaluating a multi-disciplinary institutional network of data repositories in operation since 1994 against the relatively recent criteria that constitute FAIR (Findable, Accessible, Interoperable, Reusable) data. NASA’s Earth Observation System Data and Information System (EOSDIS), with its 12 discipline-based Distributed Active Archive Centers (DAACs), preceded the definition and popularization of FAIR by over two decades. An assessment is very useful to describe how well the FAIR principles are met and to identify any improvements needed. In 2020, A “self-assessment” of EOSDIS and DAACs was performed by the ESDIS Project staff and the DAACs from the points of view of human actionability and machine actionability. More recently, a draft of a Science Mission Directorate (SMP) Program Directive (SPD-41a) has been released by NASA Headquarters for comment, where it is recommended that all SMD-funded data should follow the FAIR principles. This presentation is timely to initiate community discussion within the Information Quality Cluster (IQC) of the Earth Science Information Partners (ESIP) and help strategize and develop implementation guidelines for EOSDIS and DAACs to conform to FAIR principles.

Remote sensing↗

NASA NTRSDate not supplied

Resources at GES DISC for Agriculture Studies

● Introduction to GES DISC (Distributed Active Archive Center- DAAC) ○ Data Holding for Agriculture study ● Resources for Agricultural studies at GES DISC ○ Data Access ○ Data Services and Tools ○ Learning resources : ■ Studying seasonality ■ Studying extreme events: Floods, Flash Floods, Heat Waves, Drought ■ Data Recipe (How-To) ● Demo and hands-on

Zhong Liu↗

NASA NTRSDate not supplied

Quantitative Highlights of 20 years Aqua Data Archive and Data Usage

NASA’s Aqua satellite carries six Earth-observing instruments Atmospheric Infrared Sounder (AIRS), Advanced Microwave Scanning Radiometer for EOS (AMSR-E), Advanced Microwave Sounding Unit (AMSU), Clouds and the Earth’s Radiant Energy System (CERES), Humidity Sounder for Brazil (HSB) and Moderate Resolution Imaging Spectroradiometer (MODIS). Currently only four of six instruments are collecting data, two instruments that stopped transmitting data are AMSR-E that suffered a major anomaly in October 2011 and was powered off in March 2016 while as HSB failed in February 2003. NASA’s Earth Science Data and Information System (ESDIS) Project makes these data, along with derived products, available to worldwide data users. Since the launch of Aqua on May 4, 2002, more than 10,000 data products have been archived and distributed by NASA-funded Distributed Active Archive Centers (DAACs) that are part of NASA’s Earth Observing System Data and Information System (EOSDIS). At the end of the 2021 Fiscal Year with over 100,000 orbits data, about 1,000 Aqua data products constituted almost 16.5 % of the entire EOSDIS data archive volume (8.6 PB out of approximately 55.2 PB), and 7.5 PB of Aqua data were distributed to over half-a-million public users worldwide. By categorizing the Aqua data products and their distribution, we can get a quantitative assessment of Aqua data usage. NASA’s ESDIS Project has collected archive, distribution, and user information from EOSDIS data users since February 2000. These metrics are available through the ESDIS Metrics System (EMS). EMS information is stored in a relational database from which quantitative metrics of Aqua data use can be retrieved and analyzed. The purposes of this study are to: 1) perform a comprehensive investigation of the 20-year trend in the archive and distribution of Aqua data products; 2) identify and characterize data product usage over the last 20 years; and 3) identify and characterize the global user community for these data. In addition to revealing how Aqua data use has evolved over time, the results of this study provide insights on identifying the various user communities for different kinds of Earth science data products. Also, because of the enormous quantity of data handled by EOSDIS DAACs, the study provides guidance of the requirements for future data systems that will be needed to effectively and efficiently handle the ever-increasing amounts of Earth science data produced by future (and ongoing) Earth science missions.

Lalit Wanchoo↗