Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data archive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Quantitative Highlights of 20 years Aqua Data Archive and Data Usage

NASA’s Aqua satellite carries six Earth-observing instruments Atmospheric Infrared Sounder (AIRS), Advanced Microwave Scanning Radiometer for EOS (AMSR-E), Advanced Microwave Sounding Unit (AMSU), Clouds and the Earth’s Radiant Energy System (CERES), Humidity Sounder for Brazil (HSB) and Moderate Resolution Imaging Spectroradiometer (MODIS). Currently only four of six instruments are collecting data, two instruments that stopped transmitting data are AMSR-E that suffered a major anomaly in October 2011 and was powered off in March 2016 while as HSB failed in February 2003. NASA’s Earth Science Data and Information System (ESDIS) Project makes these data, along with derived products, available to worldwide data users. Since the launch of Aqua on May 4, 2002, more than 10,000 data products have been archived and distributed by NASA-funded Distributed Active Archive Centers (DAACs) that are part of NASA’s Earth Observing System Data and Information System (EOSDIS). At the end of the 2021 Fiscal Year with over 100,000 orbits data, about 1,000 Aqua data products constituted almost 16.5 % of the entire EOSDIS data archive volume (8.6 PB out of approximately 55.2 PB), and 7.5 PB of Aqua data were distributed to over half-a-million public users worldwide. By categorizing the Aqua data products and their distribution, we can get a quantitative assessment of Aqua data usage. NASA’s ESDIS Project has collected archive, distribution, and user information from EOSDIS data users since February 2000. These metrics are available through the ESDIS Metrics System (EMS). EMS information is stored in a relational database from which quantitative metrics of Aqua data use can be retrieved and analyzed. The purposes of this study are to: 1) perform a comprehensive investigation of the 20-year trend in the archive and distribution of Aqua data products; 2) identify and characterize data product usage over the last 20 years; and 3) identify and characterize the global user community for these data. In addition to revealing how Aqua data use has evolved over time, the results of this study provide insights on identifying the various user communities for different kinds of Earth science data products. Also, because of the enormous quantity of data handled by EOSDIS DAACs, the study provides guidance of the requirements for future data systems that will be needed to effectively and efficiently handle the ever-increasing amounts of Earth science data produced by future (and ongoing) Earth science missions.

Lalit Wanchoo↗
NASA NTRS1994

Constraint based scheduling for the Goddard Space Flight Center distributed Active Archive Center's data archive and distribution system

The Goddard Space Flight Center (GSFC) Distributed Active Archive Center (DAAC) has been operational since October 1, 1993. Its mission is to support the Earth Observing System (EOS) by providing rapid access to EOS data and analysis products, and to test Earth Observing System Data and Information System (EOSDIS) design concepts. One of the challenges is to ensure quick and easy retrieval of any data archived within the DAAC's Data Archive and Distributed System (DADS). Over the 15-year life of EOS project, an estimated several Petabytes (10(exp 15)) of data will be permanently stored. Accessing that amount of information is a formidable task that will require innovative approaches. As a precursor of the full EOS system, the GSFC DAAC with a few Terabits of storage, has implemented a prototype of a constraint-based task and resource scheduler to improve the performance of the DADS. This Honeywell Task and Resource Scheduler (HTRS), developed by Honeywell Technology Center in cooperation the Information Science and Technology Branch/935, the Code X Operations Technology Program, and the GSFC DAAC, makes better use of limited resources, prevents backlog of data, provides information about resources bottlenecks and performance characteristics. The prototype which is developed concurrently with the GSFC Version 0 (V0) DADS, models DADS activities such as ingestion and distribution with priority, precedence, resource requirements (disk and network bandwidth) and temporal constraints. HTRS supports schedule updates, insertions, and retrieval of task information via an Application Program Interface (API). The prototype has demonstrated with a few examples, the substantial advantages of using HTRS over scheduling algorithms such as a First In First Out (FIFO) queue. The kernel scheduling engine for HTRS, called Kronos, has been successfully applied to several other domains such as space shuttle mission scheduling, demand flow manufacturing, and avionics communications scheduling.

Short, Nick, Jr.↗
NASA NTRS1992

Archiving data from ground-based telescopes

The scientific throughput of a particular observing facility has been demonstrated to be multiplied with the operation of a data archive and its corresponding retrieval system. A requisite to achieve such an exploitation is a well structured observations catalog, i.e. a catalog that includes all information necessary to reduce and analyze the data even many years after its acquisition. At the same time, an information system is required that allow users to browse through the catalog at different levels of detail, adapting the amount of information presented to the actual needs of the user. Archiving data acquired with ground-based telescopes is particularly difficult because of the relative short life-time of instruments and detectors in comparison to the expected life-time of the archive. This feature differentiates ground-based originated archives radically from its spaceborne counterparts. The organization of the observations catalog becomes highly dependent on the capability of the archive to deal with new instrumental configurations. We introduce in this paper, the concept of a catalog database as opposed to the static catalog design currently in use in many archiving facilities, as a method to deal with this problem. We also present a brief review of activities currently in progress in this area.

Albrecht, M. A.↗
NASA NTRS2016

Providing Comprehensive and Consistent Access to Astronomical Observatory Archive Data: The NASA Archive Model

Since the turn of the millennium a constant concern of astronomical archives have begun providing data to the public through standardized protocols unifying data from disparate physical sources and wavebands across the electromagnetic spectrum into an astronomical virtual observatory (VO). In October 2014, NASA began support for the NASA Astronomical Virtual Observatories (NAVO) program to coordinate the efforts of NASA astronomy archives in providing data to users through implementation of protocols agreed within the International Virtual Observatory Alliance (IVOA). A major goal of the NAVO collaboration has been to step back from a piecemeal implementation of IVOA standards and define what the appropriate presence for the US and NASA astronomy archives in the VO should be. This includes evaluating what optional capabilities in the standards need to be supported, the specific versions of standards that should be used, and returning feedback to the IVOA, to support modifications as needed. We discuss a standard archive model developed by the NAVO for data archive presence in the virtual observatory built upon a consistent framework of standards defined by the IVOA. Our standard model provides for discovery of resources through the VO registries, access to observation and object data, downloads of image and spectral data and general access to archival datasets. It defines specific protocol versions, minimum capabilities, and all dependencies. The model will evolve as the capabilities of the virtual observatory and needs of the community change.

Virtual observatory; data archives; standards; IVO↗
NASA NTRS2018

A Standard Reference Model for Data Archives

An implementable Data Archive Architecture is being developed for trusted digital repositories based on the Reference Model for an Open Archival Information System (OAIS) – ISO 14721. A set of interoperable protocols and interface specifications are planned that will offer capabilities for accessing, merging, and re-using data, both within and across the operational boundaries of trustworthy digital repositories. The model will also provide support for the fundamental scientific need to verify the reproducibility of results. This standards development task is being performed by the Data Archive Interoperability (DAI) working group within the Consultative Committee for Space Data Systems (CCSDS). The architecture integrates concepts from the OAIS Reference Model, the ISO/IEC 11179 Metadata Registry (MDR) standard, the CCSDS Reference Architecture for Space Information Management (RASIM), the proposed draft recommended practice document, Information Preparation to Enable Long Term Use (IPELTU), and three decades of digital repository development for science research.

Ambacher, Bruce↗
NASA NTRS1995

Using and Distributing Spaceflight Data: The Johnson Space Center Life Sciences Data Archive

Life sciences data collected before, during and after spaceflight are valuable and often irreplaceable. The Johnson Space Center Life is hard to find, and much of the data (e.g. Sciences Data Archive has been designed to provide researchers, engineers, managers and educators interactive access to information about and data from human spaceflight experiments. The archive system consists of a Data Acquisition System, Database Management System, CD-ROM Mastering System and Catalog Information System (CIS). The catalog information system is the heart of the archive. The CIS provides detailed experiment descriptions (both written and as QuickTime movies), hardware descriptions, hardware images, documents, and data. An initial evaluation of the archive at a scientific meeting showed that 88% of those who evaluated the catalog want to use the system when completed. The majority of the evaluators found the archive flexible, satisfying and easy to use. We conclude that the data archive effectively provides key life sciences data to interested users.

Cardenas, J. A.↗
NASA NTRS2002

Description of Data Archiving Activities

Data restoration and archiving activities for this project have resulted in the restoration of 100% of the original Mariner 9 raw data set as well as many of the secondary analysis data sets. These data sets have been submitted to the Planetary Data System (PDS) Atmospheric Node, long with their PDS labels and descriptive metadata.

Simmons, K. E.↗
NASA NTRS2014

Ames Life Science Data Archive: Translational Rodent Research at Ames

The Life Science Data Archive (LSDA) office at Ames is responsible for collecting, curating, distributing and maintaining information pertaining to animal and plant experiments conducted in low earth orbit aboard various space vehicles from 1965 to present. The LSDA will soon be archiving data and tissues samples collected on the next generation of commercial vehicles; e.g., SpaceX & Cygnus Commercial Cargo Craft. To date over 375 rodent flight experiments with translational application have been archived by the Ames LSDA office. This knowledge base of fundamental research can be used to understand mechanisms that affect higher organisms in microgravity and help define additional research whose results could lead the way to closing gaps identified by the Human Research Program (HRP). This poster will highlight Ames contribution to the existing knowledge base and how the LSDA can be a resource to help answer the questions surrounding human health in long duration space exploration. In addition, it will illustrate how this body of knowledge was utilized to further our understanding of how space flight affects the human system and the ability to develop countermeasures that negate the deleterious effects of space flight. The Ames Life Sciences Data Archive (ALSDA) includes current descriptions of over 700 experiments conducted aboard the Shuttle, International Space Station (ISS), NASA/MIR, Bion/Cosmos, Gemini, Biosatellites, Apollo, Skylab, Russian Foton, and ground bed rest studies. Research areas cover Behavior and Performance, Bone and Calcium Physiology, Cardiovascular Physiology, Cell and Molecular Biology, Chronobiology, Developmental Biology, Endocrinology, Environmental Monitoring, Gastrointestinal Physiology, Hematology, Immunology, Life Support System, Metabolism and Nutrition, Microbiology, Muscle Physiology, Neurophysiology, Pharmacology, Plant Biology, Pulmonary Physiology, Radiation Biology, Renal, Fluid and Electrolyte Physiology, and Toxicology. These experiment descriptions and data can be accessed online via the public LSDA website (http://lsda.jsc.nasa.gov) and information can be requested via the Data Request form at http://lsda.jsc.nasa.gov/common/dataRequest/dataRequest.aspx or by contacting the ALSDA Office at: Alison.J.French@nasa.gov

Life Sciences↗
NASA NTRS1971

Temporary BOMEX data archive

Environmental data service facility for dissemination of Barbados oceanographic and meteorological sea/air information

Source record↗
NASA NTRS2017

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗
NASA NTRS1984

Vegetation, land-use and seasonal albedo data sets: Documentation of archived data tape

Global data bases of vegetation, land use, and land cover were compiled at a 1 deg latitude x 1 deg longitude resolution, drawing on approximately 100 published sources complemented by a large collection of satellite imagery. Six datasets prepared and archived at NCAR are described: a vegetation data set (VEGTYPE) representing natural (pre-agricultural) vegetation based on the UNESCO classification system; a cultivation intensity data set (CULTINT) defining the areal extent (expressed as %) of presently cultivated land in the 1 x 1 cells; and four integrated surface-albedo data sets (January, April, July, October) for snow-free conditions except for permanently snow-covered continental ice, incorporating natural vegetation and cultivation characteristics from the vegetation and cultivation-intensity data sets. Non-zero data are included for permanent land only, including continental ice. Documentation of the data-tape format as well as descriptions and regional maps of the individual data sets are presented.

Matthews, E.↗
NASA NTRS2007

Defining the Core Archive Data Standards of the International Planetary Data Alliance (IPDA)

A goal of the International Planetary Data Alliance (lPDA) is to develop a set of archive data standards that enable the sharing of scientific data across international agencies and missions. To help achieve this goal, the IPDA steering committee initiated a six month proj ect to write requirements for and draft an information model based on the Planetary Data System (PDS) archive data standards. The project had a special emphasis on data formats. A set of use case scenarios were first developed from which a set of requirements were derived for the IPDA archive data standards. The special emphasis on data formats was addressed by identifying data formats that have been used by PDS nodes and other agencies in the creation of successful data sets for the Planetary Data System (PDS). The dependency of the IPDA information model on the PDS archive standards required the compilation of a formal specification of the archive standards currently in use by the PDS. An ontology modelling tool was chosen to capture the information model from various sources including the Planetary Science Data Dictionary [I] and the PDS Standards Reference [2]. Exports of the modelling information from the tool database were used to produce the information model document using an object-oriented notation for presenting the model. The tool exports can also be used for software development and are directly accessible by semantic web applications.

ontology↗
NASA NTRS1993

The European HST Science Data Archive

The paper describes the European HST Science Data Archive. Particular attention is given to the flow from the HST spacecraft to the Science Data Archive at the Space Telescope European Coordinating Facility (ST-ECF); the archiving system at the ST-ECF, including the hardware and software system structure; the operations at the ST-ECF and differences with the Data Management Facility; and the current developments. A diagram of the logical structure and data flow of the system managing the European HST Science Data Archive is included.

Pasian, F.↗
NASA NTRS2019

NASA's Long-Term Astrophysics Data Archives

NASA regards data handling and archiving as an integral part of space missions, and has a strong track record of serving astrophysics data to the public, be-ginning with the IRAS satellite in 1983. Archives enable a major science return on the significant investment required to develop a space mission. In fact, the presence and accessibility of an archive can more than double the number of papers resulting from the data. In order for the community to be able to use the data, they have to be able to find the data (ease of access) and interpret the data (ease of use). Funding of archival research (e.g., the ADAP program) is also important not only for making scientific progress, but also for encouraging authors to deliver data products back to the archives to be used in future studies. NASA has also enabled a robust system that can be maintained over the long term, through technical innovation and careful attention to resource allocation. This article provides a brief overview of some of NASA's major astrophysics archive systems, including IRSA, MAST, HEASARC, KOA, NED, the Exoplanet Archive, and ADS.

L Rebull↗
NASA NTRS1997

The ASC Data Archive for the AXAF Ground Calibration

A data archive is near completion at the AXAF Science Center (ASC) to store and provide access to AXAF data. The archive is a distributed client/server system. It consists of a number of different servers which handle flat data files, relational data, replication across multiple sites, and the interface to the WWW. There is a 4GL client interface for each type of data server, C++ and Java APIs, and a number of standard clients to archive and retrieve data. The architecture is scalable and configurable in order to accommodate future data types and increasing data volumes. The first release of the system became available in August 1996 and has been successfully operated since then in support of the AXAF calibration at the George C. Marshall Space Flight Center, Huntsville, AL. This paper will present the overall archive architecture and the design of client and server components. It will also review the ground calibration data contents, the plan for flight data, and the access to the data by ASC internal users and processing pipelines.

Zografou, Panagoula↗