Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data archive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

Global data bases on distribution, characteristics and methane emission of natural wetlands: Documentation of archived data tape

Global digital data bases on the distribution and environmental characteristics of natural wetlands, compiled by Matthews and Fung (1987), were archived for public use. These data bases were developed to evaluate the role of wetlands in the annual emission of methane from terrestrial sources. Five global 1 deg latitude by 1 deg longitude arrays are included on the archived tape. The arrays are: (1) wetland data source, (2) wetland type, (3) fractional inundation, (4) vegetation type, and (5) soil type. The first three data bases on wetland locations were published by Matthews and Fung (1987). The last two arrays contain ancillary information about these wetland locations: vegetation type is from the data of Matthews (1983) and soil type from the data of Zobler (1986). Users should consult original publications for complete discussion of the data bases. This short paper is designed only to document the tape, and briefly explain the data sets and their initial application to estimating the annual emission of methane from natural wetlands. Included is information about array characteristics such as dimensions, read formats, record lengths, blocksizes and value ranges, and descriptions and translation tables for the individual data bases.

Matthews, Elaine↗
NASA NTRS2002

Description of Data Archiving Associated With the NASA Goddard Grant NAGS-9590

Data restoration and archiving activities for this project have resulted in the restoration of 100% of the original Mariner 9 raw data set as well as many of the secondary analysis data sets. These data sets have been submitted to the Planetary Data System (PDS) Atmospheric Node, along with their PDS labels and descriptive metadata. In addition, a useful visualization and analysis tool has also been developed which allows the user to compare these Mariner 1971 Ultraviolet spectral data with several choices of related data sets: Mariner 9 images, USGS geologic data, MGS MOLA topography, Viking images (Viking MDIM) and thermal inertia data (MGS TES).

Simmons, K. E.↗
NASA NTRS2018

A Concept of Operations for Earth Science Data Archive and Distribution in the Cloud

Science data systems can enable more comprehensive Earth system research by evolving to take advantage of advances in commercial computer technology services. Since their inception twenty five years ago, NASA's Earth Observing System Data and Information System (EOSDIS) Distributed Active Archive Centers (DAACs) have periodically evolved to utilize new technology and expand research using the exponential growth and diversity of Earth observations. Recently, with the advent of a maturing commercial compute services industry and upcoming high data volume missions such as the Surface Water and Ocean Topography (SWOT) mission and the NASA-Indian Space Research Organization Synthetic Aperture Radar (NISAR) mission, options were explored and a decision made to utilize commercial compute and storage services. This paper presents an overview of the concept of operations under development for the DAACs in the Cloud. We highlight the goals and expected advantages of utilizing Cloud services. We outline EOSDIS operations tenets and driving principles. A high-level view of EOSDIS system of systems target architecture serves as context for describing principle interactions. Concepts for key DAAC system and EOSDIS enterprise functions characterize automated end-to-end operations but mark nominal check and recovery points. Concepts are presented for managing Cloud resources, including organizational roles and responsibilities of the NASA project and DAAC personnel. Scenarios we use to further distinguish between what the system will do and what configuration and controls operators will have. Examples include interactions with data providers and data consumers with both in-cloud and on-premise facilities.

Moses, John F.↗
NASA NTRS1991

Data compression for data archival, browse or quick-look

Soon after space and Earth science data is collected, it is stored in one or more archival facilities for later retrieval and analysis. Since the purpose of the archival process is to keep an accurate and complete record of data, any data compression used in an archival system must be lossless, and protect against propagation of error in the storage media. A browse capability for space and Earth science data is needed to enable scientists to check the appropriateness and quality of particular data sets before obtaining the full data set(s) for detailed analysis. Browse data produced for these purposes could be used to facilitate the retrieval of data from an archival facility. Quick-look data is data obtained directly from the sensor for either previewing the data or for an application that requires very timely analysis of the space or Earth science data. Two main differences between data compression techniques appropriate to browse and quick-look cases, are that quick-look can be more specifically tailored, and it must be limited in complexity by the relatively limited computational power available on space platforms.

Dozier, Jeff↗
NASA NTRS1987

Global digital data sets of soil type, soil texture, surface slope and other properties: Documentation of archived data tape

The file structure and coding of four soils data sets derived from the Zobler (1986) world soil file is described. The data were digitized on a one-degree square grid. They are suitable for large-area studies such as climate research with general circulation models, as well as in forestry, agriculture, soils, and hydrology. The first file is a data set of codes for soil unit, land-ice, or water, for all the one-degree square cells on Earth. The second file is a data set of codes for texture, land-ice, or water, for the same soil units. The third file is a data set of codes for slope, land-ice, or water for the same units. The fourth file is the SOILWRLD data set, containing information on soil properties of land cells of both Matthews' and Food and Agriculture Organization (FAO) sources. The fourth file reconciles land-classification differences between the two and has missing data filled in.

Staub, B.↗
NASA NTRS1992

Subband coding for image data archiving

The use of subband coding on image data is discussed. An overview of subband coding is given. Advantages of subbanding for browsing and progressive resolution are presented. Implementations for lossless and lossy coding are discussed. Algorithm considerations and simple implementations of subband are given.

Glover, D.↗
NASA NTRS1993

Subband coding for image data archiving

The use of subband coding on image data is discussed. An overview of subband coding is given. Advantages of subbanding for browsing and progressive resolution are presented. Implementations for lossless and lossy coding are discussed. Algorithm considerations and simple implementations of subband systems are given.

Glover, Daniel↗
DOE OSTI2026

Model Data Archive Associated with Manuscript "Fire-altered Carbon Pools Create Disturbance Memory in Stream Dissolved Organic Carbon"

This data package supports the publication “Fire-altered Carbon Pools Create Disturbance Memory in Stream Dissolved Organic Carbon” by Li et al. (2026). The package contains processed model inputs, configuration files, restart files, simulation outputs, scripts, and visualization products used to evaluate post-fire dissolved organic carbon (DOC) dynamics in the Naches River Watershed, Washington, USA, following the 2021 Schneider Springs Fire. The modeling workflow couples ELM-BGC, the biogeochemistry-enabled Energy Exascale Earth System Model Land Model; ATS, the Advanced Terrestrial Simulator for integrated surface-subsurface hydrology; and PFLOTRAN, a reactive transport model for multicomponent aqueous geochemistry. Together, these models simulate how wildfire-induced changes in vegetation, litter, coarse woody debris, and soil organic matter influence DOC production, transport, and reaction from burned hillslopes to stream networks. The archive includes preprocessed meteorological, geospatial, hydrologic, and biogeochemical forcing data; ELM-BGC-derived DOC source terms; ATS mesh files; PFLOTRAN reactive-transport inputs; model configuration files; spin-up and transient restart files; watershed-scale diagnostic outputs; stream concentration time series; and figures or visualization files used to inspect and reproduce key results. File types include Hierarchical Data Format 5 (HDF5) files for gridded forcing and model-coupling data, model input and configuration files for ELM-BGC, ATS, and PFLOTRAN, restart and simulation-output files generated by the modeling workflow, tabular or time-series diagnostic outputs, scripts for post-processing and figure generation, and image or visualization products associated with the manuscript. Use of the package depends on the intended task. Re-running the simulations requires the relevant modeling software, including ELM-BGC, ATS, and PFLOTRAN as ATS's geochemical engine. Inspecting outputs and reproducing figures requires Python with scientific plotting libraries such as Matplotlib, and three-dimensional model outputs may be viewed with ParaView. Geographic information system files or maps may be inspected with ArcGIS Pro or comparable GIS software. The data package is intended to enable traceability, reuse, and partial reproduction of the coupled land-to-watershed hydro-biogeochemical modeling workflow used to test how wildfire disturbance affects terrestrial carbon pools and downstream DOC dynamics.

ATS↗
NASA NTRS1982

NASA participation in the 1980 PEPE/NEROS project: Data archive

Eight experimental air quality measurement systems were investigated during July and August 1980 as part of the EPA PEPE/NEROS fiel measurement program. Data from those efforts have been entered into an archive that may be accessed by other researchers. The data sets consists of airborne measurements of regional mixed layer heights and aerosol and ozone distributions as well as point measurements of meteorological parameters and ozone obtained during diurnal transitions in the planetary boundary layer. This report gives a discussion of each measurement system, a preliminary assessment of data quality, a description of the archive format for each data set, and a summary of several proposed scientific studies which will utilize these data.

Brewer, D. A.↗
NASA NTRSDate not supplied

High Performance Access to Archival Data Stored in HDF4 and HDF5 on Cloud Object Stores Without Reformatting the Files

Cloud computing offers numerous advantages for users of extensive Earth science data collections. These benefits encompass direct online access to data files and granules from any location, scalable access supporting parallel computing workflows, and flexible computing tools enabling innovative experimentation with processing techniques. However, older archival file formats designed for distinct computing systems hinder efficient access to decade-long time-series data when compared to data stored in modern cloud-optimized formats like Web Object Stores (WOS), exemplified by Amazon Web Services’ Simple Storage Service (S3). We describe DMR++ (Dataset Metadata Response plus plus), a technology facilitating efficient access to HDF5 (Hierarchical Data Format, version 5) and HDF4 files stored on WOS systems without requiring data reformatting. DMR++ achieves performance comparable to technologies like Zarr while preserving the original file structure, a substantial benefit considering the vast quantity of archival files held by organizations such as NASA. Moreover, DMR++ typically outperforms cloud-optimized versions of HDF5. Essentially an XML (Extensible Markup Language) document usually stored alongside the described data, DMR++ can also be generated on-the-fly but is generally created during data staging to the WOS. Archival files that use HDF4/5 often store large arrays of numerical data. The data in these files is often compressed, typically reducing their size by a factor of four or more. To achieve efficient access to portions of those arrays, they are 'chunked' into smaller sub-arrays, each individually compressed. The chunk size is a compromise, where spinning disks can efficiently access data in smaller chunks while S3 favors larger chunks. A simple optimization of aggregating smaller chunks that are stored adjacently, transferring them in a single access and then individually decompressing them will improve performance. NASA data pose an additional challenge: special Application Programmer Interface (API) libraries are often needed to compute some variables. These libraries are incompatible with WOS environments. Our solution involves storing computed values in the DMR++ document or a companion file, making them accessible like other variables and eliminating the need for specialized APIs. We outline specific optimizations for both satellite grid and swath data stored in HDF4-EOS2 (Earth Observing System).

James Gallagher↗
NASA NTRS2002

Conceptual Study of Intelligent Data Archives of the Future

A conceptual architecture study is under way to address the problem of getting the most scientific value from the large volumes of Earth and space science data that NASA expects to accumulate in the future. This involves efficient storage and access, but goes beyond that to facilitate intelligent data understanding and utilization through modeling realistic virtual entities with predictive capabilities. The objective of the study is to formulate ideas and concepts and to provide recommendations that lead to prototyping and implementation in the period from 2010 to 2020. The approach consists of the definition of future scenarios and needs for data usage in applications (in consultation with scientific and applications users), projection of advances in technologies, and an abstraction of an intelligent archive architecture. Strategic evolution is considered in various areas such as storage, data, information and knowledge management, data ingest and mining, user interfaces, and advances in intelligent data understanding algorithms.

Ramapriyan, H. K.↗
DOE OSTI2026

Model Data Archive for Manuscript Titled "Evaluation of a Coupled Surface–Subsurface Hydrologic Model Using Dense Water‑Level Sensors in a Mixed Urban–Rural Watershed"

This archive provides scripts, input files, and datasets used for the implementation and evaluation of a fully coupled surface–subsurface hydrologic model in the Neches River Basin, southeast Texas. The study uses the Advanced Terrestrial Simulator (ATS) to simulate coupled surface–subsurface hydrologic processes over a mixed urban–rural watershed and evaluates model performance using a dense network of 136 in situ water-level sensors, nine U.S. Geological Survey (USGS) stream gauges, and SSEBop-derived evapotranspiration estimates during the period October 2014–June 2024. The workflow is implemented primarily in Python 3 using the Watershed Workflow package. The Jupyter notebooks can be executed using open-source software such as Anaconda JupyterLab or Visual Studio Code. Other data files include TXT, CSV, XML, SHP, TIF, NetCDF, HDF5, and ExodusII files, which can be processed using the provided Python scripts. ATS input files are provided in XML format and can be edited using any commonly used text editor. This archive contains: *Scripts and input files used to generate the ATS model setup, including watershed discretization, mesh generation, parameter mapping, and model configuration. *Jupyter notebooks used for preprocessing observational data, evaluating streamflow, water levels, and evapotranspiration, computing performance metrics, and generating the figures presented in the manuscript. *ATS simulation outputs and processed observational datasets, including OneRain and DD6 water-level sensors, USGS streamflow observations, GIS data, and supporting spatial datasets used throughout the study.

Dense water-level sensor network↗
NASA NTRS2001

Status of Mars Global Surveyor Science Data Archives

The Mars Global Surveyor has been in orbit around Mars since September 1997, completing its primary mission on January 31, 2001. As of that date the spacecraft had completed more than 8000 mapping orbits. Data from its science instruments, radio science experiment, and SPICE files have been released regularly to the NASA Planetary Data System (PDS) as described in the MGS Archive Plan and Addendum and are available online. Additional information is contained in the original extended abstract.

Slavney, S.↗
NASA NTRS1993

Value added data archiving

Researchers in the Molecular Sciences Research Center (MSRC) of Pacific Northwest Laboratory (PNL) currently generate massive amounts of scientific data. The amount of data that will need to be managed by the turn of the century is expected to increase significantly. Automated tools that support the management, maintenance, and sharing of this data are minimal. Researchers typically manage their own data by physically moving datasets to and from long term storage devices and recording a dataset's historical information in a laboratory notebook. Even though it is not the most efficient use of resources, researchers have tolerated the process. The solution to this problem will evolve over the next three years in three phases. PNL plans to add sophistication to existing multilevel file system (MLFS) software by integrating it with an object database management system (ODBMS). The first phase in the evolution is currently underway. A prototype system of limited scale is being used to gather information that will feed into the next two phases. This paper describes the prototype system, identifies the successes and problems/complications experienced to date, and outlines PNL's long term goals and objectives in providing a permanent solution.

Berard, Peter R.↗
NASA NTRS2024

User’s Guide for the NASA High Efficiency Centrifugal Compressor Data Archive

The datasets contained in this archive are associated with the High Efficiency Centrifugal Compressor (HECC) in the Small Engine Components Compressor Test Facility, colloquially referred to as CE-18, at NASA Glenn Research Center. The archive is accessible at https://storage.googleapis.com/hecc-data/NASA-HECC-Data-Archive.zip. The documentation contained herein provides context for the data hosted on data.nasa.gov. The datasets and accompanying content in this document will be updated periodically as additional data is procured analyzed. The revision of the document is provided by date in the footer, and the revision updates are provided in the Revisions section. Please contact Trey Harrison (email: herbert.harrison@nasa.gov) for inquiries related to the dataset and documentation or to be added to an email list to be notified of updates and additions to the archive.

radial turbomachinery↗
NASA NTRS2017

The Mars Science Laboratory Curiosity Rover Mastcam Instruments: Preflight and In-Flight Calibration, Validation, and Data Archiving

The NASA Curiosity rover Mast Camera (Mastcam) system is a pair of fixed-focal length, multispectral, color CCD imagers mounted approximately 2 m above the surface on the rover's remote sensing mast, along with associated electronics and an onboard calibration target. The left Mastcam (M-34) has a 34 mm focal length, an instantaneous field of view (IFOV) of 0.22 mrad, and a FOV of 20 deg × 15 deg over the full 1648 × 1200 pixel span of its Kodak KAI-2020 CCD. The right Mastcam (M-100) has a 100 mm focal length, an IFOV of 0.074 mrad, and a FOV of 6.8 deg × 5.1 deg using the same detector. The cameras are separated by 24.2 cm on the mast, allowing stereo images to be obtained at the resolution of the M-34 camera. Each camera has an eight-position filter wheel, enabling it to take Bayer pattern red, green, and blue (RGB) 'true color' images, multispectral images in nine additional bands spanning approximately 400-1100 nm, and images of the Sun in two colors through neutral density-coated filters. An associated Digital Electronics Assembly provides command and data interfaces to the rover, 8 Gb of image storage per camera, 11 bit to 8 bit companding, JPEG compression, and acquisition of high-definition video. Here we describe the preflight and in-flight calibration of Mastcam images, the ways that they are being archived in the NASA Planetary Data System, and the ways that calibration refinements are being developed as the investigation progresses on Mars. We also provide some examples of data sets and analyses that help to validate the accuracy and precision of the calibration.

MSL Mastcam; quantitative multispectral↗