Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

Displaying Composite and Archived Soundings in the Advanced Weather Interactive Processing System

In a previous task, the Applied Meteorology Unit (AMU) developed spatial and temporal climatologies of lightning occurrence based on eight atmospheric flow regimes. The AMU created climatological, or composite, soundings of wind speed and direction, temperature, and dew point temperature at four rawinsonde observation stations at Jacksonville, Tampa, Miami, and Cape Canaveral Air Force Station, for each of the eight flow regimes. The composite soundings were delivered to the National Weather Service (NWS) Melbourne (MLB) office for display using the National version of the Skew-T Hodograph analysis and Research Program (NSHARP) software program. The NWS MLB requested the AMU make the composite soundings available for display in the Advanced Weather Interactive Processing System (AWIPS), so they could be overlaid on current observed soundings. This will allow the forecasters to compare the current state of the atmosphere with climatology. This presentation describes how the AMU converted the composite soundings from NSHARP Archive format to Network Common Data Form (NetCDF) format, so that the soundings could be displayed in AWl PS. The NetCDF is a set of data formats, programming interfaces, and software libraries used to read and write scientific data files. In AWIPS, each meteorological data type, such as soundings or surface observations, has a unique NetCDF format. Each format is described by a NetCDF template file. Although NetCDF files are in binary format, they can be converted to a text format called network Common data form Description Language (CDL). A software utility called ncgen is used to create a NetCDF file from a CDL file, while the ncdump utility is used to create a CDL file from a NetCDF file. An AWIPS receives soundings in Binary Universal Form for the Representation of Meteorological data (BUFR) format (http://dss.ucar.edu/docs/formats/bufr/), and then decodes them into NetCDF format. Only two sounding files are generated in AWIPS per day. One file contains all of the soundings received worldwide between 0000 UTC and 1200 UTC, and the other includes all soundings between 1200 UTC and 0000 UTC. In order to add the composite soundings into AWIPS, a procedure was created to configure, or localize, AWIPS. This involved modifying and creating several configuration text files. A unique fourcharacter site identifier was created for each of the 32 soundings so each could be viewed separately. The first three characters were based on the site identifier of the observed sounding, while the last character was based on the flow regime. While researching the localization process for soundings, the AMU discovered a method of archiving soundings so old soundings would not get purged automatically by AWl PS. This method could provide an alternative way of localizing AWl PS for composite soundings. In addition, this would allow forecasters to use archived soundings in AWIPS for case studies. A test sounding file in NetCDF format was written in order to verify the correct format for soundings in AWIPS. After the file was viewed successfully in AWIPS, the AMU wrote a software program in the Tool Command Language/Tool Kit (Tcl/Tk) language to convert the 32 composite soundings from NSHARP Archive to CDL format. The ncgen utility was then used to convert the CDL file to a NetCDF file. The NetCDF file could then be read and displayed in AWIPS.

Barrett, Joe H., III↗
NASA NTRSDate not supplied

ICARTT File Format Enhancements: Supporting FAIRness and Data Discovery of Suborbital Campaign Data

Suborbital campaigns aim to accomplish a wide variety of goals and can include a variety of platforms, instruments, and parameters measured. In 2004, the ICARTT (International Consortium for Atmospheric Research on Transport and Transformation) standards were developed to fulfill data management needs for the ICARTT campaign. The ICARTT file format is text-based and composed of a header with important data description information and the data section. Built on the NASA Ames and GTE data formats, the ICARTT format was created to facilitate data exchange and promote collaborations among the science teams for achieving the ICARTT campaign goals. Due to its success and adaptation for use in many other field campaigns, the ICARTT file format became a NASA standard in 2010 and was amended in January 2017. These changes provided many enhancements, including the requirement for variable standard names. Primarily designed for airborne field studies, ICARTT has been further utilized for ground-based studies. NASA has made a commitment to build an inclusive open science community over the next decade. Open-source science strives to make publicly funded scientific research transparent, inclusive, accessible, and reproducible. The ICARTT format can host metadata that is critical for proper use of the data, particularly for in-situ measurements, and can enhance data discovery and accessibility. However, the required fields are often free text, meaning that the information is human readable, but not machine interpretable. Furthermore, the amount and type of information provided can vary significantly between principal investigators and campaigns. To support FAIR principles and interoperability, enhancements to the ICARTT standards are recommended. Possible recommendations include potential use of controlled and consistent vocabulary for variable standard name and certain common metadata elements; standardizing timestamps for easier data comparisons and analysis; and providing guidance on variable measurement units and how they are reported. Enhancing ICARTT metadata can further streamline the process to make suborbital data more readily available to the data user and improve variable-level metadata. Providing more variable-level metadata can enhance data searching and discovery, supporting NASA’s Open-Source Science Initiative (OSSI).

Megan Buzanowicz↗
NASA NTRS2017

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗
NASA NTRS2003

New Model for Agglutinitic Glass Formation from LSCC Data

Since the return of the first lunar samples it has been well known that glass-welded aggregates (agglutinates) accumulate in lunar soil as the result of multiple processes, many of which are driven by micrometeorite impacts. The proportion of agglutinates increases with increasing exposure to the space environment, and for an individual soil the proportion of agglutinates also increases with decreasing particle size. Detailed chemical and petrographic analyses of a suite of mare soils and their agglutinate constituents prepared by the Lunar Soil Characterization Consortium appeared to confirm the "Fusion of the Finest Fraction" model for agglutinate formation (or F3) proposed by Papike et al. However, recent LSCC data for highland soils are not consistent with the F3 model and alternate models for agglutinate formation must be revisited. Instead, we suggest differential melting of soil species may be more consistent with the full range of soil data to date.

Pieters, C. M.↗
NASA NTRSDate not supplied

Simplifying Analysis of Hierarchical HDF5 and NetCDF4 Files with Xarray-Datatree

NASA’s Earth Observing System Data and Information System (EOSDIS) contains thousands of Earth science datasets from satellites, models, and field campaigns. EOSDIS data are stored in formats that are well supported by the Earth Science community. These formats include the Hierarchical Data Format (HDF), with derivative flavors such as HDF-5 and the Network Common Data Format (NetCDF-4). The HDF specification allows for a directory-like hierarchy within a single file, known as "groups". Observational data and associated metadata within a single file can be distributed amongst multiple internal groups, which can also be nested to multiple levels. Working with datasets that have a group hierarchical structure can be difficult because of the nested structure of groups. Widely used packages, such as xarray, have data models that do not accommodate the hierarchical structure within HDF files, requiring users to traverse the file and open different HDF groups as separate, unrelated objects. Xarray-datatree is a Python package developed to solve the difficulty of traversing HDFs with a hierarchical group structure by creating a tree-like hierarchical data structure in xarray. The tree-like structure allows each group to be accessed once a DataTree object is instantiated. The migration of xarray-datatree into the xarray core library will reduce barriers to accessing Earth science data by eliminating the need to understand and traverse the specific hierarchy of a grouped HDF file.

Eni Awowale↗
DOE OSTI2020

Fracture Shearing in the Eau Claire Formation

This data set consists of Computed Tomographic (CT) data for five sheared Eau Claire Formation core samples with complex and heterogeneous lithology. Each sample was prefractured, housed in a specially adapted core holder, and sheared in incremental fashion. CT scans were taken before and after each shearing event.

Computed Tomography↗
DOE OSTI2026

Stream discharge and temperature data collected within the East and Taylor Watershed, Colorado for the Lawrence Berkeley National Laboratory Watershed Function Science Focus Area (water years 2019 to 2025)

This dataset contains stream discharge and temperature data for water years 2019 to 2025 from the East and Taylor Watersheds in Colorado, United States. This data was collected to understand hydrological processes occurring in the East River and Taylor River Watersheds, Colorado, which is part of the Lawrence Berkeley National Laboratory Watershed Function Scientific Focus Area. Data includes instantaneous observed discharge using salt dilution and acoustic doppler velocimeter techniques, raw pressure transducer downloaded data, sub-hourly temperature as well as corrected water level and associated stream discharge and mean daily values. Notes on water level corrections, rating curve development and metadata provided. A rating curve is the translation of depth to streamflow. The rating curve can be used as a quantitative measure of the “quality of the data.” Data within this dataset is formatted using ESS-DIVE’s Hydrological Monitoring Reporting Format. This data package contains (1) a zip file (Stream_Discharge_Data_WY19-WY25.zip) containing stream discharge and temperature data organized by location; (2) an InstallationMethods file (InstallationMethods.csv) describing metadata about the installation; (3) a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata; (4) a data dictionary (dd.csv) file that contains terms/column_headers used throughout the files along with a definition, units, and data type; (5) a locations metadata file (locations.csv); (6) and a sensor metadata file (sensors.csv). All data files are in non-proprietary formats (csv, png, or pdf formats). Please contact Rosemary Carroll, Curtis Beutler, or Austin Shirley for any support in accessing the files. Update on 2023-05-12: Additional data from WYs 2021 and 2022 were added. Additionally, the dataset was converted using ESS-DIVE’s Hydrological Monitoring Reporting Format. Data files were reformatted to match reporting format guidance, new metadata files were added, and files were converted from excel to CSV. Update on 2025-05-16: Additional data from WYs 2022 (for locations not previously included), 2023, and 2024 were added. An additional descriptive PDF (WFSFA_Streamflow_Hydrograph_Disclaimer.pdf) was added. Metadata files were updated to reflect the addition of new data and locations. Update on 2026-05-18: Additional data from WY 2025 were added, including a new location Upper Trail Creek (TR-TCG2). Metadata files were updated to reflect the addition of new data.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2005

Northern Hemisphere Gullies on Mars: Analysis of Spacecraft Data and Implications for Formation Mechanisms

The origin of geologically recent gullies on Mars has remained controversial since the discovery of these features by Malin and Edgett in 2000. Numerous models have been proposed which invoke various physical processes as well as various agents of erosion to explain the origin of the Martian gullies. Hypotheses to explain the formation of the gullies invoke shallow liquid water aquifers, deep liquid water aquifers, melting ground ice, snowmelt, dry landslides, and carbon dioxide aquifers. We test the validity of such gully formation mechanisms by analyzing data from the Mars Global Surveyor and Mars Odyssey spacecrafts to uncover trends in the dimensional and physical properties of the gullies and their surrounding terrain. A similar study has previously been completed for gullies located in the southern hemisphere of Mars. The work presented here focuses exclusively on gullies in the northern hemisphere based on the identification of 136 Mars Orbiter Camera (MOC) images containing clear evidence of gully landforms, distributed in the northern mid and high latitudes. These sites have been analyzed in combination with Mars Orbiter Laser Altimeter (MOLA), Thermal Emission Spectrometer (TES), and Gamma Ray Spectrometer (GRS) data to provide quantitative measurements of numerous gully characteristics. Parameters measured include apparent source depth and distribution, vertical and horizontal dimensions, slopes, compass orientations, near-surface ice content, and factors controlling present-day climatic conditions.

Heldmann, J. L.↗
NASA NTRS1979

UHF coplanar-slot antenna for aircraft-to-satellite data communications

The initiative for starting the Aircraft-to-Satellite Data Relay (ASDAR) Program came from a recognition that much of the world's weather originates in the data sparse area of the tropics which are primarily ocean. The ASDAR system consists of (1) a data acquisition and control unit to acquire, store and format these data; (2) a clock to time the data sampling and transmission periods; and (3) a transmitter and low-profile upper hemisphere coverage antenna to relay the formatted data via satellite to the National Weather Service ground stations, as shown schematically. The low-profile antenna is a conformal antenna based on the coplanar-slot approach. The antenna is circular polarized and has an on-axis gain of nearly 2.5 dB and a HPBW greater than 90 deg. The discussion covers antenna design, radiation characteristics, flight testing, and system performance.

Myhre, R. W.↗
NASA NTRS1995

Alternate physical formats for storing data in HDF

Since its inception HDF has evolved to meet new demands by the scientific community to support new kinds of data and data structures, larger data sets, and larger numbers of data sets. The first generation of HDF supported simple objects and simple storage schemes. These objects were used to build more complex objects such as raster images and scientific data sets. The second generation of HDF provided alternate methods of storing data elements, making it possible to do such things as store extendible objects with in HDF, to store data externally from HDF files, and support data compression effectively. As we look to the next generation of HDF, we are considering fundamental changes to HDF, including a redefinition of the basic HDF object from a simple object to a more general, higher-level scientific data object that has certain characteristics, such as dimensionality, a more general atomic number type, and attributes. These changes suggest corresponding changes to the HDF file format itself.

Folk, Mike↗
NASA NTRS1988

New data on the formation conditions and age of the Kamensk and Gusev astroblemes

Discussed is new information recently published on the higher contents of siderophilic in the block breccia, fragments of coesite rocks in the Glubokinskiy series within the Kamenskiy astrobleme. New geological data which have refined the concept regarding the structure and age of the Kamenskiy astrobleme are presented.

Movshovich, Ye. V.↗
DOE OSTI2017

The carbon nanotube formation parameter space: data mining and mechanistic understanding for efficient resource use

Carbon nanotube (CNT) synthesis via catalytic chemical vapor deposition is relatively energy consumptive and among the least efficient reactions with respect to carbon conversion efficiency. Interestingly, these processes can be fed using a diverse set of hydrocarbon starting materials, including methane, ethylene, and acetylene, over a wide range of operating temperatures and carbon-to-hydrogen feedstock ratios. Mapping the parameter space for successful CNT growth through data extraction from published literature illuminated the most energy- and material- efficient synthetic pathways in practice to date and provided insights on thermodynamic limitation of CNT growth (i.e., the fundamental mechanisms of CNT formation). Further experimental investigations confirmed that emergent trends in the literature were the result of physicochemical constraints on the process rather than behavioral inertia in the community. The initiation temperatures for CNT growth from acetylene, ethylene, and methane feedstocks via direct experimentation were 550, 700, and 950 °C, respectively, consistent with the trend in literature-extracted mean optima (642 ± 128, 739 ± 82, and 858 ± 125 °C, respectively). These relative temperatures are consistent with a universal CNT growth mechanism, wherein all carbon feedstocks are converted to alkyne-containing species that serve as direct precursors for CNT growth. Mitigating this step with rational carbon precursor delivery, rather than relying on heat to generate the most reactive precursors in situ, could largely reduce the environmental burdens in CNT manufacturing. Indeed, manipulating the starting gas-phase composition and minimizing the thermal treatment through the use of C 2 H 2 increased carbon conversion yield by a factor of more than 10 compared to C 2 H 4 , and, consequently, should minimize hazardous volatile organic compound and polycyclic aromatic hydrocarbon emissions. Finally, the methodology utilized in this study is transferrable to guide the green synthesis of other materials and should be automated in the future for high-throughput screening of the vast process chemistry literature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Basin-Scale Relicensing Opportunity

This dataset contains 1,261 hydropower facilities with associated Federal Energy Regulatory Commission (FERC) relicensing metrics. Anticipated FERC license renewal dates are summarized by river reach for observing areas of opportunities for basin-wide collaboration on environmental mitigations. Identifying rivers and basins with facilities that have similar anticipated Federal Energy Regulatory Commission (FERC) relicensing dates may be one potential indicator that multiple licensees within the same basin may be willing to collaborate on a basin-scale relicensing effort. The hydropower relicensing process requires long-term resource allocation by agencies, hydropower owner/operators, non-governmental organizations, and tribal, state, and federal governments. Basin-scale hydropower relicensing has begun to receive attention as a potential solution for reducing licensing timelines, costs, and uncertainty which can provide benefits to a broad spectrum of participants in the licensing process. This dataset contains one a data file in comma separated (*.csv) format, a data dictionary in *.csv format, and a user guide in *.pdf format. These data are also available as an interactive web map available at:

13 HYDRO ENERGY↗
DOE OSTI2021

ESS-DIVE Reporting Format for Hydrologic Monitoring Data and Metadata

The ESS-DIVE Reporting Format (RF) for Hydrologic Monitoring Data and Metadata was designed for water level, temperature, electrical conductivity, specific conductance, dissolved oxygen, and pH measured in situ using sensors deployed in water bodies. It can also be applied to a broader suite of variables beyond its original scope. The RF is intended to provide clear and easy guidance to data generators aiming to upload consistent findable, accessible, interoperable, and reusable (FAIR) datasets. The RF includes instructions (pdf), a term guide (csv), templates (csv and xlsx), and a list of recommended vocabulary (csv) to equip the data generator with all of the tools needed to successfully apply the RF. There is also a crosswalk to some key data resources and standards (csv). Users should cite this dataset when using the RF but are encouraged to access the RF files on GitHub (https://github.com/ess-dive-community/essdive-hydrologic-monitoring) where the RF documentation is maintained and updated on the ESS-DIVE Community Space GitHub. Questions and suggestions for improvement can be submitted as GitHub issues.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1983

LOX/hydrocarbon fuel carbon formation and mixing data analysis

By applying the Priem-Heidmann Generalized-Length vaporization correlation, the computer model developed by the present study predicts the spatial variation of propellant vaporization rate using the injector cold flow results to define the streamtubes. The calculations show that the overall and local propellant vaporization rate and mixture ratio change drastically as the injection element type or the injector operating condition is changed. These results are compared with the regions of carbon formation observed in the photographic combustion testing. The correlation shows that the fuel vaporization rate and the local mixture ratio produced by the injector element have first order effects on the degree of carbon formation.

Fang, J.↗