Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1986

The CDF software package

The concepts that are fundamental to the new Common Data Format are outlined. With Pilot Climate Data System (PCDS) Version 4.0, the Common Data Format (CDF) will supersede the Climate Data File (also CDF) or earlier versions. This new format incorporates generalizations in both design and terminology that make it applicable to multidisciplinary data sets. Furthermore, the new CDF will be made available to programmers as a software package that shields them from the low-level details of file formats. The CDF interface routines create an abstract conceptual environment for the scientific programmer. The principle concept for visualizing a CDF is known as the basic grid. A basic grid is an n-dimensional block by means of which a CDF is constructed. The size of the block may vary from one CDF to another, but is consistent within any individual CDF. Thus, the basic grid serves as a fundamental uniform building block for a CDF. The number of grid dimensions and the size of each dimension are chosen by the scientist/programmer to represent the patterns by which data are structured. The uniform grid structure appears to the programmer to be propagated into each record for each variable. The CDF stores a variable value for each lattice point of the grid for each record. These data values are inserted and retrieved simply by specifying the variable's identifier, a record number, and the indices that specify the lattice point of interest.

Wilson, C. E.↗
NASA NTRS2000

IMAGE Software Suite

The IMAGE Mission is generating a truely unique set of magnetospheric measurement through a first-of-its-kind complement of remote, global observations. These data are being distributed in the Universal Data Format (UDF), which consists of data, calibration, and documentation. This is an open dataset, available to all by request to the National Space Science Data Center (NSSDC) at NASA Goddard Space Flight Center. Browse data, which consists of summary observations, is also available through the NSSDC in the Common Data Format (CDF) and graphic representations of the browse data. Access to the browse data can be achieved through the NSSDC CDAWeb services or by use of NSSDC provided software tools. This presentation documents the software tools, being provided by the IMAGE team, for use in viewing and analyzing the UDF telemetry data. Like the IMAGE data, these tools are openly available. What these tools can do, how they can be obtained, and how they are expected to evolve will be discussed.

Gallagher, Dennis L.↗
DOE OSTI2021

A Guide to Using GitHub for Developing and Versioning Data Standards and Reporting Formats

Abstract Data standardization combined with descriptive metadata facilitate data reuse, which is the ultimate goal of the Findable, Accessible, Interoperable, and Reusable (FAIR) principles. Community data or metadata standards are increasingly created through an approach that emphasizes collaboration between various stakeholders. Such an approach requires platforms for collaboration on the development process that centers on sharing information and receiving feedback. Our objective in this study was to conduct a systematic review to identify data standards and reporting formats that use version control for developing data standards and to summarize common practices, particularly in earth and environmental sciences. Out of 108 data standards and reporting formats identified in our review, 32 used GitHub as the version control platform, and no other platforms were used. We found no universally accepted methodology for developing and publishing data standards. Many GitHub repositories did not use key features that could help developers to gather user feedback, or to create and revise standards that build on previous work. We provide guidance for community‐driven standard development and associated documentation on GitHub based on a systematic review of existing practices.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2001

New tracking implementation in the Deep Space Network

As part of the Network Simplification Project, the tracking system of the Deep Space Network is being upgraded. This upgrade replaces the discrete logic sequential ranging system with a system that is based on commercial Digital Signal Processor boards. The new implementation allows both sequential and pseudo-noise types of ranging. The other major change is a modernization of the data formatting. Previously, there were several types of interfaces, delivering both intermediate data and processed data (called 'observables'). All of these interfaces were bit-packed blocks, which do not allow for easy expansion, and many of these interfaces required knowledge of the specific hardware implementations. The new interface supports four classes of data: raw (direct from the measuring equipment), derived (the observable data), interferometric (multiple antenna measurements), and filtered (data whose values depend on multiple measurements). All of the measurements are reported at the sky frequency or phase level, so that no knowledge of the actual hardware is required. The data is formatted into Standard Formatted Data Units, as defined by the Consultative Committee for Space Data Systems, so that expansion and cross-center usage is greatly enhanced.

ranging↗
NASA NTRS2007

Defining the Core Archive Data Standards of the International Planetary Data Alliance (IPDA)

A goal of the International Planetary Data Alliance (lPDA) is to develop a set of archive data standards that enable the sharing of scientific data across international agencies and missions. To help achieve this goal, the IPDA steering committee initiated a six month proj ect to write requirements for and draft an information model based on the Planetary Data System (PDS) archive data standards. The project had a special emphasis on data formats. A set of use case scenarios were first developed from which a set of requirements were derived for the IPDA archive data standards. The special emphasis on data formats was addressed by identifying data formats that have been used by PDS nodes and other agencies in the creation of successful data sets for the Planetary Data System (PDS). The dependency of the IPDA information model on the PDS archive standards required the compilation of a formal specification of the archive standards currently in use by the PDS. An ontology modelling tool was chosen to capture the information model from various sources including the Planetary Science Data Dictionary [I] and the PDS Standards Reference [2]. Exports of the modelling information from the tool database were used to produce the information model document using an object-oriented notation for presenting the model. The tool exports can also be used for software development and are directly accessible by semantic web applications.

ontology↗
NASA NTRS2003

Data Access Tools And Services At The Goddard Distributed Active Archive Center (GDAAC)

As one of the largest providers of Earth Science data from the Earth Observing System, GDAAC provides the latest data from the Moderate Resolution Imaging Spectroradiometer (MODIS), Atmospheric Infrared Sounder (AIRS), Solar Radiation and Climate Experiment (SORCE) data products via GDAAC's data pool (50TB of disk cache). In order to make this huge volume of data more accessible to the public and science communities, the GDAAC offers multiple data access tools and services: Open Source Project for Network Data Access Protocol (OPeNDAP), Grid Analysis and Display System (GrADS/DODS) (GDS), Live Access Server (LAS), OpenGlS Web Map Server (WMS) and Near Archive Data Mining (NADM). The objective is to assist users in retrieving electronically a smaller, usable portion of data for further analysis. The OPeNDAP server, formerly known as the Distributed Oceanographic Data System (DODS), allows the user to retrieve data without worrying about the data format. OPeNDAP is capable of server-side subsetting of HDF, HDF-EOS, netCDF, JGOFS, ASCII, DSP, FITS and binary data formats. The GrADS/DODS server is capable of serving the same data formats as OPeNDAP. GDS has an additional feature of server-side analysis. Users can analyze the data on the server there by decreasing the computational load on their client's system. The LAS is a flexible server that allows user to graphically visualize data on the fly, to request different file formats and to compare variables from distributed locations. Users of LAS have options to use other available graphics viewers such as IDL, Matlab or GrADS. WMS is based on the OPeNDAP for serving geospatial information. WMS supports OpenGlS protocol to provide data in GIs-friendly formats for analysis and visualization. NADM is another access to the GDAAC's data pool. NADM gives users the capability to use a browser to upload their C, FORTRAN or IDL algorithms, test the algorithms, and mine data in the data pool. With NADM, the GDAAC provides an environment physically close to the data source. NADM will benefit users with mining or offer data reduction algorithms by reducing large volumes of data before transmission over the network to the user.

Pham, Long↗
NASA NTRS2017

Development of Gridded Innovations and Observations Supplement to MERRA-2

Atmospheric reanalysis have become an important source of data for weather and climate research, owing to the continuity of the data, but especially because of the multitude of observational data included (radiosondes, commercial aircraft, retrieved data products and radiances). However, the presence of assimilated observations can vary based on numerous factors, and so it is difficult or impossible for a researcher to say with any degree of certainty how many and what type of observations contributed to the reanalysis data they are using at any give point in time or space. For example, quality control, transmission interruptions, and station outages can occasionally affect data availability. While orbital paths can be known, drift in certain instruments and the large number of available instruments makes it challenging to know which satellite is observing any region at any point in the diurnal cycle. Furthermore, there is information from the statistics generated by the data assimilation that can help understand the model and the quality of the reanalysis. Typically, the assimilated observations and their innovations are in observation-space data formats and have not been made easily available to reanalysis users.A test data set has been developed to make the MERRA-2 assimilated observations available for rapid and general use, by simplifying the data format. The observations are binned to a grid similar as MERRA-2 and saved as netCDF. This data collection includes the mean and number of observations in the bin as well as its variance. The data will also include the innovations from the data assimilation, the forecast departure and the analysis increment, as well as bias correction (for satellite radiances). We refer to this proof-of-concept data as the MERRA-2 Gridded Innovations and Observations (GIO). In this paper, we present the data format and its strengths and limitations with some initial testing and validation of the methodology.

SSMI↗
DOE OSTI2014

SU-C-BRF-04: Delivered 4D Dose Calculation for Lung-VMAT Patients Using In- Treatment CBCT and LINAC Log Data

Purpose: To verify the delivered four-dimensional (4D) dose distribution for the lung VMAT using in-treatment 4D cone-beam CT (CBCT) and LINAC log data. Methods: Three patients for lung stereotactic radiotherapy (SRT) were treated by single-arc volumetric modulated arc therapy (VMAT) created by SmartArc in Pinnacle v9.2 (Philips, USA).Kilovoltage (kV) projection images were acquired during VMAT delivery using the x-ray volumetric imager, XVI (Elekta, UK). LINAC log data were also recorded via iCom interface (Elekta, UK). Subsequently, the iCom data format was converted to Pinnacle data format, thereby allowing Pinnacle to read actual MLC positions, jaw positions, gantry angles and MUs. These parameters were categorized into four groups according to patient's respiratory phase. The patient respiratory phase was determined from kV-projection images by in-house phase recognizing software. By exporting these data back to Pinnacle, in-treatment dose distributions for each respiratory phase were obtained by combining calculated dose having a particular phase. The 4D CBCT were reconstructed by in-house program. The density in the structures on the CBCT was overridden by the average density in the corresponding structures on the planning CT for the dose calculation. The dose indexes, such as maximum, minimum, mean dose in the target were compared among the plan, in-treatment 3D calculation, and in-treatment 4D calculation. Results: The day-to-day variation of the delivered dose was observed for all three patients. The maximum difference was 5.9% for the minimum dose in the target. Difference for the mean dose was found to be small (1.1% at maximum). Conclusion: We have obtained in-treatment dose distribution for each of the different respiratory phases using in-treatment 4D CBCT and LINAC log data during lung VMAT delivery. Our result indicate a small but significant dose difference between the 3D and 4D calculations for the case of large target traveling.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2018

Development of Gridded Innovations and Observations Supplement to MERRA-2

Atmospheric reanalysis have become an important source of data for weather and climate research, owing to the continuity of the data, but especially because of the multitude of observational data included (radiosondes, commercial aircraft, retrieved data products and radiances). However, the presence of assimilated observations can vary based on numerous factors, and so it is difficult or impossible for a researcher to say with any degree of certainty how many and what type of observations contributed to the reanalysis data they are using at any give point in time or space, or their contribution to the eventual analyzed fields. For example, quality control, transmission interruptions, and station outages can occasionally affect data availability. While orbital paths are known, drift in certain instruments, cloud clearing, and the large number of available instruments makes it challenging to know which satellite is observing any region at any point in the diurnal cycle. Furthermore, there is information from the statistics generated by the data assimilation that can help understand the model and the quality of the reanalysis. Typically, the assimilated observations and their innovations are saved in observation-space data formats and are not easily made available to reanalysis users.Here, we present an early version of a data set has been developed to make the MERRA-2 assimilated observations available for rapid and general use, by simplifying the data format. The observations are binned to a grid similar as MERRA-2 and saved as netCDF. This data collection includes the mean and number of observations in the bin as well as its variance. The data will also include the innovations from the data assimilation, the forecast departure and the analysis increment, as well as bias correction (for satellite radiances). In this paper, we present the data format (called MERRA-2 Gridded Innovations and Observations or GIO) and its strengths and limitations with some initial testing and validation of the methodology.

Bosilovich, Michael G.↗
DOE OSTI2012

Release of the ENDF/B-VII.1 Evaluated Nuclear Data File

The Cross Section Evaluation Working Group (CSEWG) released the ENDF/B-VII.1 library on December 22, 2011. The ENDF/B-VII.1 library is CSEWG's latest recommended evaluated nuclear data file for use in nuclear science and technology applications, and incorporates advances made in the five years since the release of ENDF/B-VII.0, including: many new evaluation in the neutron sublibrary (423 in all and over 190 of these contain covariances), new fission product yields and a greatly improved decay data sublibrary. This summary barely touches on the five years worth of advances present in the ENDF/B-VII.1 library. We expect that these changes will lead to improved integral performance in reactors and other applications. Furthermore, the expansion of covariance data in this release will allow for better uncertainty quantification, reducing design margins and costs. The ENDF library is an ongoing and evolving effort. Currently, the ENDF data community embarking on several parallel efforts to improve library management: (1) The adoption of a continuous integration system to provide evaluators 'instant' feedback on the quality of their evaluations and to provide data users with working 'beta' quality libraries in between major releases. (2) The transition to new hierarchical data format - the Generalized Nuclear Data (GND) format. We expect GND to enable new kinds of evaluated data which cannot be accommodated in the legacy ENDF format. (3) The development of data assimilation and uncertainty propagation techniques to enable the consistent use of integral experimental data in the evaluation process.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2007

Real-Time Data Display

RT-Display is a MATLAB-based data acquisition environment designed to use a variety of commercial off-the-shelf (COTS) hardware to digitize analog signals to a standard data format usable by other post-acquisition data analysis tools. This software presents the acquired data in real time using a variety of signal-processing algorithms. The acquired data is stored in a standard Operator Interactive Signal Processing Software (OISPS) data-formatted file. RT-Display is primarily configured to use the Agilent VXI (or equivalent) data acquisition boards used in such systems as MIDDAS (Multi-channel Integrated Dynamic Data Acquisition System). The software is generalized and deployable in almost any testing environment, without limitations or proprietary configuration for a specific test program or project. With the Agilent hardware configured and in place, users can start the program and, in one step, immediately begin digitizing multiple channels of data. Once the acquisition is completed, data is converted into a common binary format that also can be translated to specific formats used by external analysis software, such as OISPS and PC-Signal (product of AI Signal Research Inc.). RT-Display at the time of this reporting was certified on Agilent hardware capable of acquisition up to 196,608 samples per second. Data signals are presented to the user on-screen simultaneously for 16 channels. Each channel can be viewed individually, with a maximum capability of 160 signal channels (depending on hardware configuration). Current signal presentations include: time data, fast Fourier transforms (FFT), and power spectral density plots (PSD). Additional processing algorithms can be easily incorporated into this environment.

Pedings, Marc↗
NASA NTRS2001

Evaluation of Incremental Releases of ECS User Interfaces and the Development of HDF/HDF-EOS Tutorials

During the reporting period, the PI has continued to serve on numerous review panels, task forces, and committees with the goal of providing input and guidance for the Earth Observing System Data and Information System (EOSDIS) program at NASA Headquarters and NASA Goddard Space Flight Center (GSFC). In addition, the PI has worked together with personnel at Simpson Weather Associates (SWA) to help create an on-line HDF/HDF-EOS tutorial for beginning and non-expert users of both the Hierarchical Data Format (HDF) and HDF-EOS data format and software libraries. Finally, the PI has worked together with personnel at SWA and the Information Technology and Systems Center (ITSC) at the University of Alabama in Huntsville (UAH) on a feasibility study regarding the use of data mining software to ascertain features from the gridded output from numerical meteorological forecast models. A summary of these activities is provided.

Emmitt, G. D.↗
DOE OSTI2016

Sensitivity Data File Formats

The format of the TSUNAMI-A sensitivity data file produced by SAMS for cases with deterministic transport solutions is given in Table 6.3.A.1. The occurrence of each entry in the data file is followed by an identification of the data contained on each line of the file and the FORTRAN edit descriptor denoting the format of each line. A brief description of each line is also presented. A sample of the TSUNAMI-A data file for the Flattop-25 sample problem is provided in Figure 6.3.A.1. Here, only two profiles out of the 130 computed are shown.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2006

Adding Hierarchical Objects to Relational Database General-Purpose XML-Based Information Managements

NETMARK is a flexible, high-throughput software system for managing, storing, and rapid searching of unstructured and semi-structured documents. NETMARK transforms such documents from their original highly complex, constantly changing, heterogeneous data formats into well-structured, common data formats in using Hypertext Markup Language (HTML) and/or Extensible Markup Language (XML). The software implements an object-relational database system that combines the best practices of the relational model utilizing Structured Query Language (SQL) with those of the object-oriented, semantic database model for creating complex data. In particular, NETMARK takes advantage of the Oracle 8i object-relational database model using physical-address data types for very efficient keyword searches of records across both context and content. NETMARK also supports multiple international standards such as WEBDAV for drag-and-drop file management and SOAP for integrated information management using Web services. The document-organization and -searching capabilities afforded by NETMARK are likely to make this software attractive for use in disciplines as diverse as science, auditing, and law enforcement.

Lin, Shu-Chun↗
DOE OSTI2018

Testing of ENDF/B-VIII.0 in the GNDS format with LLNL transport codes - 25606

ENDF/B-VIII.0 was released in the Evaluated Nuclear Data Format and the Generalized Nuclear Data Structure format (GNDS), a new international standard for storing nuclear data developed under the auspices of the Nuclear Energy Agency Working Party on Evaluation Cooperation Subgroup 38. GNDS relies on modern coding practices, and offers a structured, flexible and physics-based standard for storing and exchanging nuclear data to address limitations in legacy nuclear data formats such as ENDF-6. GNDS supports the storage of evaluated nuclear data as well as processed data suitable for both deterministic (i.e., multi-group) and Monte Carlo transport codes. Lawrence Livermore National Laboratory has implemented a complete suite of codes to test GNDS data. Simulations for the ENDF/B-VIII.0 cross-section library in GNDS format are presented. Deterministic and Monte Carlo radiation transport simulations with the Ardra and Mercury codes respectively, are validated against reactivities in fast criticality safety benchmark assemblies. Finally, Mercury simulations of {sup 238}U(n,f), {sup 237}Np(n,f) and {sup 239}Pu(n,f) reaction rates in Godiva, Jezebel and Flattop are compared to MCNP results and measured data. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2016

Methods for Specifying Scientific Data Standards and Modeling Relationships with Applications to Neuroscience [Plus Supplementary Data]

Neuroscience continues to experience a tremendous growth in data; in terms of the volume and variety of data, the velocity at which data is acquired, and in turn the veracity of data. These challenges are a serious impediment to sharing of data, analyses, and tools within and across labs. Here, we introduce BRAINformat, a novel data standardization framework for the design and management of scientific data formats. The BRAINformat library defines application-independent design concepts and modules that together create a general framework for standardization of scientific data. We describe the formal specification of scientific data standards, which facilitates sharing and verification of data and formats. We introduce the concept of Managed Objects, enabling semantic components of data formats to be specified as self-contained units, supporting modular and reusable design of data format components and file storage. We also introduce the novel concept of Relationship Attributes for modeling and use of semantic relationships between data objects. Based on these concepts we demonstrate the application of our framework to design and implement a standard format for electrophysiology data and show how data standardization and relationship-modeling facilitate data analysis and sharing. The format uses HDF5, enabling portable, scalable, and self-describing data storage and integration with modern high-performance computing for data-driven discovery. The BRAINformat library is open source, easy-to-use, and provides detailed user and developer documentation and is freely available at: https://bitbucket.org/oruebel/brainformat.

60 APPLIED LIFE SCIENCES↗
NASA NTRSDate not supplied

Additional Metadata Guidelines to Improve the Structure and Usability of HDF and NetCDF Files

The Hierarchical Data Format (HDF) and Network Common Data Form (NetCDF) are data file formats created to aid users in the creation or use of scientific data. These file formats are useful for handling large data volumes and hosting extensive metadata as global attributes or variables and are popular with the modeling community. HDF and NetCDF files are largely used with remote sensing data and have been used to support measurements from numerous campaigns, from satellite to aircraft or ground and mobile based measurements. The files from airborne field studies, however, vary greatly in terms of the file structure and the amount and content of metadata. Information relevant to the file that can be useful to the user such as the data producer, location where data was taken, variable descriptions, or information about the instrument might not be included in the file. This metadata might be present in another file in the dataset containing the same data using the International Consortium for Atmospheric Research on Transport and Transformation (ICARTT) format. Recently, the Aerosols, Clouds, and their Interactions for Earth System Models (MACIE) group started a grassroots effort to develop a set of requirements for the HDF and NetCDF files for field studies, aiming to make the data products more interoperable and usable. Particularly, these requirements seek to make the files more compliant to Climate and Forecast (CF) metadata conventions and to standardize the file structure and the global and variable attributes. These requirements would help to ensure that HDF and NetCDF files contain adequate metadata to better support their use for research, e.g., the modeling community, and to enhance the usability and interoperability of data for research communities at large. To be presented are the details of the MACIE requirements as well as examples of the implementation of these requirements for merge files and lidar observation data files.

Sean Leavor↗