Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1976

A self-documenting source-independent data format for computer processing of tensor time series

The UCLA Space Science Group has developed a fixed format intermediate data set called a block data set, which is designed to hold multiple segments of multicomponent sampled data series. The format is sufficiently general so that tensor functions of one or more independent variables can be stored in the form of virtual data. This makes it possible for the unit data records of the block data set to be arrays of a single dependent variable rather than discrete samples. The format is self-documenting with parameter, label and header records completely characterizing the contents of the file. The block data set has been applied to the filing of satellite data (of ATS-6 among others).

Mcpherron, R. L.↗
DOE OSTI2021

Closing the Gap between FAIR Data Repositories and Hierarchical Data Formats

Many in the scientific community, particularly in publicly funded research, are pushing to adhere to more accessible data standards to maximize the findability, accessibility, interoperability, and reusability (FAIR) of scientific data, especially with the growing prevalence of machine learning augmented research. Online FAIR data repositories, such as the Open Science Framework (OSF), help facilitate the adoption of these standards by providing frameworks for storage, access, search, APIs, and other features that create organized hubs of scientific data. However, the wider acceptance of such repositories is hindered by the lack of support of hierarchical data formats, such as Technical Data Management Streaming (TDMS) and Hierarchical Data Format 5 (HDF5), that many researchers rely on to organize their datasets. Various tools and strategies should be used to allow hierarchical data formats, FAIR data repositories, and scientific organizations to work more seamlessly together. A pilot project at Los Alamos National Laboratory (LANL) addresses the disconnect between them by integrating the OSF FAIR data repository with hierarchical data renderers, extending support for additional file types in their framework. The multifaceted interactive renderer displays a tree of metadata alongside a table and plot of the data channels in the file. This allows users to quickly and efficiently load large and complex data files directly in the OSF webapp. Users who are browsing files can quickly and intuitively see the files in the way they or their colleagues structured the hierarchical form and immediately grasp their contents. This solution helps bridge the gap between hierarchical data storage techniques and FAIR data repositories, making both of them more viable options for scientific institutions like LANL which have been put off by the lack of integration between them.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

ROOT for the HL-LHC: data format

This document discusses the state, roadmap, and risks of the foundational components of ROOT with respect to the experiments at the HL-LHC (Run 4 and beyond). As foundational components, the document considers in particular the ROOT input/output (I/O) subsystem. The current HEP I/O is based on the TFile container file format and the TTree binary event data format. The work going into the new RNTuple event data format aims at superseding TTree, to make RNTuple the production ROOT event data I/O that meets the requirements of Run 4 and beyond.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

The LLNL nuclear data infrastructure for the GNDS data format

The next generation of nuclear data infrastructure tools at the Livermore National Laboratory (LLNL) consists of pipeline of codes that read and process nuclear data from evaluated files saved in the new GNDS (Generalised Nuclear Data Structure) nuclear data format. The processing code FUDGE (For Updating Data and Generating Evaluations) is at the front-end of this pipeline as it reads and process the evaluated data for use in downstream transport codes. FUDGE is Python based with C and C++ extensions for computationally intensive tasks. As is the case for the evaluated data, the processed output is also saved in the GNDS format and the GIDI+ API is provided as the interface between the processed data and the transport codes. GIDI+ is a C++ based suite of codes and it includes GIDI (General Interaction Data Interface), a library for reading and writing GNDS data, and MCGIDI which is the cross section lookup, and reaction and product distribution sampling interface between Monte Carlo transport codes and the GNDS data. GIDI provides methods for easy access to the multi-group processed GNDS data and this is demonstrated through its implementation in ARDRA, the LLNL deterministic transport code. The evaluation and sampling methods in MCGIDI are available as both CPU and GPU methods which facilitates the use of MCGIDI in both traditional CPU-based as well as the next generation mixed model computational architectures. This is demonstrated through the GIDI+ implementation in MERCURY, the LLNL Monte Carlo transport code. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Validation of standardized data formats and tools for ground-level particle-based gamma-ray observatories

Context. Ground-based γ-ray astronomy is still a rather young field of research, with strong historical connections to particle physics. This is why most observations are conducted by experiments with proprietary data and analysis software, as is usual in the particle physics field. However, in recent years, this paradigm has been slowly shifting toward the development and use of open-source data formats and tools, driven by upcoming observatories such as the Cherenkov Telescope Array (CTA). In this context, a community-driven, shared data format (the gamma-astro-data-format, or GADF) and analysis tools such as Gammapy and ctools have been developed. So far, these efforts have been led by the Imaging Atmospheric Cherenkov Telescope community, leaving out other types of ground-based γ-ray instruments. Aims. We aim to show that the data from ground particle arrays, such as the High-Altitude Water Cherenkov (HAWC) observatory, are also compatible with the GADF and can thus be fully analyzed using the related tools, in this case, Gammapy. Methods. We reproduced several published HAWC results using Gammapy and data products compliant with GADF standard. We also illustrate the capabilities of the shared format and tools by producing a joint fit of the Crab spectrum including data from six different γ-ray experiments. Results. We find excellent agreement with the reference results, a powerful confirmation of both the published results and the tools involved. Conclusions. The data from particle detector arrays such as the HAWC observatory can be adapted to the GADF and thus analyzed with Gammapy. A common data format and shared analysis tools allow multi-instrument joint analysis and effective data sharing. To emphasize this, a sample of Crab nebula event lists is made public with this paper. Because of the complementary nature of pointing and wide-field instruments, this synergy will be distinctly beneficial for the joint scientific exploitation of future observatories such as the Southern Wide-field Gamma-ray Observatory and CTA.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1991

Standard formatted data units-control authority procedures

The purpose of this document is to establish a set of minimum and optional requirements for the implementation of Control Authority (CA) organizations within and among the Agencies participating in the Consultative Committee for Space Data Systems (CCSDS). By satisfying these requirements, the resultant cooperating set of CA organizations will produce a global CA service supporting information transfer with digital data under the Standard Formatted Data Unit (SFDU) concept. This service is primarily accomplished through the registration, permanent archiving, and dissemination of metadata in the form of Metadata Objects (MDO) that assist in the interpretation of data objects received in SFDU form. This Recommendation addresses the responsibilities, services, and interface protocols for a hierarchy of CA organizations. The top level, consisting of the CCSDS Secretariat and its operational agent, is unique and primarily provides a global coordination function. The lower levels are Agency CA organizations that have primary responsibility for the registration, archiving, and dissemination of MDOs. As experience is gained and technology evolves, the CA Procedures will be extended to include enhanced services and their supporting protocols. In particular, it is anticipated that eventually CA organizations will be linked via networks on a global basis, and will provide requestors with online automated access to CA services. While this Recommendation does not preclude such operations, it also does not recommend the specific protocols to be used to ensure global compatibility of these services. These recommendations will be generated as experience is gained.

Source record↗
NASA NTRS1976

Earth observation image data format

A flexible format for computer compatable tape (CCT) containing multispectral earth observation sensor data is described. The driving functions which comprise the data format requirements are summarized and general data format guidelines are discussed.

Sos, J. Y.↗
NASA NTRS1983

Standard format data units - Tools for automatic exchange of space mission data

A set of standard formatting rules for the data sets, and a standard computer-readable language with which to describe the data, are two tools which are used to create the Standard Format Data Unit (SFDU). The NASA/JPL proposal for creation and utilization of SFDUs is presented, and its relationship to recommendations from the Consultative Committee for Space Data Systems (CCSDS) is discussed. Several current and planned implementations of the SFDU concept among major space flight projects are identified. The purpose of creating the concept of an SFDU is to allow members of the science community to share national and global resource data independently of project or program. The feedback from SFDU implementation efforts is considered an essential part of the CCSDS activity. Even though the CCSDS specifically deals with space data systems, the SFDU concept can be applied to practically every data system on an open network. The SFDU is in the early phase of CCSDS standard definition work, and must go through several other phases before being formally recommended as an international standard.

Willett, J. B.↗
NASA NTRS2018

Hierarchical Data Format (HDF) Status Update

In the "Hierarchical Data Format (HDF) Status Update" talk we will give an update on the recent and upcoming HDF5 software releases, and how to upgrade HDF5-based software to the new major releases HDF5 1.10.* to assure backward and forward compatibility for the files produces with the newest versions of the HDF5 software. We will also focus on a compression feature of HDF5 and will talk about new mechanism for storing HDF5 data in Object Store.

Virtual File Driver↗
NASA NTRS2005

Data Format Standardization of Space Weather Model Output at the Community Coordinated Modeling Center

The disparate nature of space weather model output provides many challenges with regards to the portability and reuse of not only the data itself, but also any tools that are developed for analysis and visualization. We are developing and implementing a comprehensive data format standardization methodology that allows heterogeneous model output data to be stored uniformly in any common science data format. We will discuss our approach to identifying core meta-data elements that can be used to supplement raw model output data, thus creating self-descriptive files. The meta-data should also contain information describing the simulation grid. This will ultimately assists in the development of efficient data access tools capable of extracting data at any given point and time. We will also discuss our experiences standardizing the output of two global magnetospheric models, and how we plan to apply similar procedures when standardizing the output of the solar, heliospheric, and ionospheric models that are also currently hosted at the Community Coordinated Modeling Center.

Maddox, M.↗
NASA NTRS2005

Data Format Classification for Autonomous Software Defined Radios

We present maximum-likelihood (ML) coherent and noncoherent classifiers for discriminating between NRZ and Manchester coded (biphase-L) data formats for binary phase-shift-keying (BPSK) modulation. Such classification of the data format is an essential element of so-called autonomous software defined radio (SDR) receivers (similar to so-called cognitive SDR receivers in the military application) where it is desired that the receiver perform each of its functions by extracting the appropriate knowledge from the received signal and, if possible, with as little information of the other signal parameters as possible. Small and large SNR approximations to the ML classifiers are also proposed that lead to simpler implementation with comparable performance in their respective SNR regions. Numerical performance results obtained by a combination of computer simulation and, wherever possible, theoretical analyses, are presented and comparisons are made among the various configurations based on the probability of misclassification as a performance criterion. Extensions to other modulations such as QPSK are readily accomplished using the same methods described in the paper.

classification↗
NASA NTRS2008

Customer-oriented Data Formats and Services for Global Land Data Assimilation System (GLDAS) Products at the NASA GES DISC

The Global Land Data Assimilation System (GLDAS) is generating a series of land surface state (e.g., soil moisture and surface temperature) and flux (e.g., evaporation and sensible heat flux) products simulated by four land surface Models (CLM, Mosaic, Noah and VIC). These products are now accessible at the Hydrology Data and Information Services Center (HDISC), a component of NASA Goddard Earth Sciences Data and Information Services Center (GESDISC).

Fang, Hongliang↗
DOE OSTI2020

The SONATA data format for efficient description of large-scale network models

Increasing availability of comprehensive experimental datasets and of high-performance computing resources are driving rapid growth in scale, complexity, and biological realism of computational models in neuroscience. To support construction and simulation, as well as sharing of such large-scale models, a broadly applicable, flexible, and high-performance data format is necessary. To address this need, we have developed the Scalable Open Network Architecture TemplAte (SONATA) data format. It is designed for memory and computational efficiency and works across multiple platforms. The format represents neuronal circuits and simulation inputs and outputs via standardized files and provides much flexibility for adding new conventions or extensions. SONATA is used in multiple modeling and visualization tools, and we also provide reference Application Programming Interfaces and model examples to catalyze further adoption. SONATA format is free and open for the community to use and build upon with the goal of enabling efficient model building, sharing, and reproducibility.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1987

A synchronous data analyzer for the Minimum Delay Data Format (MDDF) and Launch Trajectory Acquisition System (LTAS)

The Synchronous Data Analyzer gives its user an independent source for checking the validity of raw MDDF and LTAS data. Its ability to produce real-time displays comprised of decoded parameters as well as its ability to capture several frames for non-real-time display makes it a valuable tool for monitoring and troubleshooting applications involving these data formats. Its composition of portable firmware and mostly commercial hardware makes it a low cost, easily fabricated product.

Green, Andrew J.↗