ADVANCE System Requirements
In this document, we focus mainly on testing libraries written in the legacy ENDF format. As we move to new nuclear data format, we anticipate will need to revisit this document.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
SEARCH · Engineering Papers
Results for “data formats”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems
Scientific applications at exascale generate and analyze massive amounts of data. A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems. As such, parallel I/O, the key technology enables moving data between compute nodes and storage, faces monumental challenges from new applications, memory, and storage architectures considered in the designs of exascale systems. As the storage hierarchy is expanding to include node-local persistent memory, burst buffers, etc., as well as disk-based storage, data movement among these layers must be efficient. Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond. In this paper, we describe new capabilities we have developed in Hierarchical Data Format version 5 (HDF5), the most popular parallel I/O library for scientific applications. HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems. The state-of-the-art features we describe include: Virtual Object Layer (VOL), Data Elevator, asynchronous I/O, full-featured single-writer and multiple-reader (Full SWMR), and parallel querying. In this paper, we introduce these features, their implementations, and the performance and feature benefits to applications and other libraries.
97 MATHEMATICS AND COMPUTING↗
Using Big Data Technologies with Earth Science Data in HDF5: HDF5 Scalable Solutions
HDF5 (Hierarchical Data Format 5) is open-source, high-performance software that consists of an abstract data model, library, and fileformat used for storing and managing extremely large and/or complex data collections. NASA Earth Observing System (EOS) Data and Information Systems use HDF5 as an archival format to store remote sensing data from EOS satellites. HDF5 is also used to store other types of Geoscience and Strophysical data, e.g., seismic data and data from Low-Frequency Array (LOFAR) radio telescopes. Data stored in HDF5 has reached tens of petabytes and is growing at an accelerated rate.With the growing amout of HDF5 Earth Science data to analyze and process, scientists need to adopt big data technologies including new storage paradigms such as cloud and object storage. To run models and perform data analysis they also need to utilizied efficient and diverse ways to access data, from high-performance computing's (HPC) Message Passing Interface (MPI) I/O and deep memory hierarchies (DMH) to non-HPC frameworks such as Apache Hadoop, Spark, and Drill. The HDF Group continually works to enable usage of big data technologies in HDF software.
Knox, Larry↗
Fast Multiresolution Reads of Massive Simulation Datasets
Today’s massively parallel simulation codes can produce output ranging up to many terabytes of data. Utilizing this data to support scientific inquiry requires analysis and visualization, yet the sheer size of the data makes it cumbersome or impossible to read without computational resources similar to the original simulation. We identify two broad classes of problems for reading data and present effective solutions for both. The first class of data reads depends on user requirements and available resources. Tasks such as visualization and user-guided analysis may be accomplished using only a subset of variables with a restricted spatial extent at a reduced resolution. The other class of reads requires full resolution multivariate data to be loaded, for example to restart a simulation. We show that utilizing the hierarchical multiresolution IDX data format enables scalable and efficient serial and parallel read access on a variety of hardware from supercomputers down to portable devices. Here, we demonstrate interactive view-dependent visualization and analysis of massive scientific datasets using low-power commodity hardware, and we compare read performance with other parallel file formats for both full and partial resolution data.
97 MATHEMATICS AND COMPUTING↗
An integrated environmental and health performance quantification model for pre-occupancy phase of buildings in China
To comprehensively pre-evaluate the damages to both the environment and human health due to construction activities in China, this paper presents an integrated building environmental and health performance (EHP) assessment model based on the Building Environmental Performance Analysis System (BEPAS) and the Building Health Impact Analysis System (BHIAS) models and offers a new inventory data estimation method. The new model follows the life cycle assessment (LCA) framework and the inventory analysis step involves bill of quantity (BOQ) data collection, consumption data formation, and environmental profile transformation. The consumption data are derived from engineering drawings and quotas to conduct the assessment before construction for pre-evaluation. The new model classifies building impacts into three safeguard areas: ecosystems, natural resources and human health. Thus, this model considers environmental impacts as well as damage to human wellbeing. The monetization approach, distance-to-target method and panel method are considered as optional weighting approaches. Finally, nine residential buildings of different structural types are taken as case studies to test the operability of the integrated model through application. The results indicate that the new model can effectively pre-evaluate building EHP and the structure type significantly affects the performance of residential buildings.
54 ENVIRONMENTAL SCIENCES↗
dxtbx : the diffraction experiment toolbox
Data formats for recording X-ray diffraction data continue to evolve rapidly to accommodate new detector technologies developed in response to more intense light sources. Processing the data from single-crystal X-ray diffraction experiments therefore requires the ability to read, and correctly interpret, image data and metadata from a variety of instruments employing different experimental representations. Tools that have previously been developed to address this problem have been limited either by a lack of extensibility or by inconsistent treatment of image metadata. The dxtbx software package provides a consistent interface to both image data and experimental models, while supporting a completely generic user-extensible approach to reading the data files. The library is written in a mixture of C++ and Python and is distributed as part of the cctbx under an open-source licence at http://cctbx.sourceforge.net.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
pnnl/PHANTOM
A topology-based anomaly detection tool for temporal varying data. The input is json or CSV formatted data with a timestamp field identified. The algorithm computes vectorizations (using a vectorization config file) of time windows of data and computes a topological measure of anomalousness. The output is a sequence of anomaly scores for each time window
Seppala, Garret↗
Python wrapper library and analysis functions for Geotab Altitude API [SWR-24-77]
This software library serves as a Python wrapper for Geotab's Altitude API. It streamlines querying of the API, converts loosely structured API outputs into a standardized tabular data format, and enables analysis of the resulting data tables. It also includes example notebooks showing how to use the library.
Bruchon, Matthew↗
Data Model to CSS Mappings.
This initial draft document contains formative data model content for select areas of Re-Engineering Phase 2 IDC System. The purpose of this document is to facilitate discussion among the stakeholders. It is not intended as a definitive proposal.
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
Approach guidance for outer planet Pioneer missions
An overall parametric system analysis of the use of an optical approach guidance measurement system (using a V-slit electrooptical 'star-pipper') onboard a Pioneer-type spin-stabilized spacecraft for outer planet missions. Optical measurements of satellites of Saturn and Uranus against the star background are considered. Measurement geometry and spacecraft attitude are evaluated in depth with respect to two Pioneer missions: Saturn-Uranus and Jupiter-Uranus atmospheric entry probe missions with departure from Earth in November/December, 1980. A preliminary evaluation of long-term attitude stability measurements of Pioneer 10 is presented in the Appendix. The major conclusions are: (1) at Saturn flyby, viewing some medium period satellites can provide the best measurement results. (2) At Uranus approach, viewing the planet may give best measurements if early bus-probe separation is required. (3) Final measurement accuracy depends on the data format transmitted to Earth and the applied data interpretation techniques.-
Bejczy, A. K.↗
Satellite on-board processing for earth resources data
Results of a survey of earth resources user applications and their data requirements, earth resources multispectral scanner sensor technology, and preprocessing algorithms for correcting the sensor outputs and for data bulk reduction are presented along with a candidate data format. Computational requirements required to implement the data analysis algorithms are included along with a review of computer architectures and organizations. Computer architectures capable of handling the algorithm computational requirements are suggested and the environmental effects of an on-board processor discussed. By relating performance parameters to the system requirements of each of the user requirements the feasibility of on-board processing is determined for each user. A tradeoff analysis is performed to determine the sensitivity of results to each of the system parameters. Significant results and conclusions are discussed, and recommendations are presented.
Bodenheimer, R. E.↗
Offset compensation for A/D converters
Analog-to-digital (A/D) converter eliminates dc offset in final digitized signal as well as in analog input by using digital feedback for compensation. Circuit could prove useful in data processing applications in which analog-format data are entered at high rates, as in point-of-sale data input systems.
Brokl, S. S.↗
A catalog of jovian decameter-wave radio observations from 1957 - 1978
Data from over 200,000 hours of observation of Jupiter radio emission in the decameter-wavelength band, were collected from 13 observing sites and are available on magnetic tape. Observations were made at 14 fixed frequencies from 5 to 30 MHz. The characteristics of the tape recording technique and the data format are described. The combination of overlapping data from observing sites scattered world-wide lessens the effect of the earth's daily interruption of the ground-received signal. A power spectral analysis of the data shows no evidence of periodicities within the data other than the well-known influences of Jupiter, Io, and the earth. The dependence of the occurrence probability of emission on System 3 longitude and the phase of Io varies smoothly with frequency down to 15 MHz and then appears quite different at 10 MHz. The morphology of the radio sources is both complex and stable for periods of at least months and probably much longer.
Thieman, J. R.↗
Feasibility of modifying the high resolution infrared radiation sounder (HIRS/2) for measuring spectral components of Earth radiation budget
The concept of adding four spectral channels to the 20 channel HIRS/2 instrument for the purpose of determining the origin and profile of radiant existence from the Earth's atmosphere is considered. Methods of addition of three channels at 0.5, 1.0 and 1.6 micron m to the present 0.7 micron m visible channel and an 18-25 micron m channel to the present 19 channels spaced from 3.7 micron m to 15 micron m are addressed. Optical components and physical positions were found that permit inclusion of these added channels with negligible effect on the performance of the present 20 channels. Data format changes permit inclusion of the ERB data in the 288 bits allocated to HIRS for each scan element. A lamp and collimating optic assembly may replace one of the on board radiometric black bodies to provide a reference source for the albedo channels. Some increase in instrument dimensions, weight and power will be required to accommodate the modifications.
Koenig, E. W.↗
Limited Area Coverage/high Resolution Picture Transmission, LAC/HRPT tape conversion processor user's manual
The program, LACSEG, converts LAC/HRPT data tapes to the CSC defined Universal format. The Universal formatted data tape is then processed the normal way by the FAS IMDACS system.
Obrien, S. O.↗
Trends in Planetary Data Analysis. Executive summary of the Planetary Data Workshop
Planetary data include non-imaging remote sensing data, which includes spectrometric, radiometric, and polarimetric remote sensing observations. Also included are in-situ, radio/radar data, and Earth based observation. Also discussed is development of a planetary data system. A catalog to identify observations will be the initial entry point for all levels of users into the data system. There are seven distinct data support services: encyclopedia, data index, data inventory, browse, search, sample, and acquire. Data systems for planetary science users must provide access to data, process, store, and display data. Two standards will be incorporated into the planetary data system: Standard communications protocol and Standard format data unit. The data system configuration must combine a distributed system with those of a centralized system. Fiscal constraints have made prioritization important. Activities include saving previous mission data, planning/cost analysis, and publishing of proceedings.
Evans, N.↗
A 0.15-scale study of configuration effects on the aerodynamic interaction between main rotor and fuselage
Hover and forward flight tests were conducted to investigate the mutual aerodynamic interaction between the main motor and fuselage of a conventional helicopter configuration. A 0.15-scale Model 222 two-bladed teetering rotor was combined with a 0.15-scale model of the NASA Ames 40x80-foot wind tunnel 1500 horsepower test stand fairing. Configuration effects were studied by modifying the fairing to simulate a typical helicopter forebody. Separation distance between rotor and body were also investigated. Rotor and fuselage force and moment as well as pressure data are presented in graphical and tabular format. Data was taken over a range of thrust coefficients from 0.002 to 0.007. In forward flight speed ratio was varied from 0.1 to 0.3 with shaft angle varying from +4 to -12 deg. The data show that the rotors effect on the fuselage may be considerably more important to total aircraft performance than the effect of the fuselage on the rotor.
Trept, Ted↗
A software package for the data-independent management of multidimensional data
The Common Data Format (CDF), a structure which provides true data independence for applications software and has been developed at the National Space Science Data Center, is discussed. The background to the CDF is reviewed, and the CDF is described. The conceptual organization of the CDF is discussed, and a sample CDF structure is shown and described. The implementation of CDF, its status, and its applications are examined.
Treinish, Lloyd A.↗