SEARCH · Engineering Papers

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

All sources NASA NTRS DOE OSTI

NASA NTRS1998

An Object-Oriented Interface to the CCSDS Ground Telecommand Services

The Telecommand Data Routing and Channel Services defined by the Consultative Committee for Space Data Systems (CCSDS) are flexible enough to support a myriad of commanding models. Because the standard is so broad, the traditional approach has been to implement only the portion of the standard needed by the particular spacecraft being tested/operated. Tasked with providing Telecommand Services for an entire class of spacecraft, where each spacecraft may choose any valid CCSDS commanding model, NASA Code 584 designed a common architecture capable of handling the full CCSDS protocol. The solution uses another CCSDS standard - the Standard Formatted Data Unit (SFDU) as the interface to the Telecommand Services. SFDUs provide a consistent way of labelling data objects, as well as allowing data objects to encapsulate other data objects. The resulting interface is: - Flexible: The full Data Routing and Channel Services are available via a single interface. The client (i.e. the command source) may enter commands at any layer within the protocol stack, specify any of the data aggregation or segmentation methods, and dynamically set any configuration parameter defined in the standard. - Object-oriented: Each object specifies both the data and the actions to be performed with the data. An object may contain other objects. - Expandable: New capabilities are added by defining new objects. Objects pass thru the protocol layers until they reach the applicable layer. The resulting design is: - Modular: The logic for each protocol layer is contained in a separate Application Program Interface (API). The objects used for the external interface are also used for communication between layers. - Distributable: The design can be split along any layer boundary for distribution across multiple machines. The objects ensure data consistency across platforms. This paper describes the SFDU-based interface and the resulting protocol implementation. The implementation is currently used by NASA (National Aeronautics and Space Administration) for integration & test of the microwave Anisotropy Probe (MAP) and Earth observer-I (EO-l) spacecraft. It will be used for post-launch operations of these spacecraft as well as the Imager for Magnetopause to Aurora Global Exploration (IMAGE) spacecraft.

Ray, Timothy Joseph↗

NASA NTRSDate not supplied

Surface Radiation Budget (SRB) Release 2 Shortwave 3 hourly Monthly Data in Native Format (SRB_REL2_SW_3HRLY_MONTHLY)

The SRB data include the average upward and downward fluxes, photosynthetically active radiative flux, aerosol and cloud optical depth, cloud fraction, and solar zenith angle at three hourly intervals for each day for the entire globe between 07/01/1983 and 10/31/1995. These parameters were derived with the Shortwave algorithm of the NASA World Climate Research Programme/Global Energy and Water-Cycle Experiment (WCRP/GEWEX) Surface Radiation Budget (SRB) Project. [Location=GLOBAL] [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=1998-07-26] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=1 degree; Longitude_Resolution=Ranges from 1 degree (tropics and subtropics 120 degrees (the poles); Temporal_Resolution=3 hourly; Temporal_Resolution_Range=3 hourly].

AEROSOL OPTICAL DEPTH↗

NASA NTRSDate not supplied

Surface Radiation Budget (SRB) Release 2.5 Longwave cloud props 3 hourly Data in Native Format (SRB_REL2.5_LW_CLDPROPS_3HRLY)

The Surface Radiation Budget (SRB) data set contain 47 surface and cloud properties including cloud fraction, cloud visible optical depth, cloud top temperature, cloud top pressure, cloud base pressure, and cloud water and ice content for each of 5 cloud types and in total. In addition to these cloud properties, the parameters also include surface pressure, total column ozone, day/night flag, and precipitable water. All perameters were measured at three hourly intervals for each day for the entire globe between 07/01/1983 and 06/30/2005 and serve as inputs to the Global Energy and Water-Cycle Experiment (GEWEX) longwave algorithm (GLW). [Location=GLOBAL] [Temporal_Coverage: Start_Date=1998-01-01; Stop_Date=2005-06-30] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180] [Data_Resolution: Latitude_Resolution=1 degree; Longitude_Resolution=Ranges from 1 degree (tropics and subtropics) to 120 degrees (the poles).; Temporal_Resolution=3 hourly; Temporal_Resolution_Range=3 hourly].

CLOUD TOP TEMPERATURE↗

DOE OSTI2019

TENDL: Complete Nuclear Data Library for Innovative Nuclear Science and Technology

The TENDL library is now established as one of the major nuclear data libraries in the world, striving for completeness and quality of nuclear data files for all isotopes, evaluation methods, processing and applied performance. To reach this status, some basic principles have been applied which sets it apart from other libraries: reproducible dedicated evaluations when differential data are available, through determination of nuclear models implemented in TALYS and their parameters, completeness (with or without experimental data), format and processing standardization, automation of production and reproducibility. In this paper, we will outline how such an approach has become a reality, and recall some of the past successes since the first TENDL release in 2008. Next, we will demonstrate the performance of the latest TENDL releases for different application fields, as well as new approaches for uncertainty quantification based on Bayesian inference methods and possible differential and integral adjustments. Also, current limitations of the library performances due to modelling and needs for new and more precise experimental data will be outlined.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

Improving precision and accuracy of genetic mapping with genotyping-by-sequencing data in outcrossing species

This dataset contains all data and supplementary materials from "Improving precision and accuracy of genetic mapping with genotyping-by-sequencing data in outcrossing species". An Excel file a list of all QTLs and linkage group length (in cM) obtained with two different SNP-calling methods (Tassel-Uneak and Tassel-GBS), genetic map-construction method (linkage-only and reference order-corrected) and depth filters (12x, 20x, 30x and 40x) for genetic mapping of 18 biomass yield traits in a biparental Miscanthus sinensis population using RAD-Seq SNPs is provided as "Supplementary file 1". A Perl script with the code for filtering VCF and HapMap-formatted data files is provided as “Supplementary file 2”. Phenotype data used for QTL mapping is provided as “Supplementary File 3”. A Perl script with the code for the simulation study is provided as “Supplementary file 4”.

GenotypingSimulator↗

DOE OSTI2025

Best Practices for Nuclear Experiment Data Preservation at Idaho National Laboratory: A Guide for Researchers and Reactor Operators

Preserving experimental data is essential for supporting advancements in nuclear science and ensuring the longevity of Idaho National Laboratory's contributions to reactor technology and safety. This report provides a comprehensive guide to best practices for experimental data management and preservation, focusing on standardized data formats, redundancy in storage, metadata documentation, and alignment with international standards. By following these recommendations, experimentalists and reactor operators can enhance the accessibility, reproducibility, and utility of critical datasets for regulatory review, validation computational methods, and future research.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS1976

Remote sensing, land use, and demography - A look at people through their effects on the land

Relevant causes of failure by the remote sensing community in the urban scene are analyzed. The reasons for the insignificant role of remote sensing in urban land use data collection are called the law of realism, the incompatibility of remote sensing and urban management system data formats is termed the law of nominal/ordinal systems compatibility, and the land use/population correlation dilemma is referred to as the law of missing persons. The study summarizes the three laws of urban land use information for which violations, avoidance, or ignorance have caused the decline of present remote sensing research. Particular attention is given to the rationale for urban land use information and for remote sensing. It is shown that remote sensing of urban land uses compatible with the three laws can be effectively developed by realizing the 10 percent contribution of remote sensing to urban land use planning data collection.

Paul, C. K.↗

DOE OSTI2018

CMS Analysis and Data Reduction with Apache Spark

Experimental Particle Physics has been at the forefront of analyzing the world's largest datasets for decades. The HEP community was among the first to develop suitable software and computing tools for this task. In recent times, new toolkits and systems for distributed data processing, collectively called "Big Data" technologies have emerged from industry and open source projects to support the analysis of Petabyte and Exabyte datasets in industry. While the principles of data analysis in HEP have not changed (filtering and transforming experiment-specific data formats), these new technologies use different approaches and tools, promising a fresh look at analysis of very large datasets that could potentially reduce the time-to-physics with increased interactivity. Moreover these new tools are typically actively developed by large communities, often profiting of industry resources, and under open source licensing. These factors result in a boost for adoption and maturity of the tools and for the communities supporting them, at the same time helping in reducing the cost of ownership for the end-users. In this talk, we are presenting studies of using Apache Spark for end user data analysis. We are studying the HEP analysis workflow separated into two thrusts: the reduction of centrally produced experiment datasets and the end-analysis up to the publication plot. Studying the first thrust, CMS is working together with CERN openlab and Intel on the CMS Big Data Reduction Facility. The goal is to reduce 1 PB of official CMS data to 1 TB of ntuple output for analysis. We are presenting the progress of this 2-year project with first results of scaling up Spark-based HEP analysis. Studying the second thrust, we are presenting studies on using Apache Spark for a CMS Dark Matter physics search, comparing Spark's feasibility, usability and performance to the ROOT-based analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2015

A Prescription for List-Mode Data Processing Conventions

There are a variety of algorithmic approaches available to process list-mode pulse streams to produce multiplicity histograms for subsequent analysis. In the development of the INCC v6.0 code to include the processing of this data format, we have noted inconsistencies in the “processed time” between the various approaches. The processed time, tp, is the time interval over which the recorded pulses are analyzed to construct multiplicity histograms. This is the time interval that is used to convert measured counts into count rates. The observed inconsistencies in tp impact the reported count rate information and the determination of the error-values associated with the derived singles, doubles, and triples counting rates. This issue is particularly important in low count-rate environments. In this report we will present a prescription for the processing of list-mode counting data that produces values that are both correct and consistent with traditional shift-register technologies. It is our objective to define conventions for list mode data processing to ensure that the results are physically valid and numerically aligned with the results from shift-register electronics.

List mode↗

DOE OSTI2018

Preparation and upconversion properties of rare earth doped core-shell Y(OH)3 and β-NaYF4 hybrid nanorods

Highlights: • Er{sup 3+}/Yb{sup 3+}-doped Y(OH)3@β-NaYF4@β-NaYF4 hybrid nanorods swere synthesized. • Upconversion intensity increases dramatically by ion exchange reaction, and it can be further enhanced through coating inert shells. • Formation mechanisms of the core-shell nanorods are discussed, as well as their upconversion properties. - Abstract: Er{sup 3+}/Yb{sup 3+}-doped Y(OH){sub 3}@β-NaYF{sub 4} core-shell nanorods are synthesized through in situ ion exchange reaction using Y(OH){sub 3} nanorods as templates. Further, inert shells of β-NaYF{sub 4} without activated dopants are coated by a facile hydrothermal method. Upconversion intensity increases dramatically by ion exchange reaction, and it can be further enhanced through coating inert shells. The maximal enhanced factor of inert coating reaches 6.58. On the basis of the structural tests and spectral data, formation mechanisms of the core-shell nanorods are discussed, as well as their upconversion properties.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

NASA NTRS1997

Building Multi-Discipline, Multi-Format Digital Libraries Using Clusters and Buckets

Our objective was to study the feasibility of extending the Dienst protocol to enable a multi-discipline, multi-format digital library. We implemented two new technologies: cluster functionality and publishing buckets. We have designed a possible implementation of clusters and buckets, and have prototyped some aspects of the resultant digital library. Currently, digital libraries are segregated by the disciplines they serve (computer science, aeronautics, etc.), and by the format of their holdings (reports, software, datasets, etc.). NCSTRL+ is a multi-discipline, multi-format digital library (DL) prototype created to explore the feasibility of the design and implementation issues involved with created a unified, canonical scientific and technical information (STI) DL. NCSTRL+ is based on the Networked Computer Science Technical Report Library (NCSTRL), a World Wide Web (WWW) accessible DL that provides access to over 80 university departments and laboratories. We have extended the Dienst protocol (version 4.1.8), the protocol underlying NCSTRL, to provide the ability to cluster independent collections into a logically centralized DL based upon subject category classification, type of organization, and genre of material. The concept of buckets provides a mechanism for publishing and managing logically linked entities with multiple data formats.

Nelson, Michael L.↗

DOE OSTI2022

MIBiG 3.0: a community-driven effort to annotate experimentally validated biosynthetic gene clusters

Abstract With an ever-increasing amount of (meta)genomic data being deposited in sequence databases, (meta)genome mining for natural product biosynthetic pathways occupies a critical role in the discovery of novel pharmaceutical drugs, crop protection agents and biomaterials. The genes that encode these pathways are often organised into biosynthetic gene clusters (BGCs). In 2015, we defined the Minimum Information about a Biosynthetic Gene cluster (MIBiG): a standardised data format that describes the minimally required information to uniquely characterise a BGC. We simultaneously constructed an accompanying online database of BGCs, which has since been widely used by the community as a reference dataset for BGCs and was expanded to 2021 entries in 2019 (MIBiG 2.0). Here, we describe MIBiG 3.0, a database update comprising large-scale validation and re-annotation of existing entries and 661 new entries. Particular attention was paid to the annotation of compound structures and biological activities, as well as protein domain selectivities. Together, these new features keep the database up-to-date, and will provide new opportunities for the scientific community to use its freely available data, e.g. for the training of new machine learning models to predict sequence-structure-function relationships for diverse natural products. MIBiG 3.0 is accessible online at https://mibig.secondarymetabolites.org/.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

DaYu: Optimizing Distributed Scientific Workflows by Decoding Dataflow Semantics and Dynamics

The combination of ever-growing scientific datasets and distributed workflow complexity creates I/O performance bottlenecks due to data volume, velocity, and variety. Although the increasing use of descriptive data formats (e.g., HDF5, netCDF) helps organize these datasets, it also creates obscure bottlenecks due to the need to translate high level operations into file addresses and then into low-level I/O operations. To address this challenge, we introduce DaYu, a method and toolset for analyzing (a) semantic relationships between logical datasets and file addresses, (b) how dataset operations translate into I/O, and (c) the combination across entire workflows. DaYu's analysis and visualization enables identification of critical bottlenecks and reasoning about remediation. We describe our methodology and propose optimization guidelines. Evaluation on scientific workflows demonstrates up to 3.7x performance improvements in I/O time for obscure bottlenecks. The time and storage overhead for DaYu's time-ordered data is typically under 0.2% of runtime and 0.25% of data volume, respectively.

Tang, Meng↗

DOE OSTI2020

Evaluation of Graph Analytics Frameworks Using the GAP Benchmark Suite

The analysis of connected data is an increasingly important application in high-performance computing. Such analyses can reveal fraudulent patterns in financial transactions, optimize telecommunications networks, predict information flow in social networks, etc. However, the landscape of graph analytics is highly diverse. Graph algorithms stress processor architectures differently, and no one graph can represent all topologies. Consequently, no single approach or framework is expected to be optimal for all graph analytics problems. To help make sense of this diverse landscape, we evaluated four approaches to graph analytics: GraphBLAS, Galois, BGL17, GraphIt; and compare them against hand-tuned implementations that take advantage of hardware features on our test platform. Graph- BLAS formulates graph analytics as sparse linear algebra. Galois provides syntactic constructs for data parallelism over irregular data structures. BGL17 is a generic C++ template library for implementing graph algorithms. GraphIt provides a domain- specific language to describe and optimize graph algorithms. We use the GAP Benchmark Suite to establish baseline performance and guide the side-by-side evaluation of each framework. GAP consists of 30 tests: six graph analytics algorithms (breadth- first search, single-source shortest path, PageRank, betweenness centrality, connected components, and triangle counting) run on five graphs, each with different topological characteristics (e.g., high diameter, skewed degree distribution, high average degree). High-performance reference implementations are included for each benchmark algorithm. Because a graph can be loaded into memory a number of ways (e.g., flat file on disk, compressed sparse format, data frames, retrieved from SQL or NoSQL databases), our evaluation focused on computational performance rather than I/O. Our results show the relative strengths of each framework.

Graph algorithms, Benchmarking, shared-memory prog↗

NASA NTRS1998

Earth Probe Total Ozone Mapping Spectrometer (TOMS) Data Product User's Guide

Two data products from the Earth Probe Total Ozone Mapping Spectrometer (EP/TOMS) have been archived at the Distributed Active Archive Center, in the form of Hierarchical Data Format files. The EP/ TOMS began taking measurements on July 15, 1996. The instrument measures backscattered Earth radiance and incoming solar irradiance; their ratio is used in ozone retrievals. Changes in the reflectivity of the solar diffuser used for the irradiance measurement are monitored using a carousel of three diffusers, each exposed to the degrading effects of solar irradiation at different rates. The algorithm to retrieve total column ozone compares measured Earth radiances at sets of three wavelengths with radiances calculated for different total ozone values. The initial error in the absolute scale for TOMS total ozone is 3 percent, the one standard deviation random error is 2 percent, and the drift is less than 0.5 percent over the first year of data. The Level-2 product contains the measured radiances, the derived total ozone amount, and reflectivity information for each scan position. The Level-3 product contains daily total ozone and reflectivity in a 1-degree latitude by 1.25 degrees longitude grid. Level-3 files containing estimates of LTVB at the Earth surface and tropospheric aerosol information are also available, Detailed descriptions of both HDF data-files and the CD-ROM product are provided.

McPeters, R.↗

NASA NTRS1998

ADEOS Total Ozone Mapping Spectrometer (TOMS) Data Products User's Guide

Two data products from the Total Ozone Mapping Spectrometer (ADEOS/TOMS) have been archived at the Distributed Active Archive Center, in the form of Hierarchical Data Format files. The ADEOS/ TOMS began taking measurements on September 11, 1996, and ended on June 29, 1997. The instrument measured backscattered Earth radiance and incoming solar irradiance; their ratio was used in ozone retrievals. Changes in the reflectivity of the solar diffuser used for the irradiance measurement were monitored using a carousel of three diffusers, each exposed to the degrading effects of solar irradiation at different rates. The algorithm to retrieve total column ozone compares measured Earth radiances at sets of three wavelengths with radiances calculated for different total ozone values, solar zenith angles, and optical paths. The initial error in the absolute scale for TOMS total ozone is 3 percent, the one standard deviation random error is 2 percent, and the drift is less than 0.5 percent over the 9-month data record. The Level 2 product contains the measured radiances, the derived total ozone amount, and reflectivity information for each scan position. The Level 3 product contains daily total ozone and reflectivity in a 1-degree latitude by 1.25 degrees longitude grid. The Level 3 files containing estimates of UVB at the Earth surface and tropospheric aerosol information will also be available. Detailed descriptions of both HDF data files and the CDROM product are provided.

Krueger, A.↗

DOE OSTI2017

VERA-CS Verification & Validation Plan

This report summarizes the current status of VERA-CS Verification and Validation for PWR Core Follow operation and proposes a multi-phase plan for continuing VERA-CS V&V in FY17 and FY18. The proposed plan recognizes the hierarchical nature of a multi-physics code system such as VERA-CS and the importance of first achieving an acceptable level of V&V on each of the single physics codes before focusing on the V&V of the coupled physics solution. The report summarizes the V&V of each of the single physics codes systems currently used for core follow analysis (ie MPACT, CTF, Multigroup Cross Section Generation, and BISON / Fuel Temperature Tables) and proposes specific actions to achieve a uniformly acceptable level of V&V in FY17. The report also recognizes the ongoing development of other codes important for PWR Core Follow (e.g. TIAMAT, MAMBA3D) and proposes Phase II (FY18) VERA-CS V&V activities in which those codes will also reach an acceptable level of V&V. The report then summarizes the current status of VERA-CS multi-physics V&V for PWR Core Follow and the ongoing PWR Core Follow V&V activities for FY17. An automated procedure and output data format is proposed for standardizing the output for core follow calculations and automatically generating tables and figures for the VERA-CS Latex file. A set of acceptance metrics is also proposed for the evaluation and assessment of core follow results that would be used within the script to automatically flag any results which require further analysis or more detailed explanation prior to being added to the VERA-CS validation base. After the Automation Scripts have been completed and tested using BEAVRS, the VERA-CS plan proposes the Watts Bar cycle depletion cases should be performed with the new cross section library and be included in the first draft of the new VERA-CS manual for release at the end of PoR15. Also, within the constraints imposed by the proprietary nature of plant data, as many as possible of the FY17 AMA Plant Core Follow cases should also be included in the VERA-CS manual at the end of PoR15. After completion of the ongoing development of TIAMAT for fully coupled, full core calculations with VERA-CS / BISON 1.5D, and after the completion of the refactoring of MAMBA3D for CIPS analysis in FY17, selected cases from the VERA-CS validation based should be performed, beginning with the legacy cases of Watts Bar and BEAVRS in PoR16. Finally, as potential Phase III future work some additional considerations are identified for extending the VERA-CS V&V to other reactor types such as the BWR.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2019

The DAQ and control system for the CMS Phase-1 pixel detector upgrade

In 2017 a new pixel detector was installed in the CMS detector. This so-called Phase-1 pixel detector features four barrel layers in the central region and three disks per end in the forward regions. The upgraded pixel detector requires an upgraded data acquisition (DAQ) system to accept a new data format and larger event sizes. A new DAQ and control system has been developed based on a combination of custom and commercial microTCA parts. Custom mezzanine cards on standard carrier cards provide a front-end driver for readout, and two types of front-end controller for configuration and the distribution of clock and trigger signals. Before the installation of the detector the DAQ system underwent a series of integration tests, including readout of the pilot pixel detector, which was constructed with prototype Phase-1 electronics and operated in CMS from 2015 to 2016, quality assurance of the CMS Phase-1 detector during its assembly, and testing with the CMS Central DAQ. This paper describes the Phase-1 pixel DAQ and control system, along with the integration tests and results. A description of the operational experience and performance in data taking is included.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗