Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data staging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2015

Earth Science Data Fusion with Event Building Approach

Objectives of the NASA Information And Data System (NAIADS) project are to develop a prototype of a conceptually new middleware framework to modernize and significantly improve efficiency of the Earth Science data fusion, big data processing and analytics. The key components of the NAIADS include: Service Oriented Architecture (SOA) multi-lingual framework, multi-sensor coincident data Predictor, fast into-memory data Staging, multi-sensor data-Event Builder, complete data-Event streaming (a work flow with minimized IO), on-line data processing control and analytics services. The NAIADS project is leveraging CLARA framework, developed in Jefferson Lab, and integrated with the ZeroMQ messaging library. The science services are prototyped and incorporated into the system. Merging the SCIAMACHY Level-1 observations and MODIS/Terra Level-2 (Clouds and Aerosols) data products, and ECMWF re- analysis will be used for NAIADS demonstration and performance tests in compute Cloud and Cluster environments.

Lukashin, C.↗
NASA NTRS1982

An initial model for estimating soybean development stages from spectral data

A model, utilizing a direct relationship between remotely sensed spectral data and soybean development stage, has been proposed. The model is based upon transforming the spectral data in Landsat bands to greenness values over time and relating the area of this curve to soybean development stage. Soybean development stages were estimated from data acquired in 1978 from research plots at the Purdue University Agronomy Farm as well as Landsat data acquired over sample areas of the U.S. Corn Belt in 1978 and 1979. Analysis of spectral data from research plots revealed that the model works well with reasonable variation in planting date, row spacing, and soil background. The R-squared of calculated U.S. observed development stage exceeded 0.91 for all treatment variables. Using Landsat data the calculated U.S. observed development stage gave an R-squared of 0.89 in 1978 and 0.87 in 1979. No difference in the models performance could be detected between early and late planted fields, small and large fields, or high and low yielding fields.

Henderson, K. E.↗
NASA NTRS1991

Classification of posture maintenance data with fuzzy clustering algorithms

Sensory inputs from the visual, vestibular, and proprioreceptive systems are integrated by the central nervous system to maintain postural equilibrium. Sustained exposure to microgravity causes neurosensory adaptation during spaceflight, which results in decreased postural stability until readaptation occurs upon return to the terrestrial environment. Data which simulate sensory inputs under various conditions were collected in conjunction with JSC postural control studies using a Tilt-Translation Device (TTD). The University of West Florida proposed applying the Fuzzy C-Means Clustering (FCM) Algorithms to this data with a view towards identifying various states and stages. Data supplied by NASA/JSC were submitted to the FCM algorithms in an attempt to identify and characterize cluster substructure in a mixed ensemble of pre- and post-adaptational TTD data. Following several unsuccessful trials with FCM using a full 11 dimensional data set, a set of two channels (features) were found to enable FCM to separate pre- from post-adaptational TTD data. The main conclusions are that: (1) FCM seems able to separate pre- from post-TTD subject no. 2 on the one trial that was used, but only in certain subintervals of time; and (2) Channels 2 (right rear transducer force) and 8 (hip sway bar) contain better discrimination information than other supersets and combinations of the data that were tried so far.

Bezdek, James C.↗
NASA NTRS2025

Rolling DICE to Advance Knowledge of Land-Atmosphere Interactions

The Diurnal Land-Atmosphere Coupling Experiment (DICE) aims to explore the complex interactions between the land surface and atmospheric boundary layer, which are generally not well understood and difficult to isolate in models. The project involves over 10 different models, combining expertise from both land surface and atmospheric boundary-layer modelling groups. A simple three-stage methodology is designed to assess land-atmosphere feedbacks. Stage 1: the individual components are assessed in isolation, driven and evaluated against observational data; stage 2: the impact of coupling is investigated; stage 3: the sensitivity of the stand-alone models to variations in driving data is explored. For this initial study, a three-day clear-sky period in the mid-west USA over, an assumed simple, predominantly grass surface was simulated using data from the CASES-99 field campaign. Key conclusions from the study include: (1) the memory of vegetation state within Land Surface Models (LSMs) needs attention; (2) the height of atmospheric forcing for LSMs is important, particularly for the nocturnal boundary layer, and this has implications for both observations and vertical resolution for atmospheric models; (3) land-atmosphere feedbacks reduce errors in simulated surface fluxes at the expense of the accuracy of the variables that the models are designed to simulate (e.g., temperature, humidity and wind speed); (4) problems remain in representing the stable boundary layer in atmospheric models; (5) the mixing of temperature and humidity within the boundary layer may need to be represented separately; (6) differences in daytime profiles of heat, moisture and momentum between models are mainly due to the way the models erode the inversion at the top of the boundary layer, rather than differences in the surface fluxes. Resultant variations in modelled boundary layer heights have a substantial impact on relative humidity and could partially explain variations in coupling strength between models in the GLACE experiment.

Martin Best↗
NASA NTRS1980

The NEEDS Data Base Management and Archival Mass Memory System

A Data Base Management System and an Archival Mass Memory System are being developed that will have a 10 to the 12th bit on-line and a 10 to the 13th off-line storage capacity. The integrated system will accept packetized data from the data staging area at 50 Mbps, create a comprehensive directory, provide for file management, record the data, perform error detection and correction, accept user requests, retrieve the requested data files and provide the data to multiple users at a combined rate of 50 Mbps. Stored and replicated data files will have a bit error rate of less than 10 to the -9th even after ten years of storage. The integrated system will be demonstrated to prove the technology late in 1981.

Bailey, G. A.↗
NASA NTRSDate not supplied

High Performance Access to Archival Data Stored in HDF4 and HDF5 on Cloud Object Stores Without Reformatting the Files

Cloud computing offers numerous advantages for users of extensive Earth science data collections. These benefits encompass direct online access to data files and granules from any location, scalable access supporting parallel computing workflows, and flexible computing tools enabling innovative experimentation with processing techniques. However, older archival file formats designed for distinct computing systems hinder efficient access to decade-long time-series data when compared to data stored in modern cloud-optimized formats like Web Object Stores (WOS), exemplified by Amazon Web Services’ Simple Storage Service (S3). We describe DMR++ (Dataset Metadata Response plus plus), a technology facilitating efficient access to HDF5 (Hierarchical Data Format, version 5) and HDF4 files stored on WOS systems without requiring data reformatting. DMR++ achieves performance comparable to technologies like Zarr while preserving the original file structure, a substantial benefit considering the vast quantity of archival files held by organizations such as NASA. Moreover, DMR++ typically outperforms cloud-optimized versions of HDF5. Essentially an XML (Extensible Markup Language) document usually stored alongside the described data, DMR++ can also be generated on-the-fly but is generally created during data staging to the WOS. Archival files that use HDF4/5 often store large arrays of numerical data. The data in these files is often compressed, typically reducing their size by a factor of four or more. To achieve efficient access to portions of those arrays, they are 'chunked' into smaller sub-arrays, each individually compressed. The chunk size is a compromise, where spinning disks can efficiently access data in smaller chunks while S3 favors larger chunks. A simple optimization of aggregating smaller chunks that are stored adjacently, transferring them in a single access and then individually decompressing them will improve performance. NASA data pose an additional challenge: special Application Programmer Interface (API) libraries are often needed to compute some variables. These libraries are incompatible with WOS environments. Our solution involves storing computed values in the DMR++ document or a companion file, making them accessible like other variables and eliminating the need for specialized APIs. We outline specific optimizations for both satellite grid and swath data stored in HDF4-EOS2 (Earth Observing System).

James Gallagher↗
NASA NTRS1981

Core compressor exit stage study. Volume 4: Data and performance report for the best stage configuration

The core compressor exit stage study program develops rear stage blading designs that have lower losses in their endwall boundary layer regions. The test data and performance results for the best stage configuration consisting of Rotor-B running with Stator-B are described. The technical approach in this efficiency improvement program utilizes a low speed research compressor. Tests were conducted in two ways: (1) to use four identical stages of blading to obtain test data in a true multistage environment and (2) to use a single stage of blading to compare with the multistage test results. The effects of increased rotor tip clearances and circumferential groove casing treatment are evaluated.

Wisler, D. C.↗
NASA NTRS1973

Applications of ERTS-1 Data Collection System (DCS) in the Arizona Regional Ecological Test Site (ARETS)

The author has identified the following significant results. The DCS water-stage data from the USGS streamflow gaging station on the Verde River near Camp Verde furnished information sufficient for the accurate computation of daily mean streamflow rates during the first 2 months of operation. Daily mean flow rates computed from the DCS data agreed with those computed from the digital recorder data within + or - 5% during periods of stable or slowly changing flow and within + or - 10% during periods of rapidly changing high flow. The SRP was furnished near-real time DCS information on snow moisture content and streamflow rates for use in the management and operation of the multiple-use reservoir system. The SRP, by prudent water management and the use of near-real time hydrologic data furnished by microwave and ERTS DCS telemetry, was successful in anticipating the amount of flow into the Salt and Verde Rivers and in the subsequent release of water at rates that did not create flooding in metropolitan Phoenix. Only minor flooding occurred along the Gila River west of Phoenix. According to the Maricopa County Civil Defense agency, wage and salary losses of about $11,400,000 resulted from closing of roads across the Salt River in the winter and spring of 1972-73; however, the number and duration of the closing were minimized by use of DCS data.

Schumann, H. H.↗
NASA NTRS2005

NASA Earth Sciences Data Support System and Services for the Northern Eurasia Earth Science Partnership Initiative

The presentation describes data management of NASA remote sensing data for Northern Eurasia Earth Science Partnership Initiative (NEESPI). Many types of ground and integrative (e.g., satellite, GIs) data will be needed and many models must be applied, adapted or developed for properly understanding the functioning of Northern Eurasia cold and diverse regional system. Mechanisms for obtaining the requisite data sets and models and sharing them among the participating scientists are essential. The proposed project targets integration of remote sensing data from AVHRR, MODIS, and other NASA instruments on board US- satellites (with potential expansion to data from non-US satellites), customized data products from climatology data sets (e.g., ISCCP, ISLSCP) and model data (e.g., NCEPNCAR) into a single, well-architected data management system. It will utilize two existing components developed by the Goddard Earth Sciences Data & Information Services Center (GES DISC) at the NASA Goddard Space Flight Center: (1) online archiving and distribution system, that allows collection, processing and ingest of data from various sources into the online archive, and (2) user-friendly intelligent web-based online visualization and analysis system, also known as Giovanni. The former includes various kinds of data preparation for seamless interoperability between measurements by different instruments. The latter provides convenient access to various geophysical parameters measured in the Northern Eurasia region without any need to learn complicated remote sensing data formats, or retrieve and process large volumes of NASA data. Initial implementation of this data management system will concentrate on atmospheric data and surface data aggregated to coarse resolution to support collaborative environment and climate change studies and modeling, while at later stages, data from NASA and non-NASA satellites at higher resolution will be integrated into the system.

Leptoukh, Gregory↗
NASA NTRS2006

NASA Earth Sciences Data Support System and Services for the Northern Eurasia Earth Science Partnership Initiative

The presentation describes the recently awarded ACCESS project to provide data management of NASA remote sensing data for the Northern Eurasia Earth Science Partnership Initiative (NEESPI). The project targets integration of remote sensing data from MODIS, and other NASA instruments on board US-satellites (with potential expansion to data from non-US satellites), customized data products from climatology data sets (e.g., ISCCP, ISLSCP) and model data (e.g., NCEP/NCAR) into a single, well-architected data management system. It will utilize two existing components developed by the Goddard Earth Sciences Data & Information Services Center (GES DISC) at the NASA Goddard Space Flight Center: (1) online archiving and distribution system, that allows collection, processing and ingest of data from various sources into the online archive, and (2) user-friendly intelligent web-based online visualization and analysis system, also known as Giovanni. The former includes various kinds of data preparation for seamless interoperability between measurements by different instruments. The latter provides convenient access to various geophysical parameters measured in the Northern Eurasia region without any need to learn complicated remote sensing data formats, or retrieve and process large volumes of NASA data. Initial implementation of this data management system will concentrate on atmospheric data and surface data aggregated to coarse resolution to support collaborative environment and climate change studies and modeling, while at later stages, data from NASA and non-NASA satellites at higher resolution will be integrated into the system.

Leptoukh, Gregory↗
NASA NTRSDate not supplied

Transcriptomics Processing Pipelines for Space Biology: An Open Source and Consensus-Driven Approach

Transcriptomics holds significant value in elucidating the relationship between gene expression, experimental factors, biological factors, and various types of omics data. Enhancing our understanding of these connections is paramount for foundational biology, which plays a pivotal role in devising solutions for challenges pertinent to both space travel and terrestrial life. The NASA GeneLab project, part of the Open Science Data Repository (OSDR.nasa.gov), seeks to accelerate space biology research through cataloging and democratizing ‘omics data, including transcriptomics. Since raw omics data are largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community via the Open Science Analysis Working Groups (AWGs) to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data have greater immediate value to diverse users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. As of June 2023, transcriptomics studies comprise over half of GeneLab datasets hosted on the OSDR, including data from bulk RNA-seq and Affymetrix or Agilent 1-Channel DNA microarray assays. In collaboration with the AWGs, GeneLab developed consensus processing pipelines for these transcriptomics data types that includes quality control, background correction (microarray only), data normalization and quantification, culminating in the detection and annotation of differentially expressed genes. The work presented here describes Nextflow implementations of GeneLab’s consensus transcriptomics pipelines that automates and accelerates processing of these datasets. In addition to the core data processing, these workflows also include raw data staging and a robust verification and validation program to identify errors in real-time, stop additional downstream computation, and preserve computational resources. These workflows are used to generate GeneLab processed data hosted on the OSDR, and are publicly available as open source software for others to use at: https://github.com/nasa/GeneLab_Data_Processing.

Jonathan Oribello↗
NASA NTRS1986

HOST turbine heat transfer subproject overview

The experimental part of the turbine heat transfer subproject consists of six large experiments, which are highlighted in this overview, and three of somewhat more modest scope. One of the initial efforts was the stator airfoil heat transfer program. The non-film cooled and the showerhead film cooled data have already been reported. The gill region film cooling effort is currently underway. The investigation of secondary flows in a 90 deg curved duct, was completed. The first phase examined flows with a relatively thin inlet boundary layer and low free stream turbulence. The second phase studied a thicker inlet boundary layer and higher free stream turbulence. A comparison of analytical and experimental cross flow velocity vectors is shown for the 60 deg plane. Two experiments were also conducted in the high pressure facility. One examined full coverage film cooled vanes, and the other, advanced instrumentation. The other three large experimental efforts were conducted in a rotation reference frame. An experiment to obtain gas path airfoil heat transfer coefficients in the large, low speed turbine was completed. Single-stage data with both high and low-inlet turbulence were taken. The second phase examined a one and one-half stage turbine and focused on the second vane row. Under phase 3 aerodynamic quantities such as interrow time-averaged and rms values of velocity, flow angle, inlet turbulence, and surface pressure distribution were measured.

Gladden, Herbert J.↗
NASA NTRS1977

Hardware report for experiment M133, sleep monitoring on Skylab

The purpose of the experiment was objectively to measure sleep quantity and quality during prolonged space flight with the use of automatic equipment for onboard analyses of electroencephalographic (EEG) and electro-oculographic (EOG) activity and telemetry of sleep stage data.

Booher, C. R.↗
NASA NTRS2008

Parallel-Processing Software for Creating Mosaic Images

A computer program implements parallel processing for nearly real-time creation of panoramic mosaics of images of terrain acquired by video cameras on an exploratory robotic vehicle (e.g., a Mars rover). Because the original images are typically acquired at various camera positions and orientations, it is necessary to warp the images into the reference frame of the mosaic before stitching them together to create the mosaic. [Also see "Parallel-Processing Software for Correlating Stereo Images," Software Supplement to NASA Tech Briefs, Vol. 31, No. 9 (September 2007) page 26.] The warping algorithm in this computer program reflects the considerations that (1) for every pixel in the desired final mosaic, a good corresponding point must be found in one or more of the original images and (2) for this purpose, one needs a good mathematical model of the cameras and a good correlation of individual pixels with respect to their positions in three dimensions. The desired mosaic is divided into slices, each of which is assigned to one of a number of central processing units (CPUs) operating simultaneously. The results from the CPUs are gathered and placed into the final mosaic. The time taken to create the mosaic depends upon the number of CPUs, the speed of each CPU, and whether a local or a remote data-staging mechanism is used.

Klimeck, Gerhard↗
NASA NTRS2011

Three-Dimensional Model for Preservation and Restoration of Architectural Heritage

Thc aim of the research will be to create a model, three-dimensional mathematical. implementation. consultation and assistance to "large" restoration projects that will assist the structural analysis, allowing easier display of dynamic strain. analysis and lighting noise. It could also be a valuable tool for decision support. therefore. may simulate several possible scenarios for intervention, This model appears therefore an excellent support for recovering. ordering and monitoring information about materials and data (stage of restoration. photographs. sampling points. results of diagnostic tests, etc.) collected dynamically during the "life" of the cultural heritage. allowing to document its complete history

Marchis, Elena↗
NASA NTRS2013

A Simple GPU-Accelerated Two-Dimensional MUSCL-Hancock Solver for Ideal Magnetohydrodynamics

We describe our experience using NVIDIA's CUDA (Compute Unified Device Architecture) C programming environment to implement a two-dimensional second-order MUSCL-Hancock ideal magnetohydrodynamics (MHD) solver on a GTX 480 Graphics Processing Unit (GPU). Taking a simple approach in which the MHD variables are stored exclusively in the global memory of the GTX 480 and accessed in a cache-friendly manner (without further optimizing memory access by, for example, staging data in the GPU's faster shared memory), we achieved a maximum speed-up of approx. = 126 for a sq 1024 grid relative to the sequential C code running on a single Intel Nehalem (2.8 GHz) core. This speedup is consistent with simple estimates based on the known floating point performance, memory throughput and parallel processing capacity of the GTX 480.

graphics processing units↗
NASA NTRS1974

Hightip-speed, low-loading transonic fan stage. Part 2: Data compilation

Tests were conducted on a high-tip-speed low-loading transonic fan stage to determine the performance and inlet flow distortion tolerance of the design. Test data were recorded for overall and blade element performance with both uniform and distorted inlet flows. A tabular summary of the data and a representative selection of the computer data reduction sheets are presented.

Ware, T. C.↗