Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

Verification of technical elements of the advanced spacecraft based upon the CCSDS recommendation

We are going to meet the era when advanced spacecraft such as space stations are developed and operated. The current system of the satellite operations control will need to undergo many changes. We consider that the future system will require the following functions: the function for interchanging data between international agencies, processing the various kinds of space data, and distributing data as many unspecified users require. However, we have to solve the following problems in order to satisfy these requirements: the problem of standardization of space data communication protocol, establishment of multimedia data management method, and standardization of the user interface. This paper describes three techniques to solve the above mentioned problems. That is, standardization of the data communication protocol between space and ground by AOS (Advanced Orbiting System) protocol of CCSDS (Consultative Committee for Space Data Systems) Recommendation, management of multimedia data by catalog reference, standardization of user interface by SFDU(Standard Formatted Data Unit) of CCSDS Recommendation.

Hara, Hideo↗
NASA NTRS1993

The Third NASA Goddard Conference on Mass Storage Systems and Technologies

This report contains copies of nearly all of the technical papers and viewgraphs presented at the Goddard Conference on Mass Storage Systems and Technologies held in October 1993. The conference served as an informational exchange forum for topics primarily relating to the ingestion and management of massive amounts of data and the attendant problems involved. Discussion topics include the necessary use of computers in the solution of today's infinitely complex problems, the need for greatly increased storage densities in both optical and magnetic recording media, currently popular storage media and magnetic media storage risk factors, data archiving standards including a talk on the current status of the IEEE Storage Systems Reference Model (RM). Additional topics addressed System performance, data storage system concepts, communications technologies, data distribution systems, data compression, and error detection and correction.

Kobler, Benjamin↗
NASA NTRS1995

Development and analysis of SCR requirements tables for system scenarios

We describe the use of scenarios to develop and refine requirement tables for parts of the Earth Observing System Data and Information System (EOSDIS). The National Aeronautics and Space Administration (NASA) is developing EOSDIS as part of its Mission-To-Planet-Earth (MTPE) project to accept instrument/platform observation requests from end-user scientists, schedule and perform requested observations of the Earth from space, collect and process the observed data, and distribute data to scientists and archives. Current requirements for the system are managed with tools that allow developers to trace the relationships between requirements and other development artifacts, including other requirements. In addition, the user community (e.g., earth and atmospheric scientists), in conjunction with NASA, has generated scenarios describing the actions of EOSDIS subsystems in response to user requests and other system activities. As part of a research effort in verification and validation techniques, this paper describes our efforts to develop requirements tables from these scenarios for the EOSDIS Core System (ECS). The tables specify event-driven mode transitions based on techniques developed by the Naval Research Lab's (NRL) Software Cost Reduction (SCR) project. The SCR approach has proven effective in specifying requirements for large systems in an unambiguous, terse format that enhance identification of incomplete and inconsistent requirements. We describe development of SCR tables from user scenarios and identify the strengths and weaknesses of our approach in contrast to the requirements tracing approach. We also evaluate the capabilities of both approach to respond to the volatility of requirements in large, complex systems.

Callahan, John R.↗
NASA NTRS2007

Simple, Script-Based Science Processing Archive

The Simple, Scalable, Script-based Science Processing (S4P) Archive (S4PA) is a disk-based archival system for remote sensing data. It is based on the data-driven framework of S4P and is used for data transfer, data preprocessing, metadata generation, data archive, and data distribution. New data are automatically detected by the system. S4P provides services such as data access control, data subscription, metadata publication, data replication, and data recovery. It comprises scripts that control the data flow. The system detects the availability of data on an FTP (file transfer protocol) server, initiates data transfer, preprocesses data if necessary, and archives it on readily available disk drives with FTP and HTTP (Hypertext Transfer Protocol) access, allowing instantaneous data access. There are options for plug-ins for data preprocessing before storage. Publication of metadata to external applications such as the Earth Observing System Clearinghouse (ECHO) is also supported. S4PA includes a graphical user interface for monitoring the system operation and a tool for deploying the system. To ensure reliability, S4P continuously checks stored data for integrity, Further reliability is provided by tape backups of disks made once a disk partition is full and closed. The system is designed for low maintenance, requiring minimal operator oversight.

Lynnes, Christopher↗
DOE OSTI2024

Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing

In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.

Hou, Shiyue↗
DOE OSTI2025

Bayesian Inference for the Seismic Moment Tensor Using Regional Waveforms and Teleseismic- P Polarities with a Data-Derived Distribution of Velocity Models and Source Locations

The largest source of uncertainty in any source inversion is the velocity model used in the transfer function that relates observed ground motion to the seismic moment tensor. However, standard inverse procedure often does not quantify uncertainty in the seismic moment tensor due to error in the Green’s functions from uncertain event location and Earth structure. Here, we incorporate this uncertainty into an estimation of the seismic moment tensor using a data-derived distribution of velocity models based on complementary geophysical data sets, including thickness constraints, velocity profiles, gravity data, surface-wave group velocities, and regional body-wave travel times. The data-derived distribution of velocity models is then used as a prior distribution of Green’s functions for use in Bayesian inference of an unknown seismic moment tensor using regional and teleseismic-P waveforms. The use of multiple data sets is important for gaining resolution to different components of the moment tensor. The combined likelihood is estimated using data-specific error models and the posterior of the seismic moment tensor is estimated and interpreted in terms of the most probable source type.

58 GEOSCIENCES↗
DOE OSTI2026

Solar Forecasting, Net Load Forecasting, and Data-Driven Distributed Solar Visibility Prizes (Final Technical Report)

The American-Made Solar Forecasting Prize, Net Load Forecasting Prize, and Data-Driven Distribution (3D) Solar Visibility Prize is a multimillion-dollar prize competition designed to energize U.S. solar innovation through a series of contests that accelerate the entrepreneurial process from years to months. The activities incentivized by these three prizes will support the governmentwide approach to increase American energy dominance by promoting innovation and early deployment of energy technologies, resulting in wider adoption, which is critical for secure, affordable, and reliable solar energy.

14 SOLAR ENERGY↗
NASA NTRS1984

Solar-terrestrial data access distribution and archiving

It is recommended that a central data catalog and data access network (CDC/DAN) for solar-terrestrial research be established, initially as a NASA pilot program. The system is envisioned to be flexible and to evolve as funds permit, starting from a catalog to an access network for high-resolution data. The report describes the various functional requirements for the CDC/DAN, but does not specify the hardware and software architectures as these are constantly evolving. The importance of a steering committee, working with the CDC/DAN organization, to provide scientific guidelines for the data catalog and for data storage, access, and distribution is also stressed.

Source record↗
NASA NTRS1993

Back Propagation Neural Networks for Predicting Ultimate Strengths of Unidirectional Graphite/Epoxy Tensile Specimens

The research presented herein demonstrates the feasibility of predicting ultimate strengths in simple composite structures through a neural network analysis of their acoustic emission (AE) amplitude distribution data. A series of eleven ASTM D-3039 unidirectional graphite/epoxy tensile samples were loaded to failure to generate the amplitude distributions for this analysis. A back propagation neural network was trained to correlate the AE amplitude distribution signatures generated during the first 25% of loading with the ultimate strengths of the samples. The network was trained using two sets of inputs: (1) the statistical parameters obtained from a Weibull distribution fit of the amplitude distribution data, and (2) the event frequency (amplitude) distribution itself. The neural networks were able to predict ultimate strengths with a worst case error of -8.99% for the Weibull modeled amplitude distribution data and 3.74% when the amplitude distribution itself was used to train the network. The principal reason for the improved prediction capability of the latter technique lies in the ability of the neural network to extract subtle features from within the amplitude distribution.

Walker, James L.↗
DOE OSTI2024

Effectiveness of denoising diffusion probabilistic models for fast and high-fidelity whole-event simulation in high-energy heavy-ion experiments

Artificial intelligence (AI) generative models, such as generative adversarial networks (GANs), variational autoencoders, and normalizing flows, have been widely used and studied as efficient alternatives for traditional scientific simulations. However, they have several drawbacks, including training instability and inability to cover the entire data distribution, especially for regions where data are rare. This is particularly challenging for whole-event, full-detector simulations in high-energy heavy-ion experiments, such as sPHENIX at the Relativistic Heavy Ion Collider and Large Hadron Collider experiments, where thousands of particles are produced per event and interact with the detector. This work investigates the effectiveness of denoising diffusion probabilistic models (DDPMs) as an AI-based generative surrogate model for the sPHENIX experiment that includes the heavy-ion event generation and response of the entire calorimeter stack. DDPM performance in sPHENIX simulation data is compared with a popular rival, GANs. Results show that both DDPMs and GANs can reproduce the data distribution where the examples are abundant (low-to-medium calorimeter energies). Nonetheless, DDPMs significantly outperform GANs, especially in high-energy regions where data are rare. Additionally, DDPMs exhibit superior stability compared to GANs. The results are consistent between both central and peripheral centrality heavy-ion collision events. Moreover, DDPMs offer a substantial speedup of approximately a factor of 100 compared to the traditional Geant4 simulation method.

42 ENGINEERING↗
DOE OSTI2023

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

The ability of recurrent neural networks (RNN) to model nonlinear dynamics of high dimensional process data has enabled data-driven RNN-based fault detection algorithms. Previous studies have focused on detecting faults by identifying the discrepancies in data distribution between the faulty and normal data, as reflected in prediction errors generated by RNN models. However, in industrial processes, variations in data distribution can also result from changes in normal control setpoints and compensatory control adjustments in response to disturbances, making it hard to differentiate between normal and faulty conditions. This paper proposes a fault detection method utilizing a long short-term memory (LSTM) encoder–decoder structure with regularized hidden dynamics and reversible instance normalization (RevIN) to compactly represent high-dimensional measurements for effective monitoring. During training, the hidden states of the model are regularized to form a low-dimensional latent space representation of the original multivariate time series data. As a result, the prediction errors of the latent states can be used to monitor the abnormal dynamic variations, while the reconstruction errors of the measured variables are used to monitor the abnormal static variations. Furthermore, the proposed indices can reflect operating conditions, even when the distribution of test data changes, which helps distinguish faults from normal adjustments and disturbances that controllers can settle. Here, data from numerical simulation and the Tennessee Eastman process are used to illustrate the effectiveness of the proposed fault detection method.

42 ENGINEERING↗
NASA NTRS1994

DE 1 and Viking observations associated with electron conical distributions

Data from the electron detectors on board the Swedish Viking satellite launched during a period of low solar activity and from the Dynamic Explorer (DE) 1 satellite launched during active solar coditions have been examined for the occurrence and location of electron conical distributions and several conclusions can be drawn. First, we note that most of the best examples of electron conics observed by the V-3 experiment onboard Viking occurred in the afternoon sector in the range of magneitc local time 14 hours less than Magnetic Local Time (MLT) less than 18 hours, at midaltitudes in the range 10,000 km less than h less than 13,500 km, with few occurring in the nightside auroral region, a region poorly sampled at altitudes greater than 5000 km. For the Viking data there is an association of electron conics with upper hybrid waves. DE 1 observations made by the high-altitude plasma instrument (HAPI) indicate that electron conics were observed in the midmorning sector and the late evening sector, and as has been reported earlier, the correlation with upper hybird waves was good. The HAPI did not sample the afternoon sector. The electon conics observed on both satellites occurred in the presence of at least a modest (several kilovolts) potential difference beneath the satellite with a maximum energy that was usually, but not always, equal to or greater than the maximum energy of the electron conics. Two independent sets of observations by DE 1 suggest two distinct production mechanisms for electron conics. Examiniation of DE 1 electric field measurements from the plasma wave instrument during the observation of electron conics show simultaneous parallel oscillations in the frequency range of 0.2 Hz less than f less than 0.5 Hz during one and perhaps two of four events examined, and upper hybrid waves were observed on all four events. In addition, recent observations of '90-deg' electron conics associated with auroral kilometric radiation source regions suggest a perpendicular heating mechanism produced by wave-particle interaction. Such distributions may be observed as electron conics at higher altitudes. These results suggest more than one possible source mechanism may be responsible for electron conics.

Menietti, J. D.↗
NASA NTRS1981

The sub-iron to iron ratios and the cosmic ray pathlength distribution

Data on the ratios of nuclei in the 21-25 charge range to iron, from the Danish-French cosmic ray isotope experiment on HEAO 3, are compared with cosmic ray propagation calculations based upon the latest cross sections. The results are incompatible with models in which the path length distribution has a peak at a path length larger than 1 g/sq cm, e.g., those in which there is more than 1 g/sq cm of matter around the cosmic ray source. They are most consistent with models in which the path length distribution includes substantial probabilities of having traversed very little matter.

Ormes, J. F.↗
NASA NTRS1994

Rice cultivation and methane emission: Documentation of distributed geographic data sets

High-resolution global data bases on the geographic and seasonal distribution of rice cultivation and associated methane emission, compiled by Matthews et al., were archived for public use. In addition to the primary data sets identifying location, seasonality, and methane emission from rice cultivation, a series of supporting data sets is included, allowing users not only to replicate the work of Matthews et al. but to investigate alternative cultivation and emission scenarios. The suite of databases provided, at 1 latitude by 1 longitude resolution for the globe, includes (1) locations of rice cultivation, (2) monthly arrays of actively growing rice areas, (3) countries and political subdivisions, and (4) monthly arrays of methane emission from rice cultivation. Ancillary data include (1) a listing, by country, of harvested rice areas and seasonal distribution of crop cycles and (2) country names and codes. Summary tables of zonal/monthly distributions of actively growing rice areas and of methane emissions are presented. Users should consult original publications for complete discussion of the data bases. This short paper is designed only to document formats of the distributed information and briefly describe the contents of the data sets and their initial application to evaluating the role of rice cultivation in the methane budget.

Matthews, Elaine↗
NASA NTRS1994

Constraint based scheduling for the Goddard Space Flight Center distributed Active Archive Center's data archive and distribution system

The Goddard Space Flight Center (GSFC) Distributed Active Archive Center (DAAC) has been operational since October 1, 1993. Its mission is to support the Earth Observing System (EOS) by providing rapid access to EOS data and analysis products, and to test Earth Observing System Data and Information System (EOSDIS) design concepts. One of the challenges is to ensure quick and easy retrieval of any data archived within the DAAC's Data Archive and Distributed System (DADS). Over the 15-year life of EOS project, an estimated several Petabytes (10(exp 15)) of data will be permanently stored. Accessing that amount of information is a formidable task that will require innovative approaches. As a precursor of the full EOS system, the GSFC DAAC with a few Terabits of storage, has implemented a prototype of a constraint-based task and resource scheduler to improve the performance of the DADS. This Honeywell Task and Resource Scheduler (HTRS), developed by Honeywell Technology Center in cooperation the Information Science and Technology Branch/935, the Code X Operations Technology Program, and the GSFC DAAC, makes better use of limited resources, prevents backlog of data, provides information about resources bottlenecks and performance characteristics. The prototype which is developed concurrently with the GSFC Version 0 (V0) DADS, models DADS activities such as ingestion and distribution with priority, precedence, resource requirements (disk and network bandwidth) and temporal constraints. HTRS supports schedule updates, insertions, and retrieval of task information via an Application Program Interface (API). The prototype has demonstrated with a few examples, the substantial advantages of using HTRS over scheduling algorithms such as a First In First Out (FIFO) queue. The kernel scheduling engine for HTRS, called Kronos, has been successfully applied to several other domains such as space shuttle mission scheduling, demand flow manufacturing, and avionics communications scheduling.

Short, Nick, Jr.↗