Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data distributions”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing

In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.

Hou, Shiyue↗
DOE OSTI2025

Bayesian Inference for the Seismic Moment Tensor Using Regional Waveforms and Teleseismic- P Polarities with a Data-Derived Distribution of Velocity Models and Source Locations

The largest source of uncertainty in any source inversion is the velocity model used in the transfer function that relates observed ground motion to the seismic moment tensor. However, standard inverse procedure often does not quantify uncertainty in the seismic moment tensor due to error in the Green’s functions from uncertain event location and Earth structure. Here, we incorporate this uncertainty into an estimation of the seismic moment tensor using a data-derived distribution of velocity models based on complementary geophysical data sets, including thickness constraints, velocity profiles, gravity data, surface-wave group velocities, and regional body-wave travel times. The data-derived distribution of velocity models is then used as a prior distribution of Green’s functions for use in Bayesian inference of an unknown seismic moment tensor using regional and teleseismic-P waveforms. The use of multiple data sets is important for gaining resolution to different components of the moment tensor. The combined likelihood is estimated using data-specific error models and the posterior of the seismic moment tensor is estimated and interpreted in terms of the most probable source type.

58 GEOSCIENCES↗
DOE OSTI2026

Solar Forecasting, Net Load Forecasting, and Data-Driven Distributed Solar Visibility Prizes (Final Technical Report)

The American-Made Solar Forecasting Prize, Net Load Forecasting Prize, and Data-Driven Distribution (3D) Solar Visibility Prize is a multimillion-dollar prize competition designed to energize U.S. solar innovation through a series of contests that accelerate the entrepreneurial process from years to months. The activities incentivized by these three prizes will support the governmentwide approach to increase American energy dominance by promoting innovation and early deployment of energy technologies, resulting in wider adoption, which is critical for secure, affordable, and reliable solar energy.

14 SOLAR ENERGY↗
NASA NTRS1984

Solar-terrestrial data access distribution and archiving

It is recommended that a central data catalog and data access network (CDC/DAN) for solar-terrestrial research be established, initially as a NASA pilot program. The system is envisioned to be flexible and to evolve as funds permit, starting from a catalog to an access network for high-resolution data. The report describes the various functional requirements for the CDC/DAN, but does not specify the hardware and software architectures as these are constantly evolving. The importance of a steering committee, working with the CDC/DAN organization, to provide scientific guidelines for the data catalog and for data storage, access, and distribution is also stressed.

Source record↗
NASA NTRS1993

Back Propagation Neural Networks for Predicting Ultimate Strengths of Unidirectional Graphite/Epoxy Tensile Specimens

The research presented herein demonstrates the feasibility of predicting ultimate strengths in simple composite structures through a neural network analysis of their acoustic emission (AE) amplitude distribution data. A series of eleven ASTM D-3039 unidirectional graphite/epoxy tensile samples were loaded to failure to generate the amplitude distributions for this analysis. A back propagation neural network was trained to correlate the AE amplitude distribution signatures generated during the first 25% of loading with the ultimate strengths of the samples. The network was trained using two sets of inputs: (1) the statistical parameters obtained from a Weibull distribution fit of the amplitude distribution data, and (2) the event frequency (amplitude) distribution itself. The neural networks were able to predict ultimate strengths with a worst case error of -8.99% for the Weibull modeled amplitude distribution data and 3.74% when the amplitude distribution itself was used to train the network. The principal reason for the improved prediction capability of the latter technique lies in the ability of the neural network to extract subtle features from within the amplitude distribution.

Walker, James L.↗
DOE OSTI2024

Effectiveness of denoising diffusion probabilistic models for fast and high-fidelity whole-event simulation in high-energy heavy-ion experiments

Artificial intelligence (AI) generative models, such as generative adversarial networks (GANs), variational autoencoders, and normalizing flows, have been widely used and studied as efficient alternatives for traditional scientific simulations. However, they have several drawbacks, including training instability and inability to cover the entire data distribution, especially for regions where data are rare. This is particularly challenging for whole-event, full-detector simulations in high-energy heavy-ion experiments, such as sPHENIX at the Relativistic Heavy Ion Collider and Large Hadron Collider experiments, where thousands of particles are produced per event and interact with the detector. This work investigates the effectiveness of denoising diffusion probabilistic models (DDPMs) as an AI-based generative surrogate model for the sPHENIX experiment that includes the heavy-ion event generation and response of the entire calorimeter stack. DDPM performance in sPHENIX simulation data is compared with a popular rival, GANs. Results show that both DDPMs and GANs can reproduce the data distribution where the examples are abundant (low-to-medium calorimeter energies). Nonetheless, DDPMs significantly outperform GANs, especially in high-energy regions where data are rare. Additionally, DDPMs exhibit superior stability compared to GANs. The results are consistent between both central and peripheral centrality heavy-ion collision events. Moreover, DDPMs offer a substantial speedup of approximately a factor of 100 compared to the traditional Geant4 simulation method.

42 ENGINEERING↗
NASA NTRS1994

DE 1 and Viking observations associated with electron conical distributions

Data from the electron detectors on board the Swedish Viking satellite launched during a period of low solar activity and from the Dynamic Explorer (DE) 1 satellite launched during active solar coditions have been examined for the occurrence and location of electron conical distributions and several conclusions can be drawn. First, we note that most of the best examples of electron conics observed by the V-3 experiment onboard Viking occurred in the afternoon sector in the range of magneitc local time 14 hours less than Magnetic Local Time (MLT) less than 18 hours, at midaltitudes in the range 10,000 km less than h less than 13,500 km, with few occurring in the nightside auroral region, a region poorly sampled at altitudes greater than 5000 km. For the Viking data there is an association of electron conics with upper hybrid waves. DE 1 observations made by the high-altitude plasma instrument (HAPI) indicate that electron conics were observed in the midmorning sector and the late evening sector, and as has been reported earlier, the correlation with upper hybird waves was good. The HAPI did not sample the afternoon sector. The electon conics observed on both satellites occurred in the presence of at least a modest (several kilovolts) potential difference beneath the satellite with a maximum energy that was usually, but not always, equal to or greater than the maximum energy of the electron conics. Two independent sets of observations by DE 1 suggest two distinct production mechanisms for electron conics. Examiniation of DE 1 electric field measurements from the plasma wave instrument during the observation of electron conics show simultaneous parallel oscillations in the frequency range of 0.2 Hz less than f less than 0.5 Hz during one and perhaps two of four events examined, and upper hybrid waves were observed on all four events. In addition, recent observations of '90-deg' electron conics associated with auroral kilometric radiation source regions suggest a perpendicular heating mechanism produced by wave-particle interaction. Such distributions may be observed as electron conics at higher altitudes. These results suggest more than one possible source mechanism may be responsible for electron conics.

Menietti, J. D.↗
NASA NTRS1981

The sub-iron to iron ratios and the cosmic ray pathlength distribution

Data on the ratios of nuclei in the 21-25 charge range to iron, from the Danish-French cosmic ray isotope experiment on HEAO 3, are compared with cosmic ray propagation calculations based upon the latest cross sections. The results are incompatible with models in which the path length distribution has a peak at a path length larger than 1 g/sq cm, e.g., those in which there is more than 1 g/sq cm of matter around the cosmic ray source. They are most consistent with models in which the path length distribution includes substantial probabilities of having traversed very little matter.

Ormes, J. F.↗
NASA NTRS1994

Rice cultivation and methane emission: Documentation of distributed geographic data sets

High-resolution global data bases on the geographic and seasonal distribution of rice cultivation and associated methane emission, compiled by Matthews et al., were archived for public use. In addition to the primary data sets identifying location, seasonality, and methane emission from rice cultivation, a series of supporting data sets is included, allowing users not only to replicate the work of Matthews et al. but to investigate alternative cultivation and emission scenarios. The suite of databases provided, at 1 latitude by 1 longitude resolution for the globe, includes (1) locations of rice cultivation, (2) monthly arrays of actively growing rice areas, (3) countries and political subdivisions, and (4) monthly arrays of methane emission from rice cultivation. Ancillary data include (1) a listing, by country, of harvested rice areas and seasonal distribution of crop cycles and (2) country names and codes. Summary tables of zonal/monthly distributions of actively growing rice areas and of methane emissions are presented. Users should consult original publications for complete discussion of the data bases. This short paper is designed only to document formats of the distributed information and briefly describe the contents of the data sets and their initial application to evaluating the role of rice cultivation in the methane budget.

Matthews, Elaine↗
NASA NTRS1994

Constraint based scheduling for the Goddard Space Flight Center distributed Active Archive Center's data archive and distribution system

The Goddard Space Flight Center (GSFC) Distributed Active Archive Center (DAAC) has been operational since October 1, 1993. Its mission is to support the Earth Observing System (EOS) by providing rapid access to EOS data and analysis products, and to test Earth Observing System Data and Information System (EOSDIS) design concepts. One of the challenges is to ensure quick and easy retrieval of any data archived within the DAAC's Data Archive and Distributed System (DADS). Over the 15-year life of EOS project, an estimated several Petabytes (10(exp 15)) of data will be permanently stored. Accessing that amount of information is a formidable task that will require innovative approaches. As a precursor of the full EOS system, the GSFC DAAC with a few Terabits of storage, has implemented a prototype of a constraint-based task and resource scheduler to improve the performance of the DADS. This Honeywell Task and Resource Scheduler (HTRS), developed by Honeywell Technology Center in cooperation the Information Science and Technology Branch/935, the Code X Operations Technology Program, and the GSFC DAAC, makes better use of limited resources, prevents backlog of data, provides information about resources bottlenecks and performance characteristics. The prototype which is developed concurrently with the GSFC Version 0 (V0) DADS, models DADS activities such as ingestion and distribution with priority, precedence, resource requirements (disk and network bandwidth) and temporal constraints. HTRS supports schedule updates, insertions, and retrieval of task information via an Application Program Interface (API). The prototype has demonstrated with a few examples, the substantial advantages of using HTRS over scheduling algorithms such as a First In First Out (FIFO) queue. The kernel scheduling engine for HTRS, called Kronos, has been successfully applied to several other domains such as space shuttle mission scheduling, demand flow manufacturing, and avionics communications scheduling.

Short, Nick, Jr.↗
NASA NTRS2017

Building A Cloud Based Distributed Active Data Archive Center

NASA's Earth Science Data System (ESDS) Program facilitates the implementation of NASA's Earth Science strategic plan, which is committed to the full and open sharing of Earth science data obtained from NASA instruments to all users. The Earth Science Data information System (ESDIS) project manages the Earth Observing System Data and Information System (EOSDIS). Data within EOSDIS are held at Distributed Active Archive Centers (DAACs). One of the key responsibilities of the ESDS Program is to continuously evolve the entire data and information system to maximize returns on the collected NASA data.

Earth Science Informatics↗
NASA NTRS1982

Responsibilities and practical limitations in the operation of an astronomical data center

The operation of an astronomical data center requires that many decisions be made concerning the handling of the astronomical catalogs and data distributed. Should a data center be, as Wilkins (1977) described it, passive, in that catalogs and data are collected and distributed blackbox-wise upon request, or should a data center be active and have experts in various fields to scrutinize, correct, reformat, and document data where necessary. These questions will be addressed and illustrated by describing the current operations and future goals of the Astronomical Data Center at NASA/GSFC.

Warren, W. H., Jr.↗
NASA NTRS1987

SAR image data compression for an on-line archive system

This paper summarizes the investigation of SAR image data compression for an on-line archive data distribution system. This system is planned for the ground processing system of Alaska SAR Facility (ASF) and Shuttle Imaging Radar (SIR-C). The objective of the SAR image data compression is to enable the data archive system to provide the remote users a large data base with good image quality, short response time, low transfer cost, and minimal decoding complexity. The requirements and limitations of the on-line archive data distribution system are presented. The effects of SAR image data characteristics on data compression are addressed. The users' survey results suggest that compression ratios between 10:1 and 20:1 appear suitable. Based on the algorithm evaluation results, the two-level tree-searched vector quantization technique has been recommended as the SAR image data compression algorithm for the on-line archive data distribution system.

Chang, C. Y.↗
NASA NTRS1979

Seasat-A ASVT: Commercial demonstration experiments. Results analysis methodology for the Seasat-A case studies

The SEASAT-A commercial demonstration program ASVT is described. The program consists of a set of experiments involving the evaluation of a real time data distributions system, the SEASAT-A user data distribution system, that provides the capability for near real time dissemination of ocean conditions and weather data products from the U.S. Navy Fleet Numerical Weather Central to a selected set of commercial and industrial users and case studies, performed by commercial and industrial users, using the data gathered by SEASAT-A during its operational life. The impact of the SEASAT-A data on business operations is evaluated by the commercial and industrial users. The approach followed in the performance of the case studies, and the methodology used in the analysis and integration of the case study results to estimate the actual and potential economic benefits of improved ocean condition and weather forecast data are described.

Source record↗
NASA NTRSDate not supplied

Distributed Lunar Data Platform with Advanced Machine Learning Capabilities in Support of Lunar Science and Exploration

The United States 2020 Space Policy directive declares that NASA, in cooperation with private industry, will “extend human economic activity into deep space by establishing a permanent human presence on the Moon”. This goal will require advanced data management, as well as analysis, modeling and representation of lunar information in order to prepare for Artemis human missions, lunar science investigations and exploration. To meet this requirement, we conceptualize and present an implementation strategy for a distributed platform for lunar data retrieval, inferencing and analysis, which will be based on federated learning and the NASA Celestial Mapping System (CMS). In addition to demonstrating the imperative of enabling lunar-borne data to remain in-situ but still accessible, this presentation will also include examples of how third parties could contribute both datasets and new functionality into this platform using an AI-based data import pipeline and a plug-in architecture respectively.

Artificial Intelligence↗