Enabling 5G Distributed Scientific Data Analysis through Hierarchical Architectures and Algorithms.
Abstract not provided.
Cyr, Eric C↗
SEARCH · Engineering Papers
Results for “Data Distribution”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Phenomenology and Node-Level Processing of MASINT Sensor Data across Distributed Sensor Networks
This is a report for the FY2021 annual program overview.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DISTRIBUTED DIGITAL DATA ACQUISITION SYSTEM WITH NETWORK TIME SYNCHRONIZATION
Explore the source record for details and available documents.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Array distribution in data-parallel programs
We consider distribution at compile time of the array data in a distributed-memory implementation of a data-parallel program written in a language like Fortran 90. We allow dynamic redistribution of data and define a heuristic algorithmic framework that chooses distribution parameters to minimize an estimate of program completion time. We represent the program as an alignment-distribution graph. We propose a divide-and-conquer algorithm for distribution that initially assigns a common distribution to each node of the graph and successively refines this assignment, taking computation, realignment, and redistribution costs into account. We explain how to estimate the effect of distribution on computation cost and how to choose a candidate set of distributions. We present the results of an implementation of our algorithms on several test problems.
Chatterjee, Siddhartha↗
Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data: Preprint
To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.
data management↗
Integrated Large-Scale Data Management Platform for Photovoltaic Power Conversion Equipment (PCE) Reliability Data
To meet the demand for accuracy and real-time capability of PV system degradation evaluation, massive volume data is needed to run high-fidelity and high-efficiency simulations and perform advanced data analysis. However, PV farm operators have a series of difficulties with PV inverter data, such as data collection from multiple channels, massive data storage, data management and massive data analysis. To address these challenges, we developed an integrated data management platform capable of data acquisition, processing, storage, query, and performing big data analysis utilizing AI algorithms. The platform can also achieve data correctness verification and provide an effective distributed data management solution to retrieve massive data and establish a connection to distributed computational frameworks.
data management platform↗
Global Precipitation Measurement: Benefits of Partnering with GPM Mission - Report 2
An important goal of the Global Precipitation Measurement (GPM) mission is to maximize participation by non-NASA partners both domestic and international. A consequence of this objective is the provision for NASA to provide sufficient incentives to achieve partner buy-in and commitment to the program. NASA has identified seven specific areas in which substantive incentives will be offered: (1) partners will be offered participation in governance of GPM mission science affairs including definition of data products; (2) partners will be offered use of NASA's TDRSS capability for uplink and downlink of commands and data in regards to partner provided spacecraft; (3) partners will be offered launch support for placing partner provided spacecraft in orbit conditional upon mutually agreeable co-manifest arrangements; (4) partners will be offered direct data access at the NASA-GPM server level rather than through standard data distribution channels; (5) partners will be offered the opportunity to serve as regional data archive and distribution centers for standard GPM data products; and (6) partners will be offered the option to insert their own specialized filtering and extraction software into the GPM data processing stream or to obtain specialized subsets and products over specific areas of interest (7) partners will be offered GPM developed software tools that can be run on their platforms. Each of these incentives, either individually or in combination, represents a significant advantage to partners who may wish to participate in the GPM mission.
Stocker, Erich F.↗
A land-surface testbed for EOSDIS
We propose to develop an on-line data distribution and interactive display system for the collection, archival, distribution and analysis of operational weather satellite data for applications in land surface studies. A 1,000 sq km scene of the western U.S. (centered on the Colorado Rockies) will be extracted from Advanced Very High Resolution Radiometer (AVHRR) imagery collected from morning and afternoon passes of the NOAA polar-orbiters at the direct readout stations operated by CU/CCAR. All five channels of these AVHRR data will be navigated and map registered at CU/CCAR and then be transferred to NCAR for storage in an on-line data system. Software will also be available at NCAR to process and navigate the raw AVHRR data as needed. A display workstation software, based on a Macintosh II computer, will be developed that will display and further process the AVHRR data for studies of vegetation monitoring and snowpack assessment. Various options of presently used techniques for both vegetation and snowpack monitoring will be implemented in the workstation software to provide the individual investigator with the freedom to interact with the satellite image data. The display software will be freely distributed online to interested investigators and the AVHRR data will be made available on-line to anyone interested. In addition, potential users will be sought out and connected to the on-line data archive. This experiment with an active on-line archive and interactive analysis systems will provide experience with a small scale EOSDIS.
Emery, William↗
Space shuttle Mission Control Center upgrade
The Mission Control Center (MCC) is the facility that provides centralization of all mission operations in support of the shuttle from liftoff through landing. This facility serves as the focal point for real time flight control operations supporting the shuttle, spacelab, and selected payload systems. MCC upgrade accomplishments are as follows: replaced 5 mainframe computers and the associated input/output data interfaces; developed and installed a distributed data system with 81 workstations, 5 LANs, data drivers, and a fiber optic distribution subsystem; developed over 1.5 million lines of code to support new distributed system; developed new digital voice system which provides the MCC capacity to handle th Shuttle programs demands and growth potential to handle Space Station; replaced failing digital TV equipment with digital generation equipment; replaced obsolete center screen projector; and provided the platform for the multiprogram control center.
Duffin, Pat↗
New Architecture to Support Integration and Processing of Seismic Data from Heterogeneous Sources
The Geophysical Monitoring Program (GMP) at Lawrence Livermore National Lab (LLNL) maintains a database and supporting infrastructure for geophysical data used in support of the Nuclear Detonation Detection mission. This database includes data from multiple sources, many of which do not distribute data to the public or for which there is no automated means of access. For example, Figure 1 shows (left) the distribution of waveform data in our database by source. The Incorporated Research Institutions for Seismology Data Management Center (IRISDMC) is our major source of waveform data and those data may be retrieved at will using the Federated Digital Seismograph Networks FDSN web Application Programming Interface (API). However, the next 6 most important sources of waveform data have no or only limited automated access to waveforms. As Figure 1 (right) shows, it is very common for waveform records associate with an event in our database to come from two or more sources, and in some cases data come from 10 sources. This diversity of data sources drives our need for efficient and correct integration of metadata, parametric data, and waveform data.
58 GEOSCIENCES↗
What is LAADS DAAC? How to Download Data from NASA's Source for Cloud and Aerosol Data
The Level-1 and Atmosphere Archive Distribution System (LAADS) Distributed Active Archive Center (DAAC) primarily archives and distributes data on clouds and aerosols in Earth’s atmosphere in addition to providing Level 0 and 1 Moderate Imaging Radiometer Suite (MODIS), Visible Infrared Imaging Radiometer Suite (VIIRS) and airborne data as well as data from the European Space Agency’s Sea and Land Surface Temperature Radiometer (SLSTR) and the Ocean and Land Colour Instrument (OLCI). Secondarily, LAADS DAAC serves as a backup source for MODIS and VIIRS land products. As one of twelve DAACS supported by NASA’s Earth Science Data and Information System (ESDIS), LAADS DAAC provides a gateway to data through multiple tools. The newest, View Data, allows users to preview data sets before downloading or applying any post-processing transformations. Learn about how to access LAADS DAAC data, get up-to-date information about LAADS DAAC's migration to hosting data in the cloud, and discover how to use the View Data tool.
Tassia Owen↗
Lessons Learned in Employing Data Analytics to Predict Oxidation Kinetics and Spallation Behavior of High-Temperature NiCr-Based Alloys
Machine learning (ML) can offer many advantages in predicting material properties over traditional materials development methods based solely on limited experimental investigations or physical-based simulations with the capability to reduce development cost, risk, and time. However, so far, limited efforts have been made to predict alloy oxidation kinetics and spallation behavior via ML due to the lack of consistently measured and sufficient experimental data and the inherent complexity in oxidation behavior of multicomponent high-temperature alloys. A previous study reported the ability of ML to predict oxidation kinetics of NiCr-based alloys as a function of alloy composition and operating conditions. Here, the performance of a ML model in predicting rate constants and spallation probability was evaluated in light of the roles of the data distribution of the experimental dataset (data analytics), the alloy composition, the exposure environment and the chosen oxidation approach to extracting kinetic values from the measured mass changes (but using either a simple parabolic law or a statistical cyclic oxidation model). Potential strategies to improve the predictions and enhance the extrapolative capability of the previously trained model will be discussed.
36 MATERIALS SCIENCE↗
Improving I/O Performance for Exascale Applications through Online Data Layout Reorganization
The applications being developed within the U.S. Exascale Computing Project (ECP) to run on imminent Exascale computers will generate scientific results with unprecedented fidelity and record turn-around time. Many of these codes are based on particle-mesh methods and use advanced algorithms, especially dynamic load-balancing and mesh-refinement, to achieve high performance on Exascale machines. Yet, as such algorithms improve parallel application efficiency, they raise new challenges for I/O logic due to their irregular and dynamic data distributions. Thus, while the enormous data rates of Exascale simulations already challenge existing file system write strategies, the need for efficient read and processing of generated data introduces additional constraints on the data layout strategies that can be used when writing data to secondary storage. We review these I/O challenges and introduce two online data layout reorganization approaches for achieving good tradeoffs between read and write performance. We demonstrate the benefits of using these two approaches for the ECP particle-in-cell simulation WarpX, which serves as a motif for a large class of important Exascale applications. Here, we show that by understanding application I/O patterns and carefully designing data layouts we can increase read performance by more than 80 percent.
97 MATHEMATICS AND COMPUTING↗
Flight evaluation: Ohio University omega receiver base
A flight evaluation is presented of the Ohio University Omega Receiver Base, developed under the NASA Tri-University Program in Air Transportation, to provide a vehicle for the transfer of flight-test data to NASA and to other participants in the Tri-University program. Chart recordings of flight data are given, along with chronological listings of significant events which occurred during the flight. Digital data was prepared in data-processing card form for distribution. Data include phase measurements from all eight Omega time-slots for the duration of the flight, plus event marks which serve to correlate the phase data with flight-path documentation.
Chamberlin, K. A.↗
Orbit determination requirements for TOPEX
The error sensitivity of orbit calculations in support of the NASA Ocean Surface Topography Mapping Experiment (TOPEX), which require an accuracy on the order of 5 cm, is investigated. The contributions of errors in the gravitational, atmospheric drag and solar radiation pressure models to the computed orbit are analyzed for the cases of an ideal data distribution and realistic laser ranging data coverage. It is found that the major contributor to radial orbital error is the error in the geopotential model, accounting for orbital errors of 30 to 70 cm, with the effects of solar radiation pressure, drag modeling, tracking station coordinate errors making lesser contributions. It is concluded that TOPEX accuracy goals cannot be met using ground-based laser ranging data without improving the geopotential model.
Tapley, B. D.↗
Simulation of the space station information system in Ada
The Flexible Ada Simulation Tool (FAST) is a discrete event simulation language which is written in Ada. FAST has been used to simulate a number of options for ground data distribution of Space Station payload data. The fact that Ada language is used for implementation has allowed a number of useful interactive features to be built into FAST and has facilitated quick enhancement of its capabilities to support new modeling requirements. General simulation concepts are discussed, and how these concepts are implemented in FAST. The FAST design is discussed, and it is pointed out how the used of the Ada language enabled the development of some significant advantages over classical FORTRAN based simulation languages. The advantages discussed are in the areas of efficiency, ease of debugging, and ease of integrating user code. The specific Ada language features which enable these advances are discussed.
Spiegel, James R.↗
Mercury Toolset for Spatiotemporal Metadata
Mercury (http://mercury.ornl.gov) is a set of tools for federated harvesting, searching, and retrieving metadata, particularly spatiotemporal metadata. Version 3.0 of the Mercury toolset provides orders of magnitude improvements in search speed, support for additional metadata formats, integration with Google Maps for spatial queries, facetted type search, support for RSS (Really Simple Syndication) delivery of search results, and enhanced customization to meet the needs of the multiple projects that use Mercury. It provides a single portal to very quickly search for data and information contained in disparate data management systems, each of which may use different metadata formats. Mercury harvests metadata and key data from contributing project servers distributed around the world and builds a centralized index. The search interfaces then allow the users to perform a variety of fielded, spatial, and temporal searches across these metadata sources. This centralized repository of metadata with distributed data sources provides extremely fast search results to the user, while allowing data providers to advertise the availability of their data and maintain complete control and ownership of that data. Mercury periodically (typically daily) harvests metadata sources through a collection of interfaces and re-indexes these metadata to provide extremely rapid search capabilities, even over collections with tens of millions of metadata records. A number of both graphical and application interfaces have been constructed within Mercury, to enable both human users and other computer programs to perform queries. Mercury was also designed to support multiple different projects, so that the particular fields that can be queried and used with search filters are easy to configure for each different project.
Wilson, Bruce E.↗
Resiliency-Driven Proactive Distribution System Reconfiguration With Synchrophasor Data
Measuring and enabling resiliency of electric distribution systems with increasing weather and cyber events are important. Some of the extreme events (e.g. Earthquakes, Hurricanes) and associated paths are predicted and monitored closely in advance and allow to take pre-event proactive control actions. The Distribution Phasor Measurement Units (D-PMUs) provide new opportunities and supporting such proactive actions. Here, a synchrophasor based resiliency driven pre-event reconfiguration can ensure minimizing impact of the expected event on the power distribution system and associated performance. However, the D-PMUs will also face challenges in terms of data quality similar to the transmission PMUs. The focus of this paper is to provide data mining approaches for anomaly detection in D-PMUs and proposing resiliency-driven pre-event reconfiguration with islanding as a proactive mechanisms to minimize the impact of adverse events on system using processed synchrophasors data. Results are validated for real industrial feeders and test cases with satisfactory response
controlled islanding↗