Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Probability distribution of wind retrieval error for the NASA scatterometer

The NASA scatterometer (NSCAT) is a spaceborne scatterometer scheduled to be deployed in the mid-1990s. An analysis of the wind retrieval error distribution for wind estimates based on backscatter measurements made by the NSCAT instrument is presented. The results are based on an end-to-end simulation of the scatterometer instrument and data processing. In general, the distribution of the wind speed error, when normalized, is independent of the true wind speed and direction. The wind speed error can be characterized by a normal distribution. The wind direction error is independent of the true wind speed, but depends on the true wind direction. Details for wind vectors with true wind speeds from 3 m/s to 33 m/s and true wind directions from 0 to 360 deg are presented.

Leotta, Daniel F.↗

NASA Plan for Increasing Access to the Results of Scientific Research

This plan is issued in response to the Executive Office of the President's February 22, 2013, Office of Science and Technology Policy (OSTP) Memorandum for the Heads of Executive Departments and Agencies, "Increasing Access to the Results of Federally Funded Scientific Research." Through this memorandum, OSTP directed all agencies with more than $100 million in annual research and development expenditures to prepare a plan for improving the public's access to the results of federally funded research. The National Aeronautics and Space Administration (NASA) invests on the order of $3 billion annually in fundamental and applied research and technology development1 across a broad range of topics, including space and Earth sciences, life and physical sciences, human health, aeronautics, and technology. Promoting the full and open sharing of data with research communities, private industry, academia, and the general public is one of NASA's longstanding core values. For example, NASA's space and suborbital mission personnel routinely process, archive, and distribute their data to researchers around the globe. This plan expands the breadth of NASA's open-access culture to include data and publications for all of the scientific research that the Agency sponsors.

Source record↗

A distributed data base management system

Major system design features of a distributed data management system for the NASA Deep Space Network (DSN) designed for continuous two-way deep space communications are described. The reasons for which the distributed data base utilizing third-generation minicomputers is selected as the optimum approach for the DSN are threefold: (1) with a distributed master data base, valid data is available in real-time to support DSN management activities at each location; (2) data base integrity is the responsibility of local management; and (3) the data acquisition/distribution and processing power of a third-generation computer enables the computer to function successfully as a data handler or as an on-line process controller. The concept of the distributed data base is discussed along with the software, data base integrity, and hardware used. The data analysis/update constraint is examined.

Bryan, A. I.↗

CMS centres worldwide: A new collaborative infrastructure

The CMS Experiment at the LHC is establishing a global network of inter-connected 'CMS Centres' for controls, operations and monitoring. These support: (1) CMS data quality monitoring, detector calibrations, and analysis; and (2) computing operations for the processing, storage and distribution of CMS data. We describe the infrastructure, computing, software, and communications systems required to create an effective and affordable CMS Centre. We present our highly successful operations experiences with the major CMS Centres at CERN, Fermilab, and DESY during the LHC first beam data-taking and cosmic ray commissioning work. The status of the various centres already operating or under construction in Asia, Europe, Russia, South America, and the USA is also described. We emphasise the collaborative communications aspects. For example, virtual co-location of experts in CMS Centres Worldwide is achieved using high-quality permanently-running 'telepresence' video links. Generic Web-based tools have been developed and deployed for monitoring, control, display management and outreach.

99 GENERAL AND MISCELLANEOUS↗

NASA SNPP SIPS - Following in the Path of EOS

NASA's Earth Science Data Information System (ESDIS) Project has been operating NASA's Suomi National Polar-Orbiting Partnership (SNPP) Science Data Segment (SDS) since the launch in October 2011. At launch, the SDS focused primarily on the evaluation of Sensor Data Records (SDRs) and Environmental Data Records (EDRs) produced by the Joint Polar Satellite System (JPSS), a National Oceanic and Atmosphere Administration (NOAA) Program, as to their suitability for Earth system science. During the summer of 2014, NASA transitioned to the production of standard Earth Observing System (EOS)-like science products for all instruments aboard Suomi NPP. The five Science Investigator-led Processing Systems (SIPS): Land, Ocean, Atmosphere, Ozone, and Sounder were established to produce the NASA SNPP standard Level 1, Level 2, and global Level 3 products developed by the SNPP Science Teams and to provide the products to NASA's Distributed Active Archive Centers (DAACs) for archive and distribution to the user community. The processing, archiving and distribution of data from NASA's Clouds and the Earth's Radiant Energy System (CERES) and Ozone Mapper/Profiler Suite (OMPS) Limb instruments will continue. With the implementation of the JPSS Block 2 architecture and the launch of JPSS-1, the SDS will receive SNPP data in near real-time via the JPSS Stored Mission Data Hub (JSH), as well as JPSS-1 and future JPSS-2 data. The SNPP SIPS will ingest EOS compatible Level 0 data from the EOS Data Operations System (EDOS) element for their data processing, enabling the continuous EOS-SNPP-JPSS Satellite Data Record.

near real time↗

KDD Services at the Goddard Earth Sciences Distributed Active Archive Center

NASA's Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) processes, stores and distributes earth science data from a variety of remote sensing satellites. End users of the data range from instrument scientists to global change and climate researchers to federal agencies and foreign governments. Many of these users apply data mining techniques to large volumes of data (up to 1 TB) received from the GES DAAC. However, rapid advances in processing power are enabling increases in data processing that are outpacing tape drive performance and network capacity. As a result, the proportion of data that can be distributed to users continues to decrease. As mitigation, we are migrating more data mining and mining preparation activities into the data center in order to reduce the data volume that needs to be distributed and to offer the users a more useful and manageable product. This migration of activities faces a number of technical and human-factor challenges. As data reduction and mining algorithms are normally quite specific to the user's research needs, the user's algorithm must be integrated virtually unchanged into the archive environment. Also, the archive itself is busy with everyday data archive and distribution activities and cannot be dedicated to, or even impacted by, the mining activities. Therefore, we schedule KDD 'campaigns' (similar to reprocessing campaigns), during which we schedule a wholesale retrieval of specific data products, offering users the opportunity to extract information from the data being retrieved during the campaign.

Lynnes, Christopher↗

An overview of the NASA electronic components information management system

The NASA Parts Project Office (NPPO) comprehensive data system to support all NASA Electric, Electronic, and Electromechanical (EEE) parts management and technical data requirements is described. A phase delivery approach is adopted, comprising four principal phases. Phases 1 and 2 support Space Station Freedom (SSF) and use a centralized architecture with all data and processing kept on a mainframe computer. Phases 3 and 4 support all NASA centers and projects and implement a distributed system architecture, in which data and processing are shared among networked database servers. The Phase 1 system, which became operational in February of 1990, implements a core set of functions. Phase 2, scheduled for release in 1991, adds functions to the Phase 1 system. Phase 3, to be prototyped beginning in 1991 and delivered in 1992, introduces a distributed system, separate from the Phase 1 and 2 system, with a refined semantic data model. Phase 4 extends the data model and functionality of the Phase 3 system to provide support for the NASA design community, including integration with Computer Aided Design (CAD) environments. Phase 4 is scheduled for prototyping in 1992 to 93 and delivery in 1994.

Kramer, G.↗

UH cosmic rays - Possible origin in massive stars

The origin of the Z greater than 28, ultraheavy (UH) cosmic rays in supernova explosions of massive stars, at least about 10 solar masses, is considered. For Z greater than 70, the UH data are dominated by an r-process source distribution, but for the elements just beyond iron, Z from 29 to 36, the data cannot be explained by any single process of nucleosynthesis. This problem is solved naturally in a massive-star model by secondary neutron-capture reactions occurring during core helium burning (a limited s-process) and during explosive carbon burning. Interstellar-propagation calculations have been performed with these episodes of synthesis as source distributions, and the results offer an explanation for the current UH cosmic-ray data. Further, the heavy-element synthesis during explosive carbon burning is re-examined by using more realistic initial conditions given by the post-helium-burning configuration of the star. These results are compared with earlier work and the UH cosmic-ray data. Some effects of preferential acceleration, based upon ionization potential, are considered, and experimental tests for this model are discussed.

Wefel, J. P.↗

Time-Frequency Approach for Stochastic Signal Detection

The detection of events in a stochastic signal has been a subject of great interest. One of the oldest signal processing technique, Fourier Transform of a signal contains information regarding frequency content, but it cannot resolve the exact onset of changes in the frequency, all temporal information is contained in the phase of the transform. On the other hand, Spectrogram is better able to resolve temporal evolution of frequency content, but has a trade-off in time resolution versus frequency resolution in accordance with the uncertainty principle. Therefore, time-frequency representations are considered for energetic characterisation of the non-stationary signals. Wigner Ville Distribution (WVD) is the most prominent quadratic time-frequency signal representation and used for analysing frequency variations in signals.WVD allows for instantaneous frequency estimation at each data point, for a typical temporal resolution of fractions of a second. This paper through simulations describes the way time frequency models are applied for the detection of event in a stochastic signal.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Use of Open Networks and Delay-Tolerant Protocol to Decrease WAN Latency of EOS near Real-Time Data

Since 1999, NASA's Earth Observing System Data Operations System (EDOS) project at Goddard Space Flight Center (GSFC) has provided high-rate data capture, level zero processing, and product distribution services for a majority of NASA's EOS (Earth Observing System) high-rate missions, including Terra, Aqua, Aura, ICESat, EO-1, SMAP, and OCO-2. EDOS high-rate science and engineering (150-300 Mbps) data-driven capture systems are deployed at 7 worldwide ground stations which are connected via both private (closed) and public (open) wide area networks (WANs) to the centralized EDOS Level Zero Processing Facility (LZPF) located at GSFC, where the data is processed and Level 0 products are distributed to users worldwide. All data transferred over the open networks to GSFC traverse an IPSec tunnel, providing the same level of security as a VPN connection. EDOS produces both time-based and near real-time products (session-based). Near real-time data products are produced from a single ground station contact; time-based products are produced from multiple ground station contacts. EDOS is the primary supplier of EOS Level 0 data to the NASA near real-time user community known as the Land, Atmosphere Near real-time Capability for EOS (LANCE). For the past few years, EDOS has streamlined its systems to reduce WAN latency for near real-time data delivery, including implementing Quality of Service (QoS), expanding closed network bandwidth, adding open network connections with more bandwidth, and implementing a delay-tolerant protocol to mitigate long round-trip times to remote ground stations.

Delay-Tolerant Protocol↗

DATA RECOVERY EFFORTS AT IDAHO NATIONAL LABORATORY, OAK RIDGE NATIONAL LABORATORY, AND SAVANNAH RIVER NATIONAL LABORATORY

Abstract was already submitted. Could not find the previous number. Would be fine with attaching/update of old number. Abstract Below: Modern nuclear facilities will have significant process monitoring capability for their operators. These systems will also be used for domestic safeguards applications, which has led to research over new diversion-detection algorithms. Curiously missing from these efforts are verification and validation data sets. A tri-laboratory project to locate the existing data sets and recover their data has yielded three major potential sources of data. The first is recovery of the process monitoring data of the Idaho Chemical Processing Plant, which now has a distributable package for algorithm developers. The second data set is extensive sampling and process data from Savannah River National Laboratory’s F- and H-canyon sites. Finally, high fidelity data from the start-up tests at the Barnwell Reprocessing Facility is in recovery. This paper details the data sets and compares their relative attributes.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

An MPI-IO Interface to HPSS

This paper describes an implementation of the proposed MPI-IO standard for parallel I/O. Our system uses third-party transfer to move data over an external network between the processors where it is used and the I/O devices where it resides. Data travels directly from source to destination, without the need for shuffling it among processors or funneling it through a central node. Our distributed server model lets multiple compute nodes share the burden of coordinating data transfers. The system is built on the High Performance Storage System (HPSS), and a prototype version runs on a Meiko CS-2 parallel computer.

Parallel Processing↗

Deactivation and decommissioning web log analysis using big data technology - 15710

The D and D Knowledge Management Information Tool (D and D KM- IT) is a web-based knowledge management information tool built for the D and D user community. Big data is a massive volume of structured and unstructured datasets which is so large that it's difficult to process using traditional database techniques. Web logs are the repository of files which are generated automatically for any operation on the web site. Web log files generated from the D and D KM-IT will be processed using the Apache Hadoop Framework to extract meaningful data. Hadoop is an open-source software framework for storing and processing big data in a distributed environment on large clusters of commodity hardware. Hadoop framework consists of two main layers: Hadoop distributed file system (HDFS) and the Execution engine (MapReduce). Automatic parallelization and distribution; clean and simple programming abstraction. Implementation: Web log files from the D and D KM-IT server are fetched using Apache Flume and loaded into the Hadoop distributed file system. Using the Apache Hcatalog tool, the web log data is parsed and converted into a structured table format. A Pig program is developed to query the tables to retrieve and store the significant information onto the HDFS. The extracted data is fed to business visualization tools such as Microsoft Excel for analysis and reporting. Results The reports generated through the web log data processing will be analyzed to improve the usability and performance of the D and D KM-IT. Conclusion/Future Work: User keyword information can be explored to add new trending topics to the D and D KM-IT application. Statistics generated can be compared to analysis reports from existing analytical tools.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Content Platforms Meet Data Storage, Retrieval Needs

Earth is under a constant barrage of information from space. Whether from satellites orbiting our planet, spacecraft circling Mars, or probes streaking toward the far reaches of the Solar System, NASA collects massive amounts of data from its spacefaring missions each day. NASA s Earth Observing System (EOS) satellites, for example, provide daily imagery and measurements of Earth s atmosphere, oceans, vegetation, and more. The Earth Observing System Data and Information System (EOSDIS) collects all of that science data and processes, archives, and distributes it to researchers around the globe; EOSDIS recently reached a total archive volume of 4.5 petabytes. Try to store that amount of information in your standard, four-drawer file cabinet, and you would need 90 million to get the job done. To manage the flood of information, NASA has explored technologies to efficiently collect, archive, and provide access to EOS data for scientists today and for years to come. One such technology is now providing similar capabilities to businesses and organizations worldwide.

Source record↗

Upper Atmosphere Research Satellite (UARS) trade analysis

The Upper Atmosphere Research Satellite (UARS) which will collect data pertinent to the Earth's upper atmosphere is described. The collected data will be sent to the central data handling facility (CDHF) via the UARS ground system and the data will be processed and distributed to the remote analysis computer systems (RACS). An overview of the UARS ground system is presented. Three configurations were developed for the CDHF-RACS system. The CDHF configurations are discussed. The IBM CDHF configuration, the UNIVAC CDHF configuration and the vax cluster CDHF configuration are presented. The RACS configurations, the IBM RACS configurations, UNIVAC RACS and VAX RACS are detailed. Due to the large on-line data estimate to approximately 100 GB, a mass storage system is considered essential to the UARS CDHF. Mass storage systems were analyzed and the Braegan ATL, the RCA optical disk, the IBM 3850 and the MASSTOR M860 are discussed. It is determined that the type of mass storage system most suitable to UARS is the automated tape/cartridge device. Two devices of this type, the IBM 3850 and the MASSTOR MSS are analyzed and the applicable tape/cartridge device is incorporated into the three CDHF-RACS configurations.

Fox, M. M.↗

Keyless Infrastructure Security Solution (KISS): VOLTTRON™ KSI ® Blockchain Design and Specification

This document provides the technical design specifications that are required for Keyless Infrastructure Security Solution (KISS) development, including, definitions and roles of the new KISS-related products—GridAware and LinkLite—an overview of the DNP3 communications the VOLTTRON™ DNP3 agent is using for distribution control, data flow, historian processing, setup, and installation. These technical specifications provide step-by-step instructions for configuration, implementation, and deployment of the KISS subsystem, docket creation, and verification using the GridAware dashboard. Based on this document and the KISS specification requirements and device risk assessment documents, KISS will be developed by integrating keyless signature infrastructure (KSI) blockchain to increase the cybersecurity, integrity, and trustworthiness of critical energy delivery systems (EDSs). The KISS architecture consists of a VOLTTRON™ agent-based distribution control system platform and GridAware (KSI blockchain capability) for secure energy exchanges in a decentralized electrical network.

42 ENGINEERING↗

Preparation of the Multi-Site Data Processing at the Vera C. Rubin Observatory

The Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) Camera is scheduled to start taking data in the summer of 2025. The Data Release Production will run the LSST Science Pipe software at data facilities in the US, France and the UK. The LSST Science Pipeline consists of complex directed acyclic graphs (DAGs) of tasks. Rubin will use the Production and Distributed Analysis (PanDA) workflow and workload management system to orchestrate this complex workflow and the distribution of workloads to the data facilities. When run end-to-end by a team of data production staff, this processing (the Science Pipelines, distributed by the workflow and workload management system) is referred to as a 'campaign'. This paper describes the central services and data facility specific services that support this multi-site data process model, including the service deployment infrastructure, the workload and workflow system, the Campaign Management tools, and connection to Rubin Data Management. This paper will also mention the experience of processing the Rubin Commissioning Camera data. All these are part of the effort to scale up the processing capabilities for the expected very large data volume from the LSST Camera.

Yang, Wei [SLAC]↗