Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Data Management as a Cluster Middleware Centerpiece

Through earth and space modeling and the ongoing launches of satellites to gather data, NASA has become one of the largest producers of data in the world. These large data sets necessitated the creation of a Data Management System (DMS) to assist both the users and the administrators of the data. Halcyon Systems Inc. was contracted by the NASA Center for Computational Sciences (NCCS) to produce a Data Management System. The prototype of the DMS was produced by Halcyon Systems Inc. (Halcyon) for the Global Modeling and Assimilation Office (GMAO). The system, which was implemented and deployed within a relatively short period of time, has proven to be highly reliable and deployable. Following the prototype deployment, Halcyon was contacted by the NCCS to produce a production DMS version for their user community. The system is composed of several existing open source or government-sponsored components such as the San Diego Supercomputer Center s (SDSC) Storage Resource Broker (SRB), the Distributed Oceanographic Data System (DODS), and other components. Since Data Management is one of the foremost problems in cluster computing, the final package not only extends its capabilities as a Data Management System, but also to a cluster management system. This Cluster/Data Management System (CDMS) can be envisioned as the integration of existing packages.

Zero, Jose↗
NASA NTRS1995

Flexible Revision Of Data-Processing Communications

Distributed Objects Protocol Layer (DOPL) software system implements protocol for communications among software objects programmed in C++ computer language. DOPL is application-program interface (API). Enables communication of data without regard for which specific computer(s) and operating system(s) is (are) in use. No extension of the C++ language necessary for use of DOPL.

Burleigh, Scott C.↗
NASA NTRS2000

VLBI Contribution to ITRF2000

The VLBI data set spanning Aug. 1919 to the present is a major constituent of ITRF2000 because of the time span, the geometric strength and the accuracy of the modeling. Formal velocity errors for the best stations are less than 0.3 mm/yr while the probable frame definition error for station velocities is approximately 0.4 mm/yr. The primary systematic errors are the troposphere and tidal loading (solid Earth and ocean). However, the scale of the VLBI TRF is probably only weakly affected because of continued improvements in the tropospheric mapping function and gradient estimation. There are some weaknesses in the spatial distribution of data, particularly in the southern hemisphere, and the quality of the mobile VLBI results is now suboptimal. The deployment of the TIGO system to Chile will improve the geometric coverage. The current state of the art of VLBI analysis will be discussed.

Ma, Chopo↗
NASA NTRS2016

NASA SNPP SIPS - Following in the Path of EOS

NASA's Earth Science Data Information System (ESDIS) Project has been operating NASA's Suomi National Polar-Orbiting Partnership (SNPP) Science Data Segment (SDS) since the launch in October 2011. At launch, the SDS focused primarily on the evaluation of Sensor Data Records (SDRs) and Environmental Data Records (EDRs) produced by the Joint Polar Satellite System (JPSS), a National Oceanic and Atmosphere Administration (NOAA) Program, as to their suitability for Earth system science. During the summer of 2014, NASA transitioned to the production of standard Earth Observing System (EOS)-like science products for all instruments aboard Suomi NPP. The five Science Investigator-led Processing Systems (SIPS): Land, Ocean, Atmosphere, Ozone, and Sounder were established to produce the NASA SNPP standard Level 1, Level 2, and global Level 3 products developed by the SNPP Science Teams and to provide the products to NASA's Distributed Active Archive Centers (DAACs) for archive and distribution to the user community. The processing, archiving and distribution of data from NASA's Clouds and the Earth's Radiant Energy System (CERES) and Ozone Mapper/Profiler Suite (OMPS) Limb instruments will continue. With the implementation of the JPSS Block 2 architecture and the launch of JPSS-1, the SDS will receive SNPP data in near real-time via the JPSS Stored Mission Data Hub (JSH), as well as JPSS-1 and future JPSS-2 data. The SNPP SIPS will ingest EOS compatible Level 0 data from the EOS Data Operations System (EDOS) element for their data processing, enabling the continuous EOS-SNPP-JPSS Satellite Data Record.

near real time↗
NASA NTRS2008

Transformation of OODT CAS to Perform Larger Tasks

A computer program denoted OODT CAS has been transformed to enable performance of larger tasks that involve greatly increased data volumes and increasingly intensive processing of data on heterogeneous, geographically dispersed computers. Prior to the transformation, OODT CAS (also alternatively denoted, simply, 'CAS') [wherein 'OODT' signifies 'Object-Oriented Data Technology' and 'CAS' signifies 'Catalog and Archive Service'] was a proven software component used to manage scientific data from spaceflight missions. In the transformation, CAS was split into two separate components representing its canonical capabilities: file management and workflow management. In addition, CAS was augmented by addition of a resource-management component. This third component enables CAS to manage heterogeneous computing by use of diverse resources, including high-performance clusters of computers, commodity computing hardware, and grid computing infrastructures. CAS is now more easily maintainable, evolvable, and reusable. These components can be used separately or, taking advantage of synergies, can be used together. Other elements of the transformation included addition of a separate Web presentation layer that supports distribution of data products via Really Simple Syndication (RSS) feeds, and provision for full Resource Description Framework (RDF) exports of metadata.

Mattmann, Chris↗
NASA NTRS2019

Collecting and Processing Earth Science Data Metrics at NASA ESDIS

Since the launch of Terra satellite in 1999, the number of Earth Science remote sensing data products created and distributed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) has increased from a few hundred to nearly ten thousand. NASA's Earth Science Data and Information System (ESDIS) Metrics System (EMS) collects metrics on data ingest, archive, and distribution by its Distributed Active Archive Centers (DAACs) and the Science Investigator-led Systems (SIPS), known as Data Providers. These metrics are critical in helping NASA management as well as data producers in resource planning and gaining a wide range of knowledge of data users and data usage.EMS receives flat files, or log files of data archive, ingest, and distribution either in their raw format, such as Apache web logs, or text files of log records formatted by the Data Providers. Tens of millions of records are processed each day to extract metrics on data products, user information, distribution protocols and services, and so on. The metrics are then made available to designated parties.This presentation provides an overview of the EMS processing workflow and improvement efforts made in recent years to handle ever-increasing number of data records and new metrics requirements, discusses several key steps including mapping log records to data products and identifying user communities along with geo-distribution, and demonstrates typical metrics capabilities produced by the EMS system. Challenges and potential approaches to improve the system are also discussed.

Pan, Jianfu↗
NASA NTRS1991

A new method for analyzing IRAS data to determine the dust temperature distribution

In attempting to analyze the four-band IRAS images of interstellar dust emission, it is found that an inversion theorem recently developed by Chen (1990) enables distribution of the dust to be determined as a function of temperature and thus the total dust column density, for each line of sight. The method and its application to a hypothetical IRAS data set created by assuming a power-law dust temperature distribution, which is characteristic of the actual IRAS data for the Monoceros R2 cloud, are reported. To use the method, the wavelength dependence of the dust emissivity is assumed and a simple function is fitted to the four intensity-wavelength data points. The method is shown to be very successful at retrieving the dust temperature distribution in this case and is expected to have wide applicability to astronomical problems of this type.

Xie, Taoling↗
NASA NTRS1995

Continuation: The EOSDIS testbed data system

The continuation of the EOSDIS testbed ('Testbed') has materialized from a multi-task system to a fully functional stand-alone data archive distribution center that once was only X-Windows driven to a system that is accessible by all types of users and computers via the World Wide Web. Throughout the past months, the Testbed has evolved into a completely new system. The current system is now accessible through Netscape, Mosaic, and all other servers that can contact the World Wide Web. On October 1, 1995 we will open to the public and we expect that the statistics of the type of user, where they are located, and what they are looking for will drastically change. What is the most important change in the Testbed has been the Web interface. This interface will allow more users access to the system and walk them through the data types with more ease than before. All of the callbacks are written in such a way that icons can be used to easily move around in the programs interface. The homepage offers the user the opportunity to go and get more information about each satellite data type and also information on free programs. These programs are grouped into categories for types of computers that the programs are compiled for, along with information on how to FTP the programs back to the end users computer. The heart of the Testbed is still the acquisition of satellite data. From the Testbed homepage, the user selects the 'access to data system' icon, which will take them to the world map and allow them to select an area that they would like coverage on by simply clicking that area of the map. This creates a new map where other similar choices can be made to get the latitude and longitude of the region the satellite data will cover. Once a selection has been made the search parameters page will appear to be filled out. Afterwards, the browse image will be called for once the search is completed and the images for viewing can be selected. There are several other option pages, but once an order has been selected the Testbed will bring up the order list page and the user will then be able to place their order. After the order has been completed, the Testbed will mail the user to notify them of the completed order and how the images can be picked up.

Emery, Bill↗
NASA NTRS1980

A command and data subsystem for deep space exploration based on the RCA 1802 microprocessor in a distributed configuration

The Command and Data Subsystem (CDS) is an RCA 1802microprocessor based subsystem that acts as the central nervous system for the Galileo Orbiter Spacecraft. All communication between the ground and spacecraft flows through the CDS. The CDS also distributes commands in real time, algorithmetrically expanded from a data base loaded from the ground and in response to spacecraft alarms. The distributed microprocessor system is configured as a redundant set of hardware with three microprocessors on each half. The microprocessors are surrounded by a group of special purpose hardware components which greatly enhance the ability of the software to perform its task. It is shown how the software architecture makes a distributed system of six microprocessors appear to each user as a single virtual machine, and collectively as a set of cooperating virtual machines that prevent the simultaneous presence of the several users from interfering destructively with each other.

Thomas, J. S.↗
NASA NTRS2005

Lightning Mapping and the Nowcasting of Severe Storms

This paper describes a successful research and operational collaboration between NASA scientists and NWS forecasters to improve severe stor m warnings using real-time data from a regional VHF total lightning mapping array (LMA). Key objectives of our research using LMA data ar e: a) Identification of intensifying and weakening storms using the time rate-of-change of total flash rate; b) Improved severe storm poten tial situational awareness; c) Evaluation of the potential of total f lash rate trend to improve severe storm probability of detection (POD ) and lead time; and d) Validation of mesoscale model forecasts of th understorm initiation. The LMA data are distributed for ingest and di splay in the WFO AWIPS decision support system, and archived at each WFO for case studies, event playbacks, and assessments using the NWS Warning Event Simulator. The Huntsville WFO has upgraded severe thund erstorm warnings to verified tornado warnings and avoided a false ala rm on a severe storm through the added information on storm growth, intensification, and decay that can be deduced from the magnitude and temporal trend of total flash rates. We present detailed case studies of the observed relationships between lightning activity and tornadi c storm development as determined by radar reflectivity and velocity fields, and thunderstorms forecast by the Weather Research and Foreca st (WRF) model. From these collaborative studies, forecasters can eva luate the added value of total lightning data within the forecast and warning decision-making process (http://weather.msfc.nasa.gov/sport) .

Goodman, S.↗
NASA NTRS2003

A National Virtual Specimen Database for Early Cancer Detection

Access to biospecimens is essential for enabling cancer biomarker discovery. The National Cancer Institute's (NCI) Early Detection Research Network (EDRN) comprises and integrates a large number of laboratories into a network in order to establish a collaborative scientific environment to discover and validate disease markers. The diversity of both the institutions and the collaborative focus has created the need for establishing cross-disciplinary teams focused on integrating expertise in biomedical research, computational and biostatistics, and computer science. Given the collaborative design of the network, the EDRN needed an informatics infrastructure. The Fred Hutchinson Cancer Research Center, the National Cancer Institute,and NASA's Jet Propulsion Laboratory (JPL) teamed up to build an informatics infrastructure creating a collaborative, science-driven research environment despite the geographic and morphology differences of the information systems that existed within the diverse network. EDRN investigators identified the need to share biospecimen data captured across the country managed in disparate databases. As a result, the informatics team initiated an effort to create a virtual tissue database whereby scientists could search and locate details about specimens located at collaborating laboratories. Each database, however, was locally implemented and integrated into collection processes and methods unique to each institution. This meant that efforts to integrate databases needed to be done in a manner that did not require redesign or re-implementation of existing system

distributed↗
NASA NTRS2007

A Contrast in Use of Metrics in Earth Science Data Systems

In recent years there has been a surge in the number of systems for processing, archiving and distributing remotely sensed data. Such systems, working independently as well as in collaboration, have been contributing greatly to the advances in the scientific understanding of the Earth system, as well as utilization of the data for nationally and internationally important applications. Among such systems, we consider those that are developed by or under the sponsorship of NASA to fulfill one of its strategic objectives: "Study Earth from space to advance scientific understanding and meet societal needs." NASA's Earth science data systems are of varying size and complexity depending on the requirements they are intended to meet. Some data systems are regarded as NASA's "Core Capabilities" that provide the basic infrastructure for processing, archiving and distributing a set of data products to a large and diverse user community in a robust and reliable manner. Other data systems constitute "Community Capabilities". These provide specialized and innovative services to data users and/or research products offering new scientific insight. Such data systems are generally supported by NASA through peer reviewed competition. Examples of Core Capabilities are 1. Earth Observing Data and Information System (EOSDIS) with its Distributed Active Archive Centers (DAACs), Science Investigator-led Processing Systems (SIPSs), and the EOS Clearing House (ECHO); 2. Tropical Rainfall Measurement Mission (TRMM) Science Data and Information System (TSDIS); 3. Ocean Data Processing System (ODPS); and 4. CloudSat Data Processing Center. Examples of Community Capabilities are projects under the Research, Education and Applications Solutions Network (REASON), and Advancing Collaborative Connections for Earth System Science (ACCESS) Programs. In managing these data system capabilities, it is necessary to have well-established goals and to measure progress relative to them. Progress is measured through "metrics", which can be a combination of quantitative as well as qualitative assessments. The specific metrics of interest depend on the user of the metrics as well as the type of data system. The users of metrics can be data system managers, program managers, funding agency or the public. Data system managers need metrics for assessing and improving the performance of the system and for future planning. Program managers need metrics to assess progress and the value of the data systems sponsored by them. Also, there is a difference in the metrics needed for core capabilities that tend to be more complex, larger and longer-term compared to community capabilities and the community capabilities that tend to be simpler, smaller and shorter-term. Even among community capabilities there are differences; hence the same set of metrics does not apply to all. Some provide data products to users, some provide services that enable better utilization of data or interoperability among other systems, and some are a part of a larger project where provision of data or services is only a minor activity. There is also a contrast between metrics used for internal and external purposes. Examples of internal purposes are: ensuring that the system meets its requirements, and planning for evolution and growth. Examples of external purposes are: providing to sponsors indicators of success of the systems, demonstrating the contributions of the system to overall program success, etc. This paper will consider EOSDIS, REASON and ACCESS programs to show the various types of metrics needed and how they need to be tailored to the types of data systems while maintaining the overall management goals of measuring progress and contributions made by the data systems.

Ramapriyan, Hampapuram↗
NASA NTRS2005

The Earth Science Afternoon Constellation: Preparing for Autonomous but Coordinated Operations

This paper describes how the challenges of coordinating the autonomous operations of geographically dispersed mission control centers for several small and large satellites are being overcome. The Earth Science Afternoon Constellation, also referred to as the "A-Train", is an international grouping of five NASA satellites (two major NASA EOS missions and three NASA/Earth System Science Pathfinder missions) and one French satellite orbiting in close proximity. This grouping of satellites provides scientists with the opportunity to perform coincident observations using data from two or more instruments on various satellites with measurements taken at approximately the same time. Three of the six missions are currently on-orbit, with the two missions expected to join the constellation later this year and one mission in 2007. The operational challenges are daunting for several reasons. There are several Mission Control Centers (widely separated on two continents), operating autonomously under tight budget constraints. All of the Mission Control Centers have reasons to be concerned about safety while flying in close proximity to other satellites, but most Centers did not have the resources or the desire to address this concern alone - the interfaces are too numerous and anticipated operations too costly. Clearly, an efficient approach was needed. This paper describes the steps taken to make this Earth science constellation a reality. Agreements were forged to allow the Mission Control Centers to maintain their autonomy, while ensuring their satellite's safety. Each member mission in the constellation operates independently in accordance with its own mission requirements, but the member missions have agreed to coordinate their operations, i.e., orbital positions and control to ensure the safety of the entire constellation. A centralized system was developed at NASA Goddard Space Flight Center to collect, analyze, and distribute ephemeris data used by each of the mission teams to determine the positions of the satellites in the constellation. The system issues warnings regarding possible dangerous configurations, eliminating the need for redundant capabilities at each Mission Control Center. On-orbit contingency situations were identified and analyzed; agreements were reached in advance of contingency operations to ensure that coordination between the Mission Control Centers can be handled expeditiously and fairly. In this manner, recovery from anomalous situations can be more quickly realized, thereby increasing the science return and reducing costs. The process used to develop these contingency procedures and the systems used to facilitate the contingency resolution are described as well.

Case, Warren↗
NASA NTRS2018

The CEOS Data Cube Portal: A User-Friendly, Open Source Software Solution for the Distribution, Exploration, Analysis, and Visualization of Analysis Ready Data

There is an urgent need to increase the capacity of developing countries to take part in the study and monitoring of their environments through remote sensing and space-based Earth observation technologies. The Open Data Cube (ODC) provides a mechanism for efficient storage and a powerful framework for processing and analyzing satellite data. While this is ideal for scientific research, the expansive feature space can also be daunting for end-users and decision-makers who simply require a solution which provides easy exploration, analysis, and visualization of Analysis Ready Data (ARD). Utilizing innovative web-design and a modular architecture, the Committee on Earth Observation Satellites (CEOS) has created a web-based user interface (UI) which harnesses the power of the ODC yet provides a simple and familiar user experience: the CEOS Data Cube (CDC). This paper presents an overview of the CDC architecture and the salient features of the UI. In order to provide adaptability, flexibility, scalability, and robustness, we leverage widely-adopted and well-supported technologies such as the Django web framework and the AWS Cloud platform. The fully-customizable source code of the UI is available at our public repository. Interested parties can download the source and build their own UIs. The UI empowers users by providing features that assist with streamlining data preparation, data processing, data visualization, and sub-setting ARD products in order to achieve a wide variety of Earth imaging objectives through an easy to use web interface.

User Interface↗
NASA NTRS1994

Directions in parallel programming: HPF, shared virtual memory and object parallelism in pC++

Fortran and C++ are the dominant programming languages used in scientific computation. Consequently, extensions to these languages are the most popular for programming massively parallel computers. We discuss two such approaches to parallel Fortran and one approach to C++. The High Performance Fortran Forum has designed HPF with the intent of supporting data parallelism on Fortran 90 applications. HPF works by asking the user to help the compiler distribute and align the data structures with the distributed memory modules in the system. Fortran-S takes a different approach in which the data distribution is managed by the operating system and the user provides annotations to indicate parallel control regions. In the case of C++, we look at pC++ which is based on a concurrent aggregate parallel model.

Bodin, Francois↗
NASA NTRS1983

Data systems - Optical bus will connect distributed system

The fiber optics buses currently under consideration for the NASA space station program will have data rates of at least 20 Mb/sec. The network interconnection scheme that will eventually be used is, however, uncertain. A NASA-Langley effort on graph networks includes adaptive, fault-tolerant and self-correcting operations. A breadboard program at NASA-Johnson includes evaluations of standard devices, called 'bus interface units', in an adaptive and fault-correcting chordal or concentric ring type of network. Fiber optics would allow an evolution to wavelength modulation techniques in such systems.

Swingle, W. L.↗