Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Complexity Computational Environment: Data Assimilation SERVOGrid

We are using Web (Grid) service technology to demonstrate the assimilation of multiple distributed data sources (a typical data grid problem) into a major parallel high-performance computing earthquake forecasting code. Such a linkage of Geoinformatics with Geocomplexity demonstrates the value of the Solid Earth Research Virtual Observatory (SERVO) Grid concept, and advance Grid technology by building the first real-time large-scale data assimilation grid Here we develop the next steps for both the SERVO concept and the identified need for a Solid Earth problem-solving environment. We use a challenging motivating problem of importance to NASA namely integrating NASA space geodetic observations with numerical simulations of a changing earth.

data assimiliation↗
NASA NTRS2004

QuakeSim and the Solid Earth Research Virtual Observatory

We are developing simulation and analysis tools in order to develop a solid Earth science framework for understanding and studying active tectonic and earthquake processes. The goal of QuakeSim and its extension, the Solid Earth Research Virtual Observatory (SERVO), is to study the physics of earthquakes using state-of-the-art modeling, data manipulation, and pattern recognition technologies. We are developing clearly defined accessible data formats and code protocols as inputs to simulations, which are adapted to high-performance computers. The solid Earth system is extremely complex and nonlinear resulting in computationally intensive problems with millions of unknowns. With these tools it will be possible to construct the more complex models and simulations necessary to develop hazard assessment systems critical for reducing future losses from major earthquakes. We are using Web (Grid) service technology to demonstrate the assimilation of multiple distributed data sources (a typical data grid problem) into a major parallel high-performance computing earthquake forecasting code. Such a linkage of Geoinformatics with Geocomplexity demonstrates the value of the Solid Earth Research Virtual Observatory (SERVO) Grid concept, and advances Grid technology by building the first real-time large-scale data assimilation grid.

virtual observatory↗
NASA NTRS2021

An Evaluation of Timely Communications Access Methods Using NASA Space Network

There is a growing interest among scientists to coordinate diverse and temporally responsive measurements from multiple space and ground-based observatories in order to obtain greater insights into transient scientific events than would otherwise be possible. Many transient scientific events of interest occur randomly and the scientific value of follow-up observations decays with time. As a result, transient science operations among distributed observatories require timely access to communications network services. This paper develops two complimentary methods to improve users’ network access timeliness using the NASA Space Network (SN). Descriptive models of each method are developed using Systems Modeling Language. Pathfinder experiments are designed and executed. Suitability of the methods is discussed in light of the experimental results. Innovative applications of the methods are presented. This research establishes that the as-built SN systems have adequate capabilities and capacity to support infusion of the methods to current and future users with emergent time-sensitive service needs. The minimum service access wait time was determined to be approximately 10 min and is achievable 84% of the time. An average access wait time of approximately 14 min is expected when SN resource blocking occurs. A roadmap to improve SN access timeliness is presented.

Space Communications and Navigation↗
NASA NTRSDate not supplied

North American Land Data Assimilation (NLDAS) Data and Services at NASA GES DISC

The North American Land Data Assimilation System (NLDAS, https://ldas.gsfc.nasa.gov/nldas) integrates a large quantity of observation-based and model reanalysis data to drive offline (not coupled to the atmosphere) land surface models (LSMs) to produce fields of soil moisture, snow, and surface fluxes. The NLDAS-2 data sets currently archived at, and distributed by, the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC, https://disc.gsfc.nasa.gov) have been updated to NLDAS Version 2.0 (NLDAS-2.0). This presentation provides an overview of the updated NLDAS-2.0 data at NASA GES DISC and discusses the methods for the public to access the data as well as the data services available.

Ashley Heath↗
NASA NTRSDate not supplied

GLDAS-2 Land Surface Model Data and Data Services at NASA GES DISC

The goal of the NASA Global Land Data Assimilation System (GLDAS, https://ldas.gsfc.nasa.gov/gldas(https://ldas.gsfc.nasa.gov/gldas)) is to generate optimal fields of land surface states and fluxes by ingesting satellite- and ground-based observational data products, using advanced land surface modeling and data assimilation techniques (Rodell et al., 2004).The GLDAS dataset currently archived at and distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC, https://disc.gsfc.nasa.gov/ (https://disc.gsfc.nasa.gov/)) is GLDAS Version 2 (GLDAS-2). It contains a series of output fields from the upgraded Noah-3.6, Catchment-F2.5, and VIC-4.1.2 Land Surface Models (LSMs) in the Land Information System (LIS-V7, https://lis.gsfc.nasa.gov/ (https://lis.gsfc.nasa.gov/)). GLDAS-2 has three components:GLDAS-2.0, GLDAS-2.1, and GLDAS-2.2. GLDAS-2.0 is forced entirely with the upgraded Princeton Meteorological ForcingV2.2 Dataset and provides a temporally consistent series from 1948 through 2014. GLDAS-2.1 is forced with a combination of model and observation data, with data spanning from 2000 to the present. The GLDAS-2.2 product suite uses data assimilation(DA), whereas the GLDAS-2.0 and GLDAS-2.1 products are "open-loop" (i.e., no data assimilation). The choice of forcing data, as well as DA observation source, variable, and scheme, varies for different GLDAS-2.2 products. The currently availableGLDAS-2.2 data contain a daily 0.25-degree output from the Catchment-F2.5 LSM in LIS-V7. The data are forced with the meteorological analysis fields from the operational European Centre for Medium-Range Weather Forecasts Integrated Forecasting System (ECMWF-IFS) and assimilated with GRACE and GRACE-FO data, ranging from February 1, 2003 to the present. The current GLDAS-2.0 and 2.1 Noah LSM data were reprocessed in November 2019 and January 2020 respectively and their data from Catchment and VIC LSMs are new to the GLDAS-2 collection. This presentation provides a summary of theGLDAS-2 data products, their land surface fields, and their related data services at the GES DISC; and a description of the majorGLDAS-2 climatological characteristics as well as the intercomparison with the data of the previous version.

Hydrology↗
NASA NTRS2011

Using Open and Interoperable Ways to Publish and Access LANCE AIRS Near-Real Time Data

The Atmospheric Infrared Sounder (AIRS) Near-Real Time (NRT) data from the Land Atmosphere Near real-time Capability for EOS (LANCE) element at the Goddard Earth Sciences Data and Information Services Center (GES DISC) provides information on the global and regional atmospheric state, with very low temporal latency, to support climate research and improve weather forecasting. An open and interoperable platform is useful to facilitate access to, and integration of, LANCE AIRS NRT data. As Web services technology has matured in recent years, a new scalable Service-Oriented Architecture (SOA) is emerging as the basic platform for distributed computing and large networks of interoperable applications. Following the provide-register-discover-consume SOA paradigm, this presentation discusses how to use open-source geospatial software components to build Web services for publishing and accessing AIRS NRT data, explore the metadata relevant to registering and discovering data and services in the catalogue systems, and implement a Web portal to facilitate users' consumption of the data and services.

Zhao, Peisheng↗
NASA NTRS2007

Using Virtual Observatory Services in Sky View

For over a decade Skyview has provided astronomers and the public with easy access to survey and imaging data from all wavelength regimes. SkyView has pioneered many of the concepts that underlie the Virtual Observatory. Recently SkyView has been released as a distributable package which uses VO protocols to access image and catalog services. This chapter describes how to use the Skyview as a local service and how to customize it to access additional VO services and local data.

McGlynn, Thomas A.↗
NASA NTRS2004

Value-added Data Services at the Goddard Earth Sciences Data and Information Services Center

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), in addition to serving the Earth Science community as one of the major Distributed Active Archives Centers (DAACs), provides much more than just data. Among the value-added services available to general users are subsetting data spatially and/or by parameter, online analysis (to avoid downloading unnecessarily all the data), and assistance in obtaining data from other centers. Services available to data producers and high-volume users include consulting on building new products with standard formats and metadata and construction of data management systems. A particularly useful service is data processing at the DISC (i.e., close to the input data) with the users algorithm. This can take a number of different forms: as a configuration-managed algorithm within the main processing stream; as a stand-alone program next to the on-line data storage; as build-it-yourself code within the Near-Archive Data Mining (NADM) system; or as an on-the-fly analysis with simple algorithms embedded into the web-based tools. Partnerships between the GES DISC and scientists, both producers and users, allow the scientists to concentrate on science, while the GES DISC handles the data management, e.g., formats, integration, and data processing. The existing data management infrastructure at the GES DISC supports a wide spectrum of options: from simple data support to sophisticated on-line analysis tools, producing economies of scale and rapid time-to-deploy. At the same time, such partnerships allow the GES DISC to serve the user community more efficiently and to better prioritize on-line holdings. Several examples of successful partnerships are described in the presentation.

Leptoukh, Gregory G.↗
NASA NTRS2008

The User Community and a Multi-Mission Data Project: Services, Experiences and Directions of the Space Physics Data Facility

From a user's perspective, the multi-mission data and orbit services of NASA's Space Physics Data Facility (SPDF) project offer a unique range of important data and services highly complementary to other services presently available or now evolving in the international heliophysics data environment. The VSP (Virtual Space Physics Observatory) service is an active portal to a wide range of distributed data sources. CDAWeb (Coordinate Data Analysis Web) enables plots, listings and file downloads for current data cross the boundaries of missions and instrument types (and now including data from THEMIS and STEREO). SSCWeb, Helioweb and our 3D Animated Orbit Viewer (TIPSOD) provide position data and query logic for most missions currently important to heliophysics science. OMNIWeb with its new extension to 1- and 5-minute resolution provides interplanetary parameters at the Earth's bow shock as a unique value-added data product. SPDF also maintains NASA's CDF (common Data Format) standard and a range of associated tools including translation services. These capabilities are all now available through webservices-based APIs as well as through our direct user interfaces. In this paper, we will demonstrate the latest data and capabilities now supported in these multi-mission services, review the lessons we continue to learn in what science users need and value in this class of services, and discuss out current thinking to the future role and appropriate focus of the SPDF effort in the evolving and increasingly distributed heliophysics data environment.

Fung, Shing F.↗
NASA NTRS2018

A Concept of Operations for Earth Science Data Archive and Distribution in the Cloud

Science data systems can enable more comprehensive Earth system research by evolving to take advantage of advances in commercial computer technology services. Since their inception twenty five years ago, NASA's Earth Observing System Data and Information System (EOSDIS) Distributed Active Archive Centers (DAACs) have periodically evolved to utilize new technology and expand research using the exponential growth and diversity of Earth observations. Recently, with the advent of a maturing commercial compute services industry and upcoming high data volume missions such as the Surface Water and Ocean Topography (SWOT) mission and the NASA-Indian Space Research Organization Synthetic Aperture Radar (NISAR) mission, options were explored and a decision made to utilize commercial compute and storage services. This paper presents an overview of the concept of operations under development for the DAACs in the Cloud. We highlight the goals and expected advantages of utilizing Cloud services. We outline EOSDIS operations tenets and driving principles. A high-level view of EOSDIS system of systems target architecture serves as context for describing principle interactions. Concepts for key DAAC system and EOSDIS enterprise functions characterize automated end-to-end operations but mark nominal check and recovery points. Concepts are presented for managing Cloud resources, including organizational roles and responsibilities of the NASA project and DAAC personnel. Scenarios we use to further distinguish between what the system will do and what configuration and controls operators will have. Examples include interactions with data providers and data consumers with both in-cloud and on-premise facilities.

Moses, John F.↗
NASA NTRS2008

Services, Perspective and Directions of the Space Physics Data Facility

The multi-mission data and orbit services of NASA's Space Physics Data Facility (SPDF) project offer unique capabilities supporting science of the Heliophysics Great Observatory and that are highly complementary to other services now evolving in the international heliophysics data environment. The VSPO (Virtual Space Physics Observatory) service is an active portal to a wide rage of distributed data sources. CDAWeb (Coordinated Data Analysis Web) offers plots, listings and file downloads for current data from many missions across the boundaries of missions and instrument types. CDAWeb now includes extensive new data from STEREO and THEMIS, plus new ROCSAT IPEI data, the latest data from all four TIMED instruments and high-resolution data from all DE-2 experiments. SSCWeb, Helioweb and out 3D Animated Orbit Viewer (TIPSOD) provide position data and identification of spacecraft and ground conjunctions. OMNI Web, with its new extension to 1- and 5-minute resolution, provides interplanetary parameters at the Earth's bow shock. SPDF maintains NASA's CDF (Common Data Format) standard and a range of associated tools including format translation services. These capabilities are all now available through web services based APIs, one element in SPDF's ongoing work to enable heliophysics community development of Virtual discipline Observatories (e.g. VITMO). We will demonstrate out latest data and capabilities, review the lessons we continue to learn in what science users need and value in this class of services, and discuss out current thinking to the future role and appropriate focus of the SPDF effort in the evolving and increasingly distributed heliophysics data environment.

McGuire, Robert E.↗
NASA NTRS2018

The InSAR Scientific Computing Environment 3.0: A Flexible Framework for NISAR Operational and User-Led Science Processing

The InSAR Scientific Computing Environment (ISCE) was first developed under the NASA Advanced Information Systems Technology as a flexible, extensible object-oriented framework for Interferometric Synthetic Aperture Radar (InSAR) processing. The ISCE framework uses Python 3 at the workflow level, controlling modules of compiled code for functional processing, and managing inputs, outputs, and other flow control services. The currently released version, called ISCE 2.1, is distributed to the research community through the Western North America InSAR Consortium under a research license. The ISCE team is working on the next generation of the code in order to prepare for the NASAISRO SAR (NISAR) mission operational processing. Innovations in this code include augmentation or conversion of the custom Python framework elements in ISCE with the Pyre framework, new workflows for interferometric and polarimetric stack processing, a more intuitive and graphically based user interface, and flow control for hybrid computing environments including CPU/GPU clusters, logging and error tracking facilities, and new more efficient computational modules that exploit graphical processor units (GPUs) when available. The ISCE 3.0 framework is designed to work in an operational environment as well as on a single user’s laptop or compute cluster, with services to discover capabilities and scale computations accordingly.

Buckley, Sean M.↗
NASA NTRSDate not supplied

What (and How) MERRA-2 Reanalysis Data are Used in Applied Sciences

The Modern Era Retrospective-analysis for Research and Applications, Version 2 (MERRA-2) is the global atmospheric data reanalysis for the satellite era produced by NASA’s Global Modeling and Assimilation Office (GMAO), using the Goddard Earth Observing System Model (GEOS)version 5.12.4. The data are officially distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). MERRA-2 data have been widely used by the Earth sciences and application community. Since MERRA-2 data were released in early 2016, the number of registered data users has grown steadily from 1,252 in 2016 to 6477 in 2020. By the end of October 2021, ~16 petabytes (over 360 million files) of data have been distributed to more than 18,900 users. Searching in Google Scholar (https://scholar.google.com/), we have found over 7,000 articles, published between January 2017 and May 2021, involving the use ofMERRA-2 data. The figure shows the numbers for various application areas in which theMERRA-2 data have been used, covering almost all of the application areas defined in NASA Applied Sciences (http://appliedsciences.nasa.gov). The largest number of articles are found in disaster research, with the subcategories ordered in flood, wildfires, hurricanes and cyclones, and other forms of severe weather. In this presentation, we will discuss the preliminary findings from a review of the selected literature that uses MERRA-2 data in applied sciences. The current analytic and interoperable data services at GES DISC are listed, such as the on-the-fly subset and analysis service, NASA Giovanni; THREDDS Data Server(TDS); and Python Jupyter notebooks. In addition, we will introduce two new services for supporting the open sciences: My Dashboard and Related Publications.

data management↗
NASA NTRS2017

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗
NASA NTRS2008

Modern Era Retrospective Restrospective-Analysis for Research and Applications (MERRA) Data and Services at the GES DISC

The Modern Era Retrospective-analysis for Research and Applications (MERRA) dataset is a NASA satellite era, 30 year (1979 - present), reanalysis using the Goddard Earth Observing System Data Assimilation System, Version 5 (GEOS-5). The project, run out of NASA's Global Modeling and Assimilation Office at Goddard Space Flight Center, provides the science and application communities with a state-of-the-art global analysis with emphasis on improved estimates of the hydrological cycle over a broad range of weather and climate time scales. MERRA products are generated as a long-term synthesis that places the NASA EOS suite of observations in a climate context. The MERRA analysis is performed at a horizontal resolution of 2/3 longitude x 1/2 latitude (540x361 global gridpoints) with observational analyses every 6 hours. The MERRA output data will include 3 dimensional state fields for every 6 hourly analysis cycle on 42 pressure levels (or 72 terrain following model coordinate levels) from the surface through the stratosphere. Several data products are specifically designed to support chemistry and stratosphere transport modeling. The 2 dimensional surface and atmospheric diagnostics (numbering 259) are being stored on the native grid at 1 hourly intervals. These include radiation and vertical integrals of the atmosphere for water and energy budget studies and also surface diagnostics where the diurnal cycle is important. The one hourly surface and near surface data product will also facilitate research on the integrated analysis of Earth system observations in the land, ocean and cryosphere. The MERRA products are archived and distributed by the Goddard Earth Sciences Data and Information Services Center (GES DISC) through its Modeling DISC Web (MDISC) portal. Multiple data access methods and services are available for MERRA data through MDISC: (1) Mirador offers a quick, comprehensive search of MERRA and all GES DISC archived data holdings, allowing searches on keywords, location names or latitude/longitude box, and date/time, with responses within a few seconds. (2) Giovanni is a GES DISC developed Web application that provides data visualization and analysis online. Giovanni features popular visualizations such as latitude-longitude maps, animations, cross sections, profiles, time series, etc. and some basic statistical analysis functions such as scatter plots and correlation coefficient maps. Users are able to download results in several different formats, including Google Earth. (3) On-the-fly parameter subsetting of data within a spatial/temporal window is provided through a simple select and click Web page. (4) MERRA data are also available via OPeNDAP, GrADS Data Server (GDS) and can be converted to netCDF on the fly.

Berrick, Stephen W.↗
NASA NTRS1990

The multidriver: A reliable multicast service using the Xpress Transfer Protocol

A reliable multicast facility extends traditional point-to-point virtual circuit reliability to one-to-many communication. Such services can provide more efficient use of network resources, a powerful distributed name binding capability, and reduced latency in multidestination message delivery. These benefits will be especially valuable in real-time environments where reliable multicast can enable new applications and increase the availability and the reliability of data and services. We present a unique multicast service that exploits features in the next-generation, real-time transfer layer protocol, the Xpress Transfer Protocol (XTP). In its reliable mode, the service offers error, flow, and rate-controlled multidestination delivery of arbitrary-sized messages, with provision for the coordination of reliable reverse channels. Performance measurements on a single-segment Proteon ProNET-4 4 Mbps 802.5 token ring with heterogeneous nodes are discussed.

Dempsey, Bert J.↗
NASA NTRS1995

A high performance totally ordered multicast protocol

This paper presents the Reliable Multicast Protocol (RMP). RMP provides a totally ordered, reliable, atomic multicast service on top of an unreliable multicast datagram service such as IP Multicasting. RMP is fully and symmetrically distributed so that no site bears un undue portion of the communication load. RMP provides a wide range of guarantees, from unreliable delivery to totally ordered delivery, to K-resilient, majority resilient, and totally resilient atomic delivery. These QoS guarantees are selectable on a per packet basis. RMP provides many communication options, including virtual synchrony, a publisher/subscriber model of message delivery, an implicit naming service, mutually exclusive handlers for messages, and mutually exclusive locks. It has commonly been held that a large performance penalty must be paid in order to implement total ordering -- RMP discounts this. On SparcStation 10's on a 1250 KB/sec Ethernet, RMP provides totally ordered packet delivery to one destination at 842 KB/sec throughput and with 3.1 ms packet latency. The performance stays roughly constant independent of the number of destinations. For two or more destinations on a LAN, RMP provides higher throughput than any protocol that does not use multicast or broadcast.

Montgomery, Todd↗
NASA NTRS1994

Cascaded clocks measurement and simulation findings

This paper will examine aspects related to network synchronization distribution and the cascading of timing elements. Methods of timing distribution have become a much debated topic in standards forums and among network service providers (both domestically and internationally). Essentially these concerns focus on the need to migrate their existing network synchronization plans (and capabilities) to those required for the next generation of transport technologies (namely, the Synchronous Digital Hierarchy (SDH), Synchronous Optical Networks (SONET), and Asynchronous Transfer Mode (ATM). The particular choices for synchronization distribution network architectures are now being evaluated and are demonstrating that they can indeed have a profound effect on the overall service performance levels that will be delivered to the customer. The salient aspects of these concerns reduce to the following: (1) identifying that the devil is in the details of the timing element specifications and the distribution of timing information (i.e., small design choices can have a large performance impact); (2) developing a standardized method of performance verification that will yield unambiguous results; and (3) presentation of those results. Specifically, this will be done for two general cases: an ideal input, and a noisy input to a cascaded chain of slave clocks.

Chislow, Don↗