Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Serving NASA GES DISC Multi-Spatiotemporal Earth Science Data to the GIS community

NASA Earth Science (ES) data is essential to a wide range of GIS research and applications. However, for many GIS users, searching, accessing, using and analyzing NASA ES data can be of a great challenge- ranging from the sheer data volumes, types of science parameters, and to the complexity of data encoding formats. As one of the twelve NASA Science Mission Directorate (SMD) Data Centers, Goddard Earth Sciences (GES) Data and Information Services Center (DISC) archives and distributes petabytes of ES parameters covering atmosphere, land, and ocean fields. Most data are multidimensional and multi-spatiotemporal in nature and are encoded in different science data formats (e.g, HDF, HDF-EOS, netCDF, GRIB, binary), which usually contain multiple variables and different metadata information. By far, GES DISC has been developing a number of services and online tools to help GIS users to easily explore our data products. In this presentation, we will describe our ArcGIS-based data accessing and visualization services and portals, which allow users directly exploring the multi-spatiotemporal ES data in ArcGIS clients without having to pre-download/import the data. The ArcGIS services are also compliant with the Open Geospatial Consortium (OGC) Web Coverage Service (WCS) and Web Map Service (WMS) protocols and can be accessed by any other WCS/WMS clients to get customized GES DISC EO data on-the-fly from such services.

Wei, Jennifer↗

The EOSDIS Version 0 Distributed Active Archive Center for physical oceanography and air-sea interaction

The Distributed Active Archive Center (DAAC) at the Jet Propulsion Laboratory will support scientists specializing in physical oceanography and air-sea interaction. As part of the NASA Earth Observing System Data and Information System Version 0 the DAAC will build on existing capabilities to provide services for data product generation, archiving, distribution and management of information about data. To meet scientist's immediate needs for data, existing data sets from missions such as Seasat, Geosat, the NOAA series of satellites and the Global Positioning Satellite system will be distributed to investigators upon request. In 1992, ocean topography, wave and surface roughness data from the Topex/Poseidon radar altimeter mission will be archived and distributed. New data products will be derived from Topex/Poseidon and other sensor systems based on recommendations of the science community. In 1995, ocean wind field measurements from the NASA Scatterometer will be supported by the DAAC.

Hilland, Jeffrey E.↗

Global Data Assembly Center (GDAC) report to the GHRSST Science Team

In 2017-2018 the Global Data Assembly Center (GDAC) at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) provided ingest, archive, distribution and user services for GHRSST operational data streams with improved and evolved tools, services, and tutorials and interfaced with the user community to address technical inquiries. The GDAC provided access to new GHRSST datasets as well retired a significant number of deprecated GHRSST datasets from its discovery services. The following sections summarize and document the specific achievements of the GDAC to the GHRSST community.

Finch, Chris↗

Global Data Assembly Center (GDAC) Report to the GHRSST Science Team

In 2017-2018 the Global Data Assembly Center (GDAC) at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) provided ingest, archive, distribution and user services for GHRSST operational data streams with improved and evolved tools, services, and tutorials and interfaced with the user community to address technical inquiries. The GDAC provided access to new GHRSST datasets as well retired a significant number of deprecated GHRSST datasets from its discovery services. The following sections summarize and document the specific achievements of the GDAC to the GHRSST community.

Finch, Chris↗

Global Data Assembly Center (gdac) Report to the GHRSST Science Team

In 2019-2020 the Global Data Assembly Center (GDAC) at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) provided ingest, archive, distribution and user services for GHRSST operational data streams with improved and evolved tools, services, and tutorials, and interfaced with the user community to address technical inquiries. Several new GHRSST datasets including reprocessed MODIS Aqua/Terra L2P with improved cloud screening were made available. The PO.DAAC participated in the evolving GHRSST data management re architecture and development activities. GHRSST Science Team member Edward Armstrong assumed the role of co-leader of the CEOS SST Virtual Constellation. The following sections summarize and document the specific achievements of the GDAC to the GHRSST community.

Finch, Chris↗

NASA GES DISC DAAC Satellite Data for GIS

NASA's Goddard Earth Science (GES) Data and Information Services Center (DISC) Distributed Active Archive Center (DAAC) makes available a large and continually growing collection of spatially continuous global satellite observations of environmental parameters. These products include those from the MODIS (Moderate Resolution Imaging Spectroradiometer) on both Terra and Aqua platforms, and the Tropical Rainfall Measuring Mission (TRMM). These data products are well suited for use within Geographic Information Systems (GIS), as both backdrops to cartographic products as well as spatial analysis. However, data format, file size, and other issues have limited their widespread use by traditional GIS users. To address these data usability issues, the GES DISC DAAC recently updated tools and improved documentation of conversion procedures. In addition, the GES DISC DAAC has also been working with a major GIS software vendor to incorporate the ability to read the native Hierarchial Data Format (HDF), the format in which most of the NASA data is stored. The result is the enabling of GIS users to realize the benefit of GES DISC DAAC data without a substantial expenditure in resources to incorporate these data into their GIS. Several documents regarding the potential uses of GES DISC DAAC satellite data in GIS have recently been created. These show the combinations of concurrent data from different satellite products with traditional GIS vector products for given geographic areas. These map products include satellite imagery of Hurricane Isabel and the California wildfires, and can be viewed at http://daac.gsfc.nasa.gov/MODIS/GIS/.

Nickless, Darryl↗

Space Physics Data Facility Web Services

The Space Physics Data Facility (SPDF) Web services provides a distributed programming interface to a portion of the SPDF software. (A general description of Web services is available at http://www.w3.org/ and in many current software-engineering texts and articles focused on distributed programming.) The SPDF Web services distributed programming interface enables additional collaboration and integration of the SPDF software system with other software systems, in furtherance of the SPDF mission to lead collaborative efforts in the collection and utilization of space physics data and mathematical models. This programming interface conforms to all applicable Web services specifications of the World Wide Web Consortium. The interface is specified by a Web Services Description Language (WSDL) file. The SPDF Web services software consists of the following components: 1) A server program for implementation of the Web services; and 2) A software developer s kit that consists of a WSDL file, a less formal description of the interface, a Java class library (which further eases development of Java-based client software), and Java source code for an example client program that illustrates the use of the interface.

Candey, Robert M.↗

Creating a knowledge graph to connect scientific publications and datasets for improving discovery of GES DISC’s data and services

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) archives and distributes to the public hundreds of Earth Science data collections. These collections are used in research, resulting in thousands of scientific papers published each year. As new users come to GES DISC for the data, it is important for them to understand how these data were used in the prior research. For this we are creating the Knowledge Graph that connects research paper citation and the data collection metadata. The relationships created in the graph have potential for the Web applications that utilize this information to directly connect the paper research to the GES DISC datasets and services. We will demonstrate these relationships using the Web application prototype.

Nathaniel Ross Crosby↗

Virtual Solar Observatory Distributed Query Construction

Through a prototype implementation (Tian et al., this meeting) the VSO has already demonstrated the capability of unifying geographically distributed data sources following the Web Services paradigm and utilizing mechanisms such as the Simple Object Access Protocol (SOAP). So far, four participating sites (Stanford, Montana State University, National Solar Observatory and the Solar Data Analysis Center) permit Web-accessible, time-based searches that allow browse access to a number of diverse data sets. Our latest work includes the extension of the simple, time-based queries to include numerous other searchable observation parameters. For VSO users, this extended functionality enables more refined searches. For the VSO, it is a proof of concept that more complex, distributed queries can be effectively constructed and that results from heterogeneous, remote sources can be synthesized and presented to users as a single, virtual data product.

Gurman, J. B.↗

Collecting and Processing Earth Science Data Metrics at NASA ESDIS

Since the launch of Terra satellite in 1999, the number of Earth Science remote sensing data products created and distributed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) has increased from a few hundred to nearly ten thousand. NASA's Earth Science Data and Information System (ESDIS) Metrics System (EMS) collects metrics on data ingest, archive, and distribution by its Distributed Active Archive Centers (DAACs) and the Science Investigator-led Systems (SIPS), known as Data Providers. These metrics are critical in helping NASA management as well as data producers in resource planning and gaining a wide range of knowledge of data users and data usage.EMS receives flat files, or log files of data archive, ingest, and distribution either in their raw format, such as Apache web logs, or text files of log records formatted by the Data Providers. Tens of millions of records are processed each day to extract metrics on data products, user information, distribution protocols and services, and so on. The metrics are then made available to designated parties.This presentation provides an overview of the EMS processing workflow and improvement efforts made in recent years to handle ever-increasing number of data records and new metrics requirements, discusses several key steps including mapping log records to data products and identifying user communities along with geo-distribution, and demonstrates typical metrics capabilities produced by the EMS system. Challenges and potential approaches to improve the system are also discussed.

Pan, Jianfu↗

Diaspora: Resilience-enabling services for science from HPC to edge

Scientific applications of interest to DOE must increasingly engage distributed resources (e.g., instruments, remote computers, data stores, edge devices) and deliver more stringent levels of service (e.g., uninterrupted processing of experiment data streams). In such systems, state is distributed and components can fail in many ways, often silently, making application resilience a major concern. Addressing the resilience needs of such applications requires methods for gaining knowledge of resources and applications and for translating that knowledge into action. We are working on addressing these needs in the context of multi-messenger astronomy, where detecting and responding to unusual transient events in multiple cosmic messengers (gravitational wave, electromagnetic, high- energy particles) from different instruments leads to a federated learning problem.

47 OTHER INSTRUMENTATION↗

NSSDC and WDC-A-R/S document availability and distribution services

Documents available from the National Space Science Data Center (NSSDC) and the World Data Center A for Rockets Satellites are described. The availability, costs, ordering procedures for documents presently available, and the procedures for obtaining future documents are given. NSSDC, established by NASA to further the widest practicable use of reduced data obtained from space science investigations and to provide investigators with an active repository for such data, is responsible for the active collection, organization, storage, announcement, retrieval, dissemination, and exchange of data received from satellite experiments. Information on sounding rocket investigations is also collected.

Source record↗

NSSDC and WDC-A-R and S document availability and distribution services

Documents available from the national space science data center (NSSDC) and the world data center A for rockets and satellites (WDC-A-R&S), are described. The availability, costs, ordering procedures for documents presently available, and the procedures for obtaining future documents are given.

Source record↗

Complexity Computational Environment: Data Assimilation SERVOGrid

We are using Web (Grid) service technology to demonstrate the assimilation of multiple distributed data sources (a typical data grid problem) into a major parallel high-performance computing earthquake forecasting code. Such a linkage of Geoinformatics with Geocomplexity demonstrates the value of the Solid Earth Research Virtual Observatory (SERVO) Grid concept, and advance Grid technology by building the first real-time large-scale data assimilation grid Here we develop the next steps for both the SERVO concept and the identified need for a Solid Earth problem-solving environment. We use a challenging motivating problem of importance to NASA namely integrating NASA space geodetic observations with numerical simulations of a changing earth.

data assimiliation↗

Reasoning Service Exemplars for NASA’s Data and Reasoning Fabric

Future operations for Urban and Advanced Air Mobility are enabled by a distributed network of reliable and secured data and reasoning services referred to here as a fabric. In the aggregate, such a system must be all encompassing and mission agnostic, but specific use-cases are still needed to improve understanding and drive design paradigms. For this purpose, three reasoning service exemplars for Target Selection and Routing, Trajectory Generation, and Battery Health Management were developed and integrated into a specific NASA proposed data and reasoning fabric. These services were then used to build a mission reasoning application for lightning strike reconnaissance developed in collaboration with the Civil Air Patrol. Autonomous mission execution was then demonstrated using a multivehicle simulation platform with a full envelope 6 Degree-of-Freedom dynamics model for a concept electric Vertical Takeoff and Landing aircraft.

Autonomy↗

Middleware Case Study: MeDICi

In many application domains in science and engineering, data produced by sensors, instruments and networks is naturally processed by software applications structured as a pipeline . Pipelines comprise a sequence of software components that progressively process discrete units of data to produce a desired outcome. For example, in a Web crawler that is extracting semantics from text on Web sites, the first stage in the pipeline might be to remove all HTML tags to leave only the raw text of the document. The second step may parse the raw text to break it down into its constituent grammatical parts, such as nouns, verbs and so on. Subsequent steps may look for names of people or places, interesting events or times so documents can be sequenced on a time line. Each of these steps can be written as a specialized program that works in isolation with other steps in the pipeline. In many applications, simple linear software pipelines are sufficient. However, more complex applications require topologies that contain forks and joins, creating pipelines comprising branches where parallel execution is desirable. It is also increasingly common for pipelines to process very large files or high volume data streams which impose end-to-end performance constraints. Additionally, processes in a pipeline may have specific execution requirements and hence need to be distributed as services across a heterogeneous computing and data management infrastructure. From a software engineering perspective, these more complex pipelines become problematic to implement. While simple linear pipelines can be built using minimal infrastructure such as scripting languages, complex topologies and large, high volume data processing requires suitable abstractions, run-time infrastructures and development tools to construct pipelines with the desired qualities-of-service and flexibility to evolve to handle new requirements. The above summarizes the reasons we created the MeDICi Integration Framework (MIF) that is designed for creating high-performance, scalable and modifiable software pipelines. MIF exploits a low friction, robust, open source middleware platform and extends it with component and service-based programmatic interfaces that make implementing complex pipelines simple. The MIF run-time automatically handles queues between pipeline elements in order to handle request bursts, and automatically executes multiple instances of pipeline elements to increase pipeline throughput. Distributed pipeline elements are supported using a range of configurable communications protocols, and the MIF interfaces provide efficient mechanisms for moving data directly between two distributed pipeline elements.

99 GENERAL AND MISCELLANEOUS↗

NASA GES DISC's Customized Services for Climatology and Meteorology

At the NASA Goddard Earth Sciences (GES) Data and Information Service Center (DISC), we have archived and distributed more than 2,400 Earth science data products, from different missions or projects containing more than 100 M data files/granules with a total volume size nearly 2 PB that broadly serve user needs in science areas such as Atmospheric Composition, Water & Energy Cycles and Climate Variability. To date, GES DISC has developed many pertinent services to facilitate the usage of data products by our research communities, represented by approximately 24,000 registered users. We are facing the big data with increasingly archival volume and data types, moreover, we also encounter increasing users' demands and the demands are more diversified. It is still a challenge for us to better understand exactly what our users' needs are, even after developing more than 70 services, including well-known online tools such as Giovanni and MERRA subsetter. In this presentation, we will try to address how we can accommodate the users' needs from two applicational user communities, Air Quality and Wind Energy, from data or service discovery to guide them properly utilize the data and services to fit their needs.

customizable services for climate and meteorology↗

The User Community and a Multi-Mission Data Project: Services, Experiences and Directions of the Space Physics Data Facility

From a user's perspective, the multi-mission data and orbit services of NASA's Space Physics Data Facility (SPDF) project offer a unique range of important data and services highly complementary to other services presently available or now evolving in the international heliophysics data environment. The VSP (Virtual Space Physics Observatory) service is an active portal to a wide range of distributed data sources. CDAWeb (Coordinate Data Analysis Web) enables plots, listings and file downloads for current data cross the boundaries of missions and instrument types (and now including data from THEMIS and STEREO). SSCWeb, Helioweb and our 3D Animated Orbit Viewer (TIPSOD) provide position data and query logic for most missions currently important to heliophysics science. OMNIWeb with its new extension to 1- and 5-minute resolution provides interplanetary parameters at the Earth's bow shock as a unique value-added data product. SPDF also maintains NASA's CDF (common Data Format) standard and a range of associated tools including translation services. These capabilities are all now available through webservices-based APIs as well as through our direct user interfaces. In this paper, we will demonstrate the latest data and capabilities now supported in these multi-mission services, review the lessons we continue to learn in what science users need and value in this class of services, and discuss out current thinking to the future role and appropriate focus of the SPDF effort in the evolving and increasingly distributed heliophysics data environment.

Fung, Shing F.↗