Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

The D3 Middleware Architecture

DARWIN is a NASA developed, Internet-based system for enabling aerospace researchers to securely and remotely access and collaborate on the analysis of aerospace vehicle design data, primarily the results of wind-tunnel testing and numeric (e.g., computational fluid-dynamics) model executions. DARWIN captures, stores and indexes data; manages derived knowledge (such as visualizations across multiple datasets); and provides an environment for designers to collaborate in the analysis of test results. DARWIN is an interesting application because it supports high-volumes of data. integrates multiple modalities of data display (e.g., images and data visualizations), and provides non-trivial access control mechanisms. DARWIN enables collaboration by allowing not only sharing visualizations of data, but also commentary about and views of data. Here we provide an overview of the architecture of D3, the third generation of DARWIN. Earlier versions of DARWIN were characterized by browser-based interfaces and a hodge-podge of server technologies: CGI scripts, applets, PERL, and so forth. But browsers proved difficult to control, and a proliferation of computational mechanisms proved inefficient and difficult to maintain. D3 substitutes a pure-Java approach for that medley: A Java client communicates (though RMI over HTTPS) with a Java-based application server. Code on the server accesses information from JDBC databases, distributed LDAP security services, and a collaborative information system. D3 is a three tier-architecture, but unlike 'E-commerce' applications, the data usage pattern suggests different strategies than traditional Enterprise Java Beans - we need to move volumes of related data together, considerable processing happens on the client, and the 'business logic' on the server-side is primarily data integration and collaboration. With D3, we are extending DARWIN to handle other data domains and to be a distributed system, where a single login allows a user transparent access to test results from multiple servers and authority domains.

Walton, Joan↗

Satellite provided customer premises services: A forecast of potential domestic demand through the year 2000. Volume 1: Executive summary

Development of a forecast of the total domestic telecommunications demand, identification of that portion of the telecommunications demand suitable for transmission by satellite systems, identification of that portion of the satellite market addressable by CPS systems, identification of that portion of the satellite market addressable by Ka-band CPS system, and postulation of a Ka-band CPS network on a nationwide and local level were achieved. The approach employed included the use of a variety of forecasting models, a parametric cost model, a market distribution model and a network optimization model. Forecasts were developed for: 1980, 1990, 2000; voice, data and video services; terrestrial and satellite delivery modes; and C, Ku and Ka-bands.

Kratochvil, D.↗

Hydrological Data at the NASA GES DISC: Current Capabilities and New Opportunities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth science data centers that document, process, archive and distribute data from Earth observation missions and projects. GES DISC maintains an archive of several hydrology datasets, including the Land Data Assimilation Systems (LDAS) and the Gravity Recovery and Climate Experiment (GRACE) Data Assimilation for Drought Monitoring (GRACE-DA-DM) data products. These datasets include model output of heat fluxes, rain, snow, soil temperature, soil moisture, and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at GES DISC ranges from hourly to monthly, and spatial resolutions range from 0.1° to 1.0°. The GES DISC provides services which enable users to aggregate, temporally and spatially subset, regrid, and visualize archived data including the GES DISC Subsetter, Hydrology Data Rods, and the Geospatial Interactive Online Visualization and Analysis Infrastructure (GIOVANNI). The Hydrology Data Rods service optimally reorganizes large hydrological data sets as extended time series, providing more efficient access for the hydrological community. The time series data (aka “data rods”) were integrated into hydrology community tools, such as the Data Rods Explorer on HydroShare. Furthermore, the GES DISC is in the process of migrating its data and services to the cloud. Hydrological data available at the GES DISC are now available in the Amazon Web Services (AWS) cloud (us-west-2 region) providing users Direct S3 data access and the capability for cloud computing operations. In this presentation, the hydrology data products and services currently available at the GES DISC will be summarized. Also discussed are the migration to the cloud, user support through this transition, and the status of migrating the data rods service to the cloud.

Ashley Heath↗

Applied Meteorology Unit (AMU)

This Quarter's Highlights incllude: completion of the second phase of verifying the performance of the MesoNAM weather model at Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS). (1) The data was delivered to the 45th Weather Squadron (45 WS) and the final report was distributed (2) Staff completed modifying and updating lightning c1imatologies for KSC/CCAFS and other airfields around central Florida. We delivered the tool to the National Weather Service in Melbourne and 45 WS and distributed the final report (3) Staff completed modifying the AMU peak wind tool by analyzing wind tower data to determine peak wind behavior during times of onshore and offshore flow. This was delivered the to the 45 WS and distributed the final report.

Bauman, William↗

Reusing Information Management Services for Recommended Decadal Study Missions to Facilitate Aerosol and Cloud Studies

NASA Earth Sciences Division (ESD) has made great investments in the development and maintenance of data management systems and information technologies, to maximize the use of NASA generated Earth science data. With information management system infrastructure in place, mature and operational, very small delta costs are required to fully support data archival, processing, and data support services required by the recommended Decadal Study missions. This presentation describes the services and capabilities of the Goddard Space Flight Center (GSFC) Earth Sciences Data and Information Services Center (GES DISC) and the reusability for these future missions. The GES DISC has developed a series of modular, reusable data management components currently in use. They include data archive and distribution (Simple, Scalable, Script-based, Science [S4] Product Archive aka S4PA), data processing (S4 Processor for Measurements aka S4PM), data search (Mirador), data browse, visualization, and analysis (Giovanni), and data mining services. Information management system components are based on atmospheric scientist inputs. Large development and maintenance cost savings can be realized through their reuse in future missions.

Kempler, Steve↗

A Contrast in Use of Metrics in Earth Science Data Systems

In recent years there has been a surge in the number of systems for processing, archiving and distributing remotely sensed data. Such systems, working independently as well as in collaboration, have been contributing greatly to the advances in the scientific understanding of the Earth system, as well as utilization of the data for nationally and internationally important applications. Among such systems, we consider those that are developed by or under the sponsorship of NASA to fulfill one of its strategic objectives: "Study Earth from space to advance scientific understanding and meet societal needs." NASA's Earth science data systems are of varying size and complexity depending on the requirements they are intended to meet. Some data systems are regarded as NASA's "Core Capabilities" that provide the basic infrastructure for processing, archiving and distributing a set of data products to a large and diverse user community in a robust and reliable manner. Other data systems constitute "Community Capabilities". These provide specialized and innovative services to data users and/or research products offering new scientific insight. Such data systems are generally supported by NASA through peer reviewed competition. Examples of Core Capabilities are 1. Earth Observing Data and Information System (EOSDIS) with its Distributed Active Archive Centers (DAACs), Science Investigator-led Processing Systems (SIPSs), and the EOS Clearing House (ECHO); 2. Tropical Rainfall Measurement Mission (TRMM) Science Data and Information System (TSDIS); 3. Ocean Data Processing System (ODPS); and 4. CloudSat Data Processing Center. Examples of Community Capabilities are projects under the Research, Education and Applications Solutions Network (REASON), and Advancing Collaborative Connections for Earth System Science (ACCESS) Programs. In managing these data system capabilities, it is necessary to have well-established goals and to measure progress relative to them. Progress is measured through "metrics", which can be a combination of quantitative as well as qualitative assessments. The specific metrics of interest depend on the user of the metrics as well as the type of data system. The users of metrics can be data system managers, program managers, funding agency or the public. Data system managers need metrics for assessing and improving the performance of the system and for future planning. Program managers need metrics to assess progress and the value of the data systems sponsored by them. Also, there is a difference in the metrics needed for core capabilities that tend to be more complex, larger and longer-term compared to community capabilities and the community capabilities that tend to be simpler, smaller and shorter-term. Even among community capabilities there are differences; hence the same set of metrics does not apply to all. Some provide data products to users, some provide services that enable better utilization of data or interoperability among other systems, and some are a part of a larger project where provision of data or services is only a minor activity. There is also a contrast between metrics used for internal and external purposes. Examples of internal purposes are: ensuring that the system meets its requirements, and planning for evolution and growth. Examples of external purposes are: providing to sponsors indicators of success of the systems, demonstrating the contributions of the system to overall program success, etc. This paper will consider EOSDIS, REASON and ACCESS programs to show the various types of metrics needed and how they need to be tailored to the types of data systems while maintaining the overall management goals of measuring progress and contributions made by the data systems.

Ramapriyan, Hampapuram↗

Proposed Application for an Entity Component System in an Energy Services Interface

An Entity Component System is a data-oriented architecture originally developed to streamline video game performance. Despite being quite new, Entity Component Systems are relatively well established within the video game industry due to the cutting edge nature of research into performance, especially around graphics. However, Entity Component Systems have not been widely examined or adopted outside of that industry. We propose adopting an Entity Component Systems framework to serve the needs of an Energy Service Interfaces. We examine the needs of an Energy Service Interface, give an overview of open-source Entity Component Systems (ECSs) libraries, examine some preliminary performance results for ECSs, and explore the traditional approach to fulfilling the needs of an Energy Service Interface (ESI) with database architectures.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The Earth Observing System - A multidisciplinary system for the long-term acquisition of earth science data from space

The development of the Earth Observing System (EOS) requirements is examined. An envisioned, EOS is to be NASA's multidisciplinary approach to acquisition of earth science data in the 1990s. The rationale and assumptions for the EOS are presented, together with a discussion of anticipated operations and use environments. Planned instrumentation concepts are discussed, involving a broad array of multiuse sensors operating across the electromagnetic spectrum from the ultraviolet to the microwave and including both active and passive techniques. Concepts for the near-polar orbiting observatory system, and essentially permanent man-serviced and -tended facility, are presented, along with concepts for information capture, processing and distribution and for common data-base referencing of acquired data. Milestones are indicated that are to be passed before a commitment to program execution can be made later in this decade.

Broome, D. R., Jr.↗

Enabling NVM for Data-Intensive Scientific Services

Specialized, transient data services are playing an increasingly prominent role in data-intensive scientific computing. These services offer flexible, on-demand pairing of applications with storage hardware using semantics that are optimized for the problem domain. Concurrent with this trend, upcoming scientific computing and big data systems will be deployed with emerging NVM technology to achieve the highest possible price/productivity ratio. Clearly, therefore, we must develop techniques to facilitate the confluence of specialized data services and NVM technology. In this work we explore how to enable the composition of NVM resources within transient distributed services while still retaining their essential performance characteristics. Our approach involves eschewing the conventional distributed file system model and instead projecting NVM devices as remote microservices that leverage user-level threads, RPC services, RMA-enabled network transports, and persistent memory libraries in order to maximize performance. We describe a prototype system that incorporates these concepts, evaluate its performance for key workloads on an exemplar system, and discuss how the system can be leveraged as a component of future data-intensive architectures.

Carns, Philip↗

Production Operations Management System (POMS) for Fermilab Experiments

The Production Operations Management System (POMS) software allows production teams and analysis groups across multiple Fermilab experiments to launch, modify and monitor large scale campaigns of related Monte Carlo or data processing jobs, and currently manages the majority of production computing of experiments at Fermilab.POMS provides a web service interface that enables automated job submission on distributed resources according to customers’ requests, as well as subsequent monitoring of those submissions as well as recovery of failed submissions, debugging and record keeping.POMS interfaces with existing HEP data access, processing, movement and monitoring tools at Fermilab, including Jobsub (for job submission), SAM (for data management), and Landscape (for monitoring), and in combination with them handles the creation of intermediate datasets of files for multistage campaigns.POMS visualizes these campaigns with an interactive GUI interface. An important feature of POMS is a one-to-one connection between the GUI campaign visualizer and a text based representation of the complete campaign configuration. An extensible library of template campaign configurations is available. The templates are user modifiable and map cleanly to and from the GUI description of the campaign.This paper will discuss the current usage, history, and some technical details of POMS, as well as planned future extensions, particularly interfacing with Rucio as well as SAM for file movement.

Mengel, Marc↗

Blockchain for Fault-Tolerant Grid Operations

Distribution system fault/failure has a direct impact on customers. Distribution systems are not designed with the contingency of one or more elements failing as the transmission system, however, distribution systems are more likely to have faults in comparison to transmission and generation systems. Most of the distribution systems have a radial design with the protection system assumption of unidirectional power flow. The radial design results in the disconnection of customers for any given component failure. The rapid deployment of distributed energy resources conflicts with the assumption of unidirectional power flow. To maintain/improve the distribution system fault tolerance for a grid with high penetration of distributed energy resources requires improvement. Blockchain can add value to improve fault-tolerant grid operations. That can be achieved using blockchain’s core features of distributed consensus-based decision-making process and immutability. In the process of preparing for an event or system restoration, there is a need for reliable data and system situational awareness. The system situational awareness enables accurate planning for possible scenarios and system restoration after an event, an area where blockchain’s distributed architecture can help. The value of blockchain for a fault-tolerant grid operation is elaborated with three use cases: 1) fault location, isolation, and service restoration; 2) topology identification; and 3) data configuration. These use cases demonstrate how blockchain-based architecture can facilitate bridging the vulnerabilities of the existing implementation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

ldlms_plugins_llnl

ldlms_plugins_llnl contains LLNL-developed plugins for LDMS (Lightweight Distributed Metric Service), which is software for low overhead monitoring, transport and storage of cluster metric data. These plugins add to LDMS the capability to monitor Lustre file system clients, metadata servers, and object storage servers.

Morrone, ChristopherJ↗

Astronomical Software Directory Service

With the support of NASA's Astrophysics Data Program (NRA 92-OSSA-15), we have developed the Astronomical Software Directory Service (ASDS): a distributed, searchable, WWW-based database of software packages and their related documentation. ASDS provides integrated access to 56 astronomical software packages, with more than 16,000 URLs indexed for full-text searching. Users are performing about 400 searches per month. A new aspect of our service is the inclusion of telescope and instrumentation manuals, which prompted us to change the name to the Astronomical Software and Documentation Service. ASDS was originally conceived to serve two purposes: to provide a useful Internet service in an area of expertise of the investigators (astronomical software), and as a research project to investigate various architectures for searching through a set of documents distributed across the Internet. Two of the co-investigators were then installing and maintaining astronomical software as their primary job responsibility. We felt that a service which incorporated our experience in this area would be more useful than a straightforward listing of software packages. The original concept was for a service based on the client/server model, which would function as a directory/referral service rather than as an archive. For performing the searches, we began our investigation with a decision to evaluate the Isite software from the Center for Networked Information Discovery and Retrieval (CNIDR). This software was intended as a replacement for Wide-Area Information Service (WAIS), a client/server technology for performing full-text searches through a set of documents. Isite had some additional features that we considered attractive, and we enjoyed the cooperation of the Isite developers, who were happy to have ASDS as a demonstration project. We ended up staying with the software throughout the project, making modifications to take advantage of new features as they came along, as well as influencing the software development. The Web interface to the search engine is provided by a gateway program written in C++ by a consultant to the project (A. Warnock).

Hanisch, Robert J.↗

Distributed Operations Planning

Maestro software provides a secure and distributed mission planning system for long-term missions in general, and the Mars Exploration Rover Mission (MER) specifically. Maestro, the successor to the Science Activity Planner, has a heavy emphasis on portability and distributed operations, and requires no data replication or expensive hardware, instead relying on a set of services functioning on JPL institutional servers. Maestro works on most current computers with network connections, including laptops. When browsing down-link data from a spacecraft, Maestro functions similarly to being on a Web browser. After authenticating the user, it connects to a database server to query an index of data products. It then contacts a Web server to download and display the actual data products. The software also includes collaboration support based upon a highly reliable messaging system. Modifications made to targets in one instance are quickly and securely transmitted to other instances of Maestro. The back end that has been developed for Maestro could benefit many future missions by reducing the cost of centralized operations system architecture.

Fox, Jason↗

Applications of Satellite Data to Support Improvements in Irrigation and Groundwater Management in California

In agricultural regions around the world, threats to water supplies from drought and groundwater depletion are driving increased demand for tools to advance agricultural water use efficiency and support sustainable groundwater management. Satellite mapping of evapotranspiration (ET) from irrigated agricultural lands can provide agricultural producers and water resource managers with information that can be used to both optimize ag water use and improve estimates of groundwater withdrawals for irrigation. We describe the development of two remote sensing-based tools for ET mapping in California, including important lessons in terms of system design, partnership development, and transition to operations. For irrigation management, the integration of satellite data and surface sensor networks to provide timely delivery of information on crop water requirements can make irrigation scheduling more practical, convenient, and accurate.Developed through a partnership between NASA and the CA Department of Water Resources, the Satellite Irrigation Management Support (SIMS) framework integrates satellite data with information from agricultural weather networks to map crop canopy development and crop water requirements at the scale of individual fields. Information is distributed to agricultural producers and water managers via a web-based interface and web data services. SIMS also provides an API that facilitates integration with other irrigation decision support tools, such as CropManage and IrriQuest. Field trials using these integrated tools have shown that they can be used to sustain yields while improving water use efficiency and nutrient management. For sustainable groundwater management, the combination of satellite-derived estimates of ET and data on surface water deliveries for irrigation can increase the accuracy of estimates of groundwater pumping. We are developing an OpenET platform to facilitate access to ET data from multiple models and accelerate operational use of ET data in support of a range of water management applications, including implementation of the Sustainable Groundwater Management Act in CA. By providing a shared basis for decision making, we anticipate that the OpenET platform will accelerate implementation of solutions for sustainable groundwater management.

Satellite↗

NASA's Earthdata Forum: Bringing the Science Community Together

The Earthdata Forum is an actively maintained and operational user forum for all participating NASA EOSDIS Distributed Active Archive Centers (DAACs) and GCMD. The Forum serves as a cross-DAAC platform from which user communities can obtain authoritative information relating to NASA Earth Science. The Atmospheric Science Data Center (ASDC) DAAC takes on the role of both Project Manager and Hosting Organization. As Project Manager, the ASDC leads and facilitates the project's efforts. The Forum launched February 2020 with five participating DAACs: Atmospheric Science Data Center (ASDC), Crustal Dynamics Data Information System (CDDIS), Goddard Earth Science Data and Information Services Center (GES DISC), Global Hydrology Resource Center (GHRC), and Land Processes DAAC (LP DAAC). Since February 2020, the of Level 1 and Atmosphere Archive and Distribution System DAAC (LAADS), the Ocean Biology DAAC (OBDAAC) and Global Change Master Directory (GCMD) have also joined the Forum. The Forum assumes both a proactive and reactive approach to NASA products scientific queries, allowing the scientific community to ask questions about NASA's products, tools, and services and receive announcements and news regarding products, new data releases, and such. In this presentation, we will describe the Earthdata Forum approach to support the scientific community in detail and discuss the successes and challenges of this approach.

Ingrid Garcia-Solera↗

"Sensor Web Evolution - Webs of Webs for NASA Science - Focus on small Uninhabited Aerial Systems (sUAS)"

This paper will describe the evolution of information collection, derivation and delivery mechanisms in webs of NASA sensor webs, with a focus on recent advancements in small Uninhabited Aerial Systems (sUAS). I will discuss the movement to "Fog Computing", also known as Edge Computing. Fog Computing facilitates the distribution of common operations and networking between edge devices and cloud computing facilities, optimizing the production of actionable intelligence. Initially, sUASs utilized onboard data collection as standard, with minimal data downloaded directly. Information products were derived in conventional computational environments, generally desk top computers, and information products made available to the Science Community in weeks or months. With the increased availability, and increasingly lower costs, of beyond line of sight (BLOS) satellite based communication, transmission rates and data volumes increased, and processing migrated to Cloud based services. Contemporary sUASs are moving some of that information product derivation to on vehicle services, and are creating a distributed Cloud/Fog environment. I will describe the technological advances that have made this possible, including low power multi-core Central Processing Units (CPU), and, more recently, the availability of high end Graphical Processing Units (GPU) that consume only a few watts. Intelligent system software, leveraging these hardware advances, finally allows for information product generation on-board, rather than simple data collection. Additionally, intelligent flight control systems now support mutual vehicle to vehicle collaboration, allowing sUASs to create ad-hoc sensor webs on demand, as required. Also discussed will be the lessons learned by the Authors' development of data systems for NASA's large High Altitude Long Endurance (HALE) UASs like Predator and Global Hawk, and how those lessons are being applied to sUAS development. This paper will focus on application, rather a deep dive into the technology, and will highlight improving data management through these new technologies.

Sensor Web↗

Information/user services, summary

Various aspects of the technology which are available to users now are described. Specific topics discussed include: data distribution: availability of and access to data; technology transfer: system use illustrations and availability of training; hardware systems descriptions: processing hardware constructed and available; data processing techniques: individual processing techniques; and future developments: a sampling of future technology.

Landgrebe, D. A.↗