Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

The microeconomics of residential photovoltaics: Tariffs, network operation and maintenance, and ancillary services in distribution-level electricity markets

Here, we develop a microeconomic model of a distribution-level electricity market that takes explicit account of residential photovoltaics (PV) adoption. The model allows us to study the consequences of most tariffs on PV adoption and the consequences of increased residential PV adoption under the assumption of economic sustainability for electric utilities. We also validated the model using U.S. data and extend it to consider different pricing schemes for operation and maintenance costs of the distribution network and for ancillary services. Results show that net metering promotes more environmental benefits and social welfare than other tariffs. But, if costs to operate the distribution network increase, net metering will amplify the unequal distribution of surplus among households. In conclusion, maintaining the economic sustainability of electric utilities under net metering may become extremely difficult unless the uneven distribution of surplus is legitimated by environmental benefits.

14 SOLAR ENERGY↗

Harmonizing Multi-Disciplinary Data for Applied Research: A Comprehensive Analysis of GES DISC Datasets through NLP and TF-IDF Techniques

Data centers distribute data encompassing multiple disciplines, making it necessary to evaluate dataset applicability to applied research. Typically, these datasets are generated by specialized science teams and consist of single-discipline data, such as atmospheric temperature, pressure, and precipitation, leading to unique dataset formats and access services. However, in applied research, the utilization of datasets from multiple disciplines is commonly necessary. The increasing availability of research literature citing Earth Science datasets presents an opportunity to analyze the usage of datasets in multi-disciplinary research. This study proposes a novel approach wherein research publications citing datasets archived at the GES DISC (Goddard Earth Sciences Data and Information Services Center) are collected, and each publication is associated with specific research topics through the application of Natural Language Processing (NLP), using the Term Frequency Inverse Document Frequency (TF-IDF) technique on the publication titles and abstracts. Through this analysis, we gain insights into the distribution of dataset disciplines as they are being used in various applied research areas. This knowledge is essential for the development of dataset tools and services tailored to effectively support applied research studies, as it enhances a data center's comprehension of how datasets from multiple disciplines are integrated into research endeavors.

Infometrics↗

Global Change Data Center: Mission, Organization, Major Activities, and 2001 Highlights

Rapid efficient access to Earth sciences data is fundamental to the Nation's efforts to understand the effects of global environmental changes and their implications for public policy. It becomes a bigger challenge in the future when data volumes increase further and missions with constellations of satellites start to appear. Demands on data storage, data access, network throughput, processing power, and database and information management are increased by orders of magnitude, while budgets remain constant and even shrink. The Global Change Data Center's (GCDC) mission is to provide systems, data products, and information management services to maximize the availability and utility of NASA's Earth science data. The specific objectives are (1) support Earth science missions be developing and operating systems to generate, archive, and distribute data products and information; (2) develop innovative information systems for processing, archiving, accessing, visualizing, and communicating Earth science data; and (3) develop value-added products and services to promote broader utilization of NASA Earth Sciences Enterprise (ESE) data and information. The ultimate product of GCDC activities is access to data and information to support research, education, and public policy.

Wharton, Stephen W.↗

Integration of end-user Cloud storage for CMS analysis

End-user Cloud storage is increasing rapidly in popularity in research communities thanks to the collaboration capabilities it offers, namely synchronisation and sharing. CERN IT has implemented a model of such storage named, CERNBox, integrated with the CERN AuthN and AuthZ services. To exploit the use of the end-user Cloud storage for the distributed data analysis activity, the CMS experiment has started the integration of CERNBox as a Grid resource. This will allow CMS users to make use of their own storage in the Cloud for their analysis activities as well as to benefit from synchronisation and sharing capabilities to achieve results faster and more effectively. It will provide an integration model of Cloud storages in the Grid, which is implemented and commissioned over the world’s largest computing Grid infrastructure, Worldwide LHC Computing Grid (WLCG). In this paper, we present the integration strategy and infrastructure changes needed in order to transparently integrate end-user Cloud storage with the CMS distributed computing model. We describe the new challenges faced in data management between Grid and Cloud and how they were addressed, along with details of the support for Cloud storage recently introduced into the WLCG data movement middleware, FTS3. Finally, the commissioning experience of CERNBox for the distributed data analysis activity is also presented.

97 MATHEMATICS AND COMPUTING↗

Towards Cross-Facility Workflows Orchestration through Distributed Automation

Modern science relies on end-to-end workflows that incorporate experimental instruments and utilize edge, cloud, or high-performance computing and storage resources. These components are geographically dispersed across various user facilities and interconnected through high-speed networks. In this paper, we present Zambeze, an automated distributed framework designed to facilitate this new class of cross-facility workflows. Utilizing swarm intelligence principles, Zambeze orchestrates science campaigns by managing distributed autonomous agents. These agents can offer a suite of services, including computing, storage, and data management. We demonstrate the feasibility of Zambeze through a real-world application involving electron microscopy, enhanced with Artificial Intelligence capabilities.

Skluzacek, Tyler↗

Nuclear Physics Network Requirements Review Report

The Energy Sciences Network (ESnet) is the Office of Science’s high-performance network user facility, delivering highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the U.S. Department of Energy (DOE) science mission by connecting each and every DOE lab and its user facilities. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) Program and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet connects DOE national laboratories, user facilities, and major experiments so scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large data sets, and access distributed data repositories. While ESnet provides network connectivity, it cannot be characterized as an internet service provider as it is specifically built to provide a range of network services that are tailored to meet the unique requirements of DOE’s data-intensive science.

97 MATHEMATICS AND COMPUTING↗

Nuclear Physics Network Requirements Review Report

The Energy Sciences Network (ESnet) is the Office of Science’s high-performance network user facility, delivering highly reliable data transport capabilities optimized for the requirements of data-intensive science. In essence, ESnet is the circulatory system that enables the U.S. Department of Energy (DOE) science mission by connecting each and every DOE lab and its user facilities. ESnet is funded and stewarded by the Advanced Scientific Computing Research (ASCR) Program, and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory (LBNL). ESnet is widely regarded as a global leader in the research and education networking community. ESnet connects DOE national laboratories, user facilities, and major experiments so scientists can use remote instruments and computing resources as well as share data with collaborators, transfer large data sets, and access distributed data repositories. While ESnet provides network connectivity, it cannot be characterized as an internet service provider as it is specifically built to provide a range of network services that are tailored to meet the unique requirements of DOE’s data-intensive science. In short, ESnet’s mission is to enable and accelerate scientific discovery by delivering unparalleled network infrastructure, capabilities, and tools. ESnet’s vision is summarized by these three points. 1. Scientific progress will be completely unconstrained by the physical location of instruments, people, computational resources, or data. 2. Collaborations at every scale, in every domain, will have the information and tools they need to achieve maximum benefit from scientific facilities, global networks, and emerging network capabilities. 3. ESnet will foster the partnerships and pioneer the technologies necessary to ensure that these transformations occur. In May 2019, ESnet and the Office of Nuclear Physics (NP) of the DOE Office of Science organized an ESnet requirements review of NP-supported activities. Preparation for this event included identification of key stakeholders to the process: program and facility management, research groups, technology providers, and a number of external observers. These individuals were asked to prepare formal case study documents about their relationship to the NP program to build a complete understanding of the current, near-term, and long-term status, expectations, and processes that will support the science going forward. A series of pre-planning meetings better prepared case study authors for this task and facilitated a smooth transition to the in-person review.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

ESGF Future Architecture Report (V.1.1)

This document reports the findings of the Earth System Grid Federation Future Architecture Face-to-Face Meeting held in the UK 5-7 November 2019, attended by technical representatives from the ESGF partner organisations. ESGF is an International federated archive that provides access to Earth science data. It is fundamentally a distributed file management system with its main function being data dissemination through the provision of services for efficient replication and search. It has been in operation for over ten years and has undergone continuous improvements and enhancements over this time span. The Future Architecture is a major review of the software system including a fundamental reassessment of its requirements with a view to develop a new overall system architecture. This has been initiated in the light of challenges around the maintenance of the existing software legacy and a changing technology landscape and practices in the communities it serves. The work is being carried out under the oversight of the ESGF Executive Committee1,2 and is coordinated activity between ESGF and the European IS-ENES3 project.

54 ENVIRONMENTAL SCIENCES↗

A cursory look at shuttle flight operations.

The purpose of the shuttle system is to permit us to do more useful and necessary things in space at minimum cost. The flight operation subsystem must be consistent with these goals. This review of the predicted payload and mission types reveals a ground facility functionally similar to the Skylab Control Center is needed to provide trajectory control and system management services for the spacecraft and payloads and to provide rapid, effective payload data acquisition and distribution to the user. An automated mission management system must be added to translate user requirements into specific missions, directives, schedules, and plans.

Tindall, H. W., Jr.↗

Payload carrier systems for conducting sortie mode science

The capabilities and characteristics of the payload carriers developed to provide structural and operational interfaces between the Space Shuttle and the various types of experiments designed to operate in the sortie mode are discussed. The Spacelab is a flexible laboratory system composed of interchangeable elements that can be put together in eight different combinations of pallets and pressurized modules, and provides considerable standard services to users in such areas as equipment installation, power distribution, thermal control, command and data management, software, pointing systems and crew participation. A modular three-axis pointing control system designated the Annular Suspension and Pointing System, is being developed to provide additional pointing capabilities to those payloads that require capabilities not provided by the Spacelab instrument pointing system. Two engineering models of the Spacelab pallet have been designated Orbital Flight Test Pallets which, together with a special experiment support structure, are intended for initial and operational payloads that do not constitute a complete Spacelab mission. The simplest and smallest payload carriers are the Getaway Special cans, intended for small, self-contained, self-sufficient payloads, and the orbiter middeck lockers. In this way, most of the user requirements for Shuttle sortie missions identified to date can be fulfilled.

Jean, O. C.↗

Results of the Collaborative Energy and Water Cycle Information Services (CEWIS) Workshop on Heterogeneous Dataset Analysis Preparation

In support of the NASA Energy and Water Cycle Study (NEWS), the Collaborative Energy and Water Cycle Information Services (CEWIS), sponsored by NEWS Program Manager Jared Entin, was initiated to develop an evolving set of community-based data and information services that would facilitate users to locate, access, and bring together multiple distributed heterogeneous energy and water cycle datasets. The CEWIS workshop, June 15-16, 2010, at NASA/GSFC, was the initial step of the process, starting with identifying and scoping the issues, as defined by the community.

Kempler, Steven↗

Cognitive Communications and Networking Technology Infusion Study Report

As the envisioned next-generation SCaN Network transitions into an end-to-end “system of systems” with new enabling capabilities, it is anticipated that the introduction of machine learning, artificial intelligence, and other cognitive strategies into the network infrastructure will result in increased mission science return, improved resource efficiencies, and increased autonomy and reliability. This enhanced set of cognitive capabilities will be implemented via a “space cloud” concept to achieve a service-oriented architecture with distributed cognition, de-centralized routing, and shared, on-orbit data processing. The enabling cognitive communications and networking capabilities that may facilitate the desired network enhancements are identified in this document, and the associated enablers of these capabilities, such as technologies and standards, are described in detail.

Knoblock, Eric J.↗

Development and Implementation of Dynamic Scripts to Execute Cycled GSI/WRF Forecasts

The Weather Research and Forecasting (WRF) numerical weather prediction (NWP) model and Gridpoint Statistical Interpolation (GSI) data assimilation (DA) are the operational systems that make up the North American Mesoscale (NAM) model and the NAM Data Assimilation System (NDAS) analysis used by National Weather Service forecasters. The Developmental Testbed Center (DTC) manages and distributes the code for the WRF and GSI, but it is up to individual researchers to link the systems together and write scripts to run the systems, which can take considerable time for those not familiar with the code. The objective of this project is to develop and disseminate a set of dynamic scripts that mimic the unique cycling configuration of the operational NAM to enable researchers to develop new modeling and data assimilation techniques that can be easily transferred to operations. The current version of the SPoRT GSI/WRF Scripts (v3.0.1) is compatible with WRF v3.3 and GSI v3.0.

Zavodsky, Bradley↗

Development and Implementation of Dynamic Scripts to Execute Cycled GSI/WRF Forecasts

The Weather Research and Forecasting (WRF) numerical weather prediction (NWP) model and Gridpoint Statistical Interpolation (GSI) data assimilation (DA) are the operational systems that make up the North American Mesoscale (NAM) model and the NAM Data Assimilation System (NDAS) analysis used by National Weather Service forecasters. The Developmental Testbed Center (DTC) manages and distributes the code for the WRF and GSI, but it is up to individual researchers to link the systems together and write scripts to run the systems, which can take considerable time for those not familiar with the code. The objective of this project is to develop and disseminate a set of dynamic scripts that mimic the unique cycling configuration of the operational NAM to enable researchers to develop new modeling and data assimilation techniques that can be easily transferred to operations. The current version of the SPoRT GSI/WRF Scripts (v3.0.1) is compatible with WRF v3.3 and GSI v3.0.

Zavodsky, Bradley↗

Fault-Tolerant and Elastic Streaming MapReduce with Decentralized Coordination

The MapReduce programming model, due to its simplicity and scalability, has become an essential tool for processing large data volumes in distributed environments. Recent Stream Processing Systems (SPS) extend this model to provide low-latency analysis of high-velocity continuous data streams. However, integrating MapReduce with streaming poses challenges: first, the runtime variations in data characteristics such as data-rates and key-distribution cause resource overload, that inturn leads to fluctuations in the Quality of the Service (QoS); and second, the stateful reducers, whose state depends on the complete tuple history, necessitates efficient fault-recovery mechanisms to maintain the desired QoS in the presence of resource failures. We propose an integrated streaming MapReduce architecture leveraging the concept of consistent hashing to support runtime elasticity along with locality-aware data and state replication to provide efficient load-balancing with low-overhead fault-tolerance and parallel fault-recovery from multiple simultaneous failures. Our evaluation on a private cloud shows up to 2:8 improvement in peak throughput compared to Apache Storm SPS, and a low recovery latency of 700 -1500 ms from multiple failures.

97 MATHEMATICS AND COMPUTING↗

Extending the Remote Control Capabilities in the CMS Detector Control System with Remote Procedure Call Services

The CMS Detector Control System (DCS) is implemented as a large distributed and redundant system, with applications interacting and sharing data in multiple ways. The CMS XML-RPC is a software toolkit implementing the standard Remote Procedure Call (RPC) protocol, using the Extensible Mark-up Language (XML) and a custom lightweight variant using the JavaScript Object Notation (JSON) to model, encode and expose resources through the Hypertext Transfer Protocol (HTTP). The CMS XML-RPC toolkit complies with the standard specification of the XML-RPC protocol that allows system developers to build collaborative software architectures with self-contained and reusable logic, and with encapsulation of well-defined processes. The implementation of this protocol introduces not only a powerful communication method to operate and exchange data with web-based applications, but also a new programming paradigm to design service-oriented software architectures within the CMS DCS domain. This paper presents details of the CMS XML-RPC implementation in WinCC Open Architecture (OA) Control Language using an object-oriented approach.

43 PARTICLE ACCELERATORS↗

Using Open and Interoperable Ways to Publish and Access LANCE AIRS Near-Real Time Data

The Atmospheric Infrared Sounder (AIRS) Near-Real Time (NRT) data from the Land Atmosphere Near real-time Capability for EOS (LANCE) element at the Goddard Earth Sciences Data and Information Services Center (GES DISC) provides information on the global and regional atmospheric state, with very low temporal latency, to support climate research and improve weather forecasting. An open and interoperable platform is useful to facilitate access to, and integration of, LANCE AIRS NRT data. As Web services technology has matured in recent years, a new scalable Service-Oriented Architecture (SOA) is emerging as the basic platform for distributed computing and large networks of interoperable applications. Following the provide-register-discover-consume SOA paradigm, this presentation discusses how to use open-source geospatial software components to build Web services for publishing and accessing AIRS NRT data, explore the metadata relevant to registering and discovering data and services in the catalogue systems, and implement a Web portal to facilitate users' consumption of the data and services.

Zhao, Peisheng↗

The D3 Middleware Architecture

DARWIN is a NASA developed, Internet-based system for enabling aerospace researchers to securely and remotely access and collaborate on the analysis of aerospace vehicle design data, primarily the results of wind-tunnel testing and numeric (e.g., computational fluid-dynamics) model executions. DARWIN captures, stores and indexes data; manages derived knowledge (such as visualizations across multiple datasets); and provides an environment for designers to collaborate in the analysis of test results. DARWIN is an interesting application because it supports high-volumes of data. integrates multiple modalities of data display (e.g., images and data visualizations), and provides non-trivial access control mechanisms. DARWIN enables collaboration by allowing not only sharing visualizations of data, but also commentary about and views of data. Here we provide an overview of the architecture of D3, the third generation of DARWIN. Earlier versions of DARWIN were characterized by browser-based interfaces and a hodge-podge of server technologies: CGI scripts, applets, PERL, and so forth. But browsers proved difficult to control, and a proliferation of computational mechanisms proved inefficient and difficult to maintain. D3 substitutes a pure-Java approach for that medley: A Java client communicates (though RMI over HTTPS) with a Java-based application server. Code on the server accesses information from JDBC databases, distributed LDAP security services, and a collaborative information system. D3 is a three tier-architecture, but unlike 'E-commerce' applications, the data usage pattern suggests different strategies than traditional Enterprise Java Beans - we need to move volumes of related data together, considerable processing happens on the client, and the 'business logic' on the server-side is primarily data integration and collaboration. With D3, we are extending DARWIN to handle other data domains and to be a distributed system, where a single login allows a user transparent access to test results from multiple servers and authority domains.

Walton, Joan↗