Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2014

Federated Giovanni: A Distributed Web Service for Analysis and Visualization of Remote Sensing Data

The Geospatial Interactive Online Visualization and Analysis Interface (Giovanni) is a popular tool for users of the Goddard Earth Sciences Data and Information Services Center (GES DISC) and has been in use for over a decade. It provides a wide variety of algorithms and visualizations to explore large remote sensing datasets without having to download the data and without having to write readers and visualizers for it. Giovanni is now being extended to enable its capabilities at other data centers within the Earth Observing System Data and Information System (EOSDIS). This Federated Giovanni will allow four other data centers to add and maintain their data within Giovanni on behalf of their user community. Those data centers are the Physical Oceanography Distributed Active Archive Center (PO.DAAC), MODIS Adaptive Processing System (MODAPS), Ocean Biology Processing Group (OBPG), and Land Processes Distributed Active Archive Center (LP DAAC). Three tiers are supported: Tier 1 (GES DISC-hosted) gives the remote data center a data management interface to add and maintain data, which are provided through the Giovanni instance at the GES DISC. Tier 2 packages Giovanni up as a virtual machine for distribution to and deployment by the other data centers. Data variables are shared among data centers by sharing documents from the Solr database that underpins Giovanni's data management capabilities. However, each data center maintains their own instance of Giovanni, exposing the variables of most interest to their user community. Tier 3 is a Shared Source model, in which the data centers cooperate to extend the infrastructure by contributing source code.

Giovanni↗
NASA NTRS2018

Air Traffic Management-eXploration Testbed for Urban Air Mobility Research and Development

The presentation will describe the architecture, current capabilities and some future enhancements of the testbed that is being developed at the National Aeronautics and Space Administration (NASA) to enable benefit, impact, safety and cost assessments for accelerating the deployment of air traffic management concept and technologies in the national airspace system. The testbed will support analysis of operational feasibility of urban air mobility operations, a part of NASA's Air Traffic Management eXploration project, and provide the data needed by regulatory agencies charged with public safety. Introduction of concepts and technologies, especially new concepts and technologies, is difficult and often takes decades because of the inability to assess the operational impact of the interaction between the proposed concept and technology and operationally deployed systems in terms of system-wide safety, traffic flow efficiency, roles and workload of controllers and traffic managers, and impact on airlines and other operators. To overcome these limitations, the testbed is developing infrastructure to enable mathematical modeling, human-in-the-loop evaluations and testing with operational systems in a simulated environment. In addition to the difficulty of establishing communications between geographically distributed systems, downloading/installing software, and management of startup, error-handling and shutdown, a major impediment for conducting simulations and human-in-the-loop testing with operational systems is the tedious manual scenario generation process. Several of these difficulties have been addressed in the current state of the testbed. The testbed can be described in terms of the following elements (1) web-based frontend and backend, (2) Testbed Builder, (3) Data Distribution Service, (4) Component Library, (5) Simulation Management, and (6) Scenario Generation. The web-based frontend and backend enable the user to interact with the testbed for tasks such as composing a simulation, running a simulation and retrieving output data. The Testbed Builder application launched from the web frontend is a graphical user interface for the user to drag-and-drop and connect predefined blocks for composing a simulation/scenario generation task. The Builder writes a set of instructions for Simulation Management based on the links between the blocks and the block properties such as the component (executable) associated with a particular block. Management of the distributed simulation is accomplished by Execution and Component Managers. Execution Manager interprets the instructions provided by the Builder to instruct the Component Managers to download components from the Component Library to specified computers and to start them up. Once started, the components communicate with each other by publishing messages and subscribing to messages that are delivered by the Data Distribution Service. The Scenario Generation capability can be used for creating traffic scenarios for Multi-Aircraft Control System, which has been used extensively at NASA for human-in-the-loop-based concept evaluations. The presentation will provide a testbed enabled example scenario of Multi-Aircraft Control System based simulation in which the urban air mobility pilot using the conflict detection and resolution system would interact with the air traffic controllers for resolving conflicts with other aircraft during terminal area operations.

Testbed↗
NASA NTRS2018

Air Traffic Management-eXploration Testbed for Urban Air Mobility Research and Development

The presentation will describe the architecture, current capabilities and some future enhancements of the testbed that is being developed at the National Aeronautics and Space Administration (NASA) to enable benefit, impact, safety and cost assessments for accelerating the deployment of air traffic management concept and technologies in the national airspace system. The testbed will support analysis of operational feasibility of urban air mobility operations, a part of NASA's Air Traffic Management eXploration project, and provide the data needed by regulatory agencies charged with public safety. Introduction of concepts and technologies, especially new concepts and technologies, is difficult and often takes decades because of the inability to assess the operational impact of the interaction between the proposed concept and technology and operationally deployed systems in terms of system-wide safety, traffic flow efficiency, roles and workload of controllers and traffic managers, and impact on airlines and other operators. To overcome these limitations, the testbed is developing infrastructure to enable mathematical modeling, human-in-the-loop evaluations and testing with operational systems in a simulated environment. In addition to the difficulty of establishing communications between geographically distributed systems, downloading/installing software, and management of startup, error-handling and shutdown, a major impediment for conducting simulations and human-in-the-loop testing with operational systems is the tedious manual scenario generation process. Several of these difficulties have been addressed in the current state of the testbed. The testbed can be described in terms of the following elements- (1) web-based frontend and backend, (2) Testbed Builder, (3) Data Distribution Service, (4) Component Library, (5) Simulation Management, and (6) Scenario Generation. The web-based frontend and backend enable the user to interact with the testbed for tasks such as composing a simulation, running a simulation and retrieving output data. The Testbed Builder application launched from the web frontend is a graphical user interface for the user to drag-and-drop and connect predefined blocks for composing a simulation/scenario generation task. The Builder writes a set of instructions for Simulation Management based on the links between the blocks and the block properties such as the component (executable) associated with a particular block. Management of the distributed simulation is accomplished by Execution and Component Managers. Execution Manager interprets the instructions provided by the Builder to instruct the Component Managers to download components from the Component Library to specified computers and to start them up. Once started, the components communicate with each other by publishing messages and subscribing to messages that are delivered by the Data Distribution Service. The Scenario Generation capability can be used for creating traffic scenarios for Multi-Aircraft Control System, which has been used extensively at NASA for human-in-the-loop-based concept evaluations. The presentation will provide a testbed enabled example scenario of Multi-Aircraft Control System based simulation in which the urban air mobility pilot using the conflict detection and resolution system would interact with the air traffic controllers for resolving conflicts with other aircraft during terminal area operations.

Simulation↗
DOE OSTI2023

DGaaS: GPU as a Service on Distributed Computing System

In the rapidly evolving landscape of scientific computing, Graphics Processing Units (GPUs) have become indispensable for their unparalleled ability to handle parallel tasks in complex calculations, simulations, and data analysis. Their utility is further magnified in machine learning and AI applications, where they significantly accelerate model training and predictive analytics. Within this context, the Triton Inference Server emerges as a pivotal open-source tool, specializing in AI inferencing and optimizing GPU utilization across various platforms and frameworks. This paper presents an in-depth study on distributed High Throughput Computing (HTC), specifically focusing on the HTCondor framework and its resource provisioning tools, GlideinWMS and HEPCloud. These systems enable large-scale scientific experiments like CMS and DUNE to efficiently access and utilize vast computational resources. The paper explores the core architectural components of GlideinWMS, including jobs, user pools, and worker nodes, and discusses their integration with GPUs and the Triton server. The primary aim of this research is to develop a solution that optimizes GPU utilization by leveraging Glideins and containers. This approach allows computational jobs, particularly those involving AI models, to use GPUs only when essential, thereby facilitating efficient sharing of limited GPU resources. To validate this architecture, the study conducted three key tests involving custom scripts, container-based servers, and Triton server deployments. However, the study faces challenges, notably in locating the Triton server and ensuring secure remote access. To address these issues, future work will focus on developing a proxy mechanism and enhancing security protocols. In conclusion, this study offers a comprehensive roadmap for effective and efficient GPU utilization in distributed High Throughput Computing. It aims to contribute significantly to the scientific community by solving pressing problems and implementing robust solutions in collaboration with the GlideinWMS and HEPCloud teams. The research sets the stage for a more efficient, scalable, and cost-effective paradigm in scientific computing.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1990

Contents of the NASA ocean data system archive, version 11-90

The National Aeronautics and Space Administration (NASA) Ocean Data System (NODS) archive at the Jet Propulsion Laboratory (JPL) includes satellite data sets for the ocean sciences and global-change research to facilitate multidisciplinary use of satellite ocean data. Parameters include sea-surface height, surface-wind vector, sea-surface temperature, atmospheric liquid water, and surface pigment concentration. NODS will become the Data Archive and Distribution Service of the JPL Distributed Active Archive Center for the Earth Observing System Data and Information System (EOSDIS) and will be the United States distribution site for Ocean Topography Experiment (TOPEX)/POSEIDON data and metadata.

Smith, Elizabeth A.↗
DOE OSTI2021

Enhancing Active Distribution Systems Resilience by Fully Distributed Self-Healing Strategy

Distributed restoration can exploit smart grid technologies to enhance the resilience of active distribution networks toward a self-healing smart grid. However, the large number of decision variables, especially the binary ones for reconfiguration, bring challenges to developing scalable distributed distribution service restoration (DDSR) strategies. This paper proposes a fully distributed solution procedure based on the alternating direction method of multipliers (ADMM) for mixed-integer programming problems and applies to develop the DDSR framework. The method consists of relax-drive-polish phases, 1) relaxing binary variables, and applying the convex ADMM as a warm start; 2) driving the solutions toward Boolean values through a proximal operator; 3) fixing the obtained binding binary variables and solving the rest of the problem to polish results and achieve a high-quality suboptimal solution. Then, an autonomous clustering strategy and consensus ADMM are integrated with the proposed method to realize the fully distributed cluster-based framework of DDSR. This framework can first determine DER scheduling and switch status for reconfiguration to energize the out-of-service areas from local faults, and then provide the load restoration solution in a distributed manner for total blackouts in large-scale distribution networks. Furthermore, the effectiveness and scalability of the proposed DDSR framework are demonstrated through testing on the IEEE 123-node, IEEE 8500-node, and synthetic 100k-node test feeders.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

SYMBIOSYS: A Methodology for Performance Analysis of Composable HPC Data Services

Microservices are a powerful new way of building, customizing, and deploying distributed services owing to their flexibility and maintainability. Several large-scale distributed platforms have emerged to serve the growing needs of data-centric workloads and services in commercial computing. Concurrently, high-performance computing (HPC) systems and software are rapidly evolving to meet the demands of diversified applications and heterogeneity. The interplay of hardware factors, software configuration parameters, and the flexibility offered with a microservice architecture makes it nontrivial to estimate the optimal service instantiation for a given application workload. Further, this problem is exacerbated when considering that these services operate in a dynamic and heterogeneous HPC environment. An optimally integrated service can be vastly more performant than a haphazardly integrated one. Existing performance tools for HPC either fail to understand the request-response model of communication inherent to microservices or they operate within a narrow scope, limiting the insight that can be gleaned from employing them in isolation. We propose a methodology for integrated performance analysis of HPC microservices frameworks and applications called SYMBIOSYS. We describe its design and implementation within the context of the Mochi framework. This integration is achieved by combining distributed callpath profiling and tracing with a performance data exchange strategy that collects fine-grained, low-level metrics from the RPC communication library and network layers. The result is a portable, low-overhead performance analysis setup that provides a holistic profile of the dependencies among microservices and how they interact with the Mochi RPC software stack. Using HEPnOS, a production-quality Mochi data service, we demonstrate the low-overhead operation of SYMBIOSYS at scale and use it to identify the root causes of poorly performing service configurations.

microservices↗
NASA NTRS2019

Planetary Boundary Layer Height from AIRS, MERRA-2, and GPS Radio Occultation Data Products at NASA GES DISC, and Insights from Their Profiles Intercomparison

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) provides archive and distribution services for several data products in the Planetary Boundary Layer (PBL) category. As a new variable added to the Atmospheric Infrared Sounder (AIRS) Version 6 support product, the PBL height from AIRS is derived based on the gradients of the retrieved relative humidity profile, and provides the atmospheric pressure at the top of the PBL over the ocean. The GES DISC also hosts the Modern-Era Retrospective analysis for Research and Applications-2 (MERRA-2) product generated by the Goddard Earth Observing System Model Version 5 (GEOS-5) data assimilation system. The PBL height from MERRA-2 is based on the total eddy diffusion coefficient of heat. The monthly PBL height has been made available in the Giovanni system (Giovanni is a Web-based application developed by the GES DISC providing a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science remote sensing data). Recently, the GES DISC began serving the global PBL height climatology product derived from the COSMIC/FORMOSAT-3 and TerraSAR-X Global Positioning System (GPS) radio occultation (RO) measurements from June 2006 to December 2015. In a previous study, we presented the monthly PBL height data from AIRS and MERRA-2 and demonstrated the GES DISC services which support data intercomparison, such as access, plotting, sub-setting, regridding, and generation of a multi-year monthly mean. We also provided intercomparison results, and found that different PBL height definitions contributed to significant differences of PBL height values between AIRS and MERRA-2. In this work, we present the 10-year seasonal climatologies from the AIRS, MERRA-2 and GPS-RO. We also used the cross section and vertical profile services in Giovanni to display and analyze the vertical atmosphere structure over regions where the PBL height derived from the AIRS and MERRA-2 are quite different. The examination of the AIRS and MERRA-2 three-dimensional data found that the relative humidity profiles had larger differences than the temperature profiles. The MERRA-2 gives more details than the AIRS for the vertical distribution of the humidity.

Ding, Feng↗
DOE OSTI2020

Data-Centric Communication Framework for Multicast IEC 61850 Routable GOOSE Messages over the WAN in Modern Power Systems

In this paper, a data-centric communication framework is proposed for multicast routable generic object-oriented substation event (GOOSE) messages (MRGM) over the wide area network (WAN) for effective substation-to-substation (SS2SS) and substation to control center (SS2CC) communications. In this structure, the IEC 61850 GOOSE message is transmitted over the WAN using the data distribution service (DDS) as a fast, reliable, and secure data-centric communication middleware. The main feature of this framework is its multicast capability, where several authorized subscribers can receive a published message simultaneously. This can significantly improve the system monitoring and control of the protection systems in modern smart grids, where intelligent schemes can be applied. The effectiveness of the proposed platform, in terms of total end-to-end delay between participants, is evaluated through experimental results obtained from the actual hardware-based test setup developed at the Florida International University (FIU) smart grid testbed. The results demonstrate that the latency between sending and receiving a GOOSE message among participants is within its maximum time span defined by the IEC 61850-90-5 working group for communications over the WAN.

42 ENGINEERING↗
NASA NTRS2021

PBL Height from AIRS, GPS RO, and MERRA-2 Products in NASA GES DISC and Their 10 Year Seasonal Mean Intercomparison

Within the planetary boundary layer (PBL), surface forcing response, drag, turbulence, and vertical mixing are important processes and play a more critical role here than in the overlying “free atmosphere”. The PBL Height (PBLH) is an important parameter in climate models, weather forecasts, and air quality prediction. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) provides data processing, archiving, and distribution services for numerous Earth science products. PBLH is a parameter in three products served by GES DISC, which are from the Atmospheric Infrared Sounder (AIRS), the Global Positioning System (GPS) radio occultation (RO) experiment, and the NASA reanalysis product Modern-Era Retrospective analysis for Research and Applications – 2 (MERRA-2). These products have different spatial and temporal resolutions and coverages, and their PBLH definitions are also different. To better serve the PBL research community, we have summarized the specifications of these products. A ten-year seasonal mean intercomparison is also conducted to provide further guidance to users. The intercomparison results show that MERRA-2 has a much shallower PBL than AIRS and GPS RO. An experimental study indicates the different PBLH definition in MERRA-2 caused smaller values of PBLH. The improvement of the water vapor retrieval in AIRS version 7 over version 6 results in the version 7 PBLH agreeing better with GPS RO and MERRA-2 than version 6, especially near the equator and low latitudes.

Feng Ding↗
DOE OSTI2022

FY 2021 Isolated Grids and Grid-Connected Turbine Reference Systems; Microgrids, Infrastructure Resilience, and Advanced Controls Launchpad (MIRACL)

For individuals, businesses, and communities focused on building resilient electrical grid infrastructure, wind energy can provide an affordable, accessible, and compatible distributed energy resource option that also enhances the capabilities of local grid operations. The Microgrids, Infrastructure Resilience, and Advanced Controls Launchpad (MIRACL) project is a multi-year distributed wind research effort, driven through a partnership between four Department of Energy National Laboratories and industry to develop and improve the planning, design, and operation of wind-centered microgrids to complement solar, energy storage, and other distributed energy resources for grid-tied and isolated operation (U.S. Department of Energy, 2021). This report documents the application of methods developed through the initial three years of the MIRACL project to two real-world distributed wind reference systems. Specifically, the methods demonstrated in this report include 1) a market valuation framework to comprehensively value the services distributed wind can provide and 2) a resilience framework that enables stakeholders to characterize distribution system resilience and compare grid investment decisions from a resilience perspective. Additional methods mentioned in this report include distributed hybrid system design methods for grid resilience, advanced control co-simulation platforms, and power hardware-in-the-loop (PHIL) models. Preliminary results from these additional methods are presented in this report and will be demonstrated and/or applied to the reference systems in the coming year. The purpose of applying these methods to reference systems is to drive technology transfer of the theories, methodologies, and technologies developed under the MIRACL project and increase the number of referenceable case studies available to stakeholders interested in additional value-added capabilities of wind systems beyond bulk energy supply (i.e. kilowatt-hours).

17 WIND ENERGY↗
NASA NTRS2014

Temperature and Humidity Profiles in the TqJoint Data Group of AIRS Version 6 Product for the Climate Model Evaluation

The Atmospheric Infrared Sounder (AIRS) mission is entering its 13th year of global observations of the atmospheric state, including temperature and humidity profiles, outgoing long-wave radiation, cloud properties, and trace gases. Thus AIRS data have been widely used, among other things, for short-term climate research and observational component for model evaluation. One instance is the fifth phase of the Coupled Model Intercomparison Project (CMIP5) which uses AIRS version 5 data in the climate model evaluation. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the home of processing, archiving, and distribution services for data from the AIRS mission. The GES DISC, in collaboration with the AIRS Project, released data from the version 6 algorithm in early 2013. The new algorithm represents a significant improvement over previous versions in terms of greater stability, yield, and quality of products. The ongoing Earth System Grid for next generation climate model research project, a collaborative effort of GES DISC and NASA JPL, will bring temperature and humidity profiles from AIRS version 6. The AIRS version 6 product adds a new "TqJoint" data group, which contains data for a common set of observations across water vapor and temperature at all atmospheric levels and is suitable for climate process studies. How different may the monthly temperature and humidity profiles in "TqJoint" group be from the "Standard" group where temperature and water vapor are not always valid at the same time? This study aims to answer the question by comprehensively comparing the temperature and humidity profiles from the "TqJoint" group and the "Standard" group. The comparison includes mean differences at different levels globally and over land and ocean. We are also working on examining the sampling differences between the "TqJoint" and "Standard" group using MERRA data.

climate model evaluation↗
NASA NTRS2016

Constructing an AIRS Climatology for Data Visualization and Analysis to Serve the Climate Science and Application Communities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the home of processing, archiving, and distribution services for NASA sounders: the present Aqua AIRS mission and the succeeding SNPP CrIS mission. The AIRS mission is entering its 15th year of global observations of the atmospheric state, including temperature and humidity profiles, outgoing longwave radiation, cloud properties, and trace gases. The GES DISC, in collaboration with the AIRS Project, released product from the version 6 algorithm in early 2013. Giovanni, a Web-based application developed by the GES DISC, provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science remote sensing data without having to download the data. Most important variables from version 6 AIRS product are available in Giovanni. We are developing a climatology product using 14-year AIRS retrievals. The study can be a good start for the long term climatology from NASA sounders: the AIRS and the succeeding CrIS. This presentation will show the impacts to the climatology product from different aggregation methods. The climatology can serve climate science and application communities in data visualization and analysis, which will be demonstrated using a variety of functions in version 4 Giovanni. The highlights of these functions include user-defined monthly and seasonal climatology, inter annual seasonal time series, anomaly analysis.

AIRS↗