Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “cluster computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS2005

[Earth Science Technology Office's Computational Technologies Project]

This grant supported the effort to characterize the problem domain of the Earth Science Technology Office's Computational Technologies Project, to engage the Beowulf Cluster Computing Community as well as the High Performance Computing Research Community so that we can predict the applicability of said technologies to the scientific community represented by the CT project and formulate long term strategies to provide the computational resources necessary to attain the anticipated scientific objectives of the CT project. Specifically, the goal of the evaluation effort is to use the information gathered over the course of the Round-3 investigations to quantify the trends in scientific expectations, the algorithmic requirements and capabilities of high-performance computers to satisfy this anticipated need.

Fischer, James↗
NASA NTRS2015

Computational Approaches to Simulation and Optimization of Global Aircraft Trajectories

This study examines three possible approaches to improving the speed in generating wind-optimal routes for air traffic at the national or global level. They are: (a) using the resources of a supercomputer, (b) running the computations on multiple commercially available computers and (c) implementing those same algorithms into NASA’s Future ATM Concepts Evaluation Tool (FACET) and compares those to a standard implementation run on a single CPU. Wind-optimal aircraft trajectories are computed using global air traffic schedules. The run time and wait time on the supercomputer for trajectory optimization using various numbers of CPUs ranging from 80 to 10,240 units are compared with the total computational time for running the same computation on a single desktop computer and on multiple commercially available computers for potential computational enhancement through parallel processing on the computer clusters. This study also re-implements the trajectory optimization algorithm for further reduction of computational time through algorithm modifications and integrates that with FACET to facilitate the use of the new features which calculate time-optimal routes between worldwide airport pairs in a wind field for use with existing FACET applications. The implementations of trajectory optimization algorithms use MATLAB, Python, and Java programming languages. The performance evaluations are done by comparing their computational efficiencies and based on the potential application of optimized trajectories. The paper shows that in the absence of special privileges on a supercomputer, a cluster of commercially available computers provides a good option for computing wind-optimal trajectories for national and global air traffic system studies.

Ng, Hok K.↗
NASA NTRSDate not supplied

A Study of Parallel Scalability and Dynamic Workload Balancing in GlennICE

The Glenn Icing Computational Environment (GlennICE) is a computational tool designed to calculate ice growth on complex three-dimensional geometries. It utilizes user-supplied computational fluid dynamics solutions for the geometry of interest. Key developments include advancements in convergence of collection efficiency, trajectory optimization, and refinement methodology. These improvements have significantly enhanced GlennICE’s efficiency for practical engineering applications. A recent study focused on benchmarking GlennICE’s scalability in a parallel environment using static scheduling. Findings indicated a potential twofold increase in efficiency through workload balance enhancements. This paper presents an analysis of the solver’s new workload balancing improvements, incorporating shared memory and dynamic scheduling routines. Results demonstrate a highly efficient and consistent algorithm across high-performance computing clusters.

Computational Icing↗
NASA NTRS2002

Architecture-Adaptive Computing Environment: A Tool for Teaching Parallel Programming

Recently, networked and cluster computation have become very popular. This paper is an introduction to a new C based parallel language for architecture-adaptive programming, aCe C. The primary purpose of aCe (Architecture-adaptive Computing Environment) is to encourage programmers to implement applications on parallel architectures by providing them the assurance that future architectures will be able to run their applications with a minimum of modification. A secondary purpose is to encourage computer architects to develop new types of architectures by providing an easily implemented software development environment and a library of test applications. This new language should be an ideal tool to teach parallel programming. In this paper, we will focus on some fundamental features of aCe C.

Dorband, John E.↗
DOE OSTI2019

Hydraulic Study of the South Carolina DOT Catch Basin Type 25

Proper design of surface drainage of both rural and urban highways is essential to minimize flooding and provide traffic safety. Inlets collect the excess storm water from a roadway and discharge it to storm drains. Knowing the performance of inlets (or hydraulic efficiency), defined as the percentage of intercepted flow to the total street flow, is necessary in the design of inlet spacing. The location of inlets can be determined such that the system can transport all or a majority of the road surface flow during rain events. A new type of a catch basin with an inlet grate, Type 25 (CB25), to use as drainage on South Carolina’s freeways was designed by the South Carolina Department of Transportation engineers. Because it is a new design, a study was necessary to establish the hydraulic capacity of the grate. Computational fluid dynamics (CFD) modeling was chosen for this purpose, instead of more expensive and time consuming laboratory testing. CFD has been widely used in design of new transportation infrastructure, as well as performance analysis of the existing structures. Three dimensional computational fluid dynamic simulations were developed by scientists at Argonne’s Transportation Research and Analysis Computing Center with the use of high performance cluster computing. The work was performed under an inter-agency agreement between the U.S. Department of Transportation and the U.S Department of Energy and funded by the South Carolina Department of Transportation.

42 ENGINEERING↗
DOE OSTI2018

Cyber-Infrastructure for Data-Intensive Geospatial Computing

With the recent advent of heterogeneous High-performance Computing (HPC) to handle EO “Big Data” workloads, there is a need for a unified Cyber-infrastructure (CI) platform that can bridge the best of many HPC worlds. In this chapter, we discuss such a CI platform being developed at Geographic Information Science and Technology (GIST) group using novel and innovative techniques, and emerging technologies that are scalable to large-scale supercomputers. The CI platform utilizes a wide variety of computing such as GPGPU, distributed, real-time and cluster computing, which are being brought together architecturally to enable data-driven analysis, scientific understanding of earth system models, and research collaboration. This development addresses the need for close integration of EO and other geospatial information in the face of growing volumes of the data, and facilitates spatio-temporal analysis of disparate and dynamic data streams. Horizontal scalability and linear throughput are supported in the heart of the platform itself. It is being used to support very broad application areas, ranging from high-resolution settlement mapping, national bioenergy infrastructure to urban information and mobility systems. The platform provides spatio-temporal decision support capabilities in planning, policy and operational missions for US federal agencies. Also, the platform is designed to be functionally and technologically sustainable for continued support of the US energy and environment mission for the coming decades.

Karthik, Rajasekar↗
NASA NTRSDate not supplied

Reconstruction of Thermal Protection System Aeroheating using a Green’s Function Approach

Inverse heat transfer (IHT) techniques are often used to reconstruct the surface heating conditions on spacecraft thermal protection systems (TPS) during atmospheric entry. Current IHT techniques for entry spacecraft applications, however, demand substantial computational resources, and are impractical for analyses such as uncertainty quantification and real-time health monitoring. In this paper, a Green’s function sensor fusion approach is used to reconstruct the TPS surface aeroheating conditions on experimental spaceflight and ground test systems from collocated temperature and heat flux sensors embedded in the TPS. The algorithm leverages Green’s functions to model the heat conduction within the spacecraft TPS and stabilizes the recovery of the surface heating condition using the direct heat flux sensor measurement. The algorithm is validated using arc-jet ground test data and applied to the reconstruction of the Mars 2020 backshell heating during Martian atmospheric entry. The performance of the algorithm is benchmarked against a current state-of-the-art IHT framework, FIAT_Opt. The Green’s function-based reconstruction algorithm recovers the net hot-wall heat flux absorbed by the TPS and the incident heat flux from the atmospheric entry environment in close agreement with FIAT_Opt. Notably, computation of the surface heating condition is completed in three orders of magnitude less time with the Green’s function sensor fusion approach using a consumer-grade PC, versus with FIAT_Opt running on a high performance computer cluster. The efficiency of the algorithm is leveraged to compute the uncertainty contributions of input parameters to the total uncertainty in reconstructed Mars 2020 backshell heating for the full atmospheric entry heat pulse. The sensitivity analysis uncovers that, at different times throughout the entry heat pulse, uncertainties in the TPS specific heat, thermal conductivity, and emissivity are all dominant drivers of the reconstruction uncertainty. These results demonstrate Green’s functions and sensor-fusion techniques as promising IHT approaches to reconstruct atmospheric entry environments from TPS-embedded measurements, and highlight how these techniques may give access to post-flight analyses previously hindered by the prohibitive cost of current methods.

Kenneth McAfee↗
NASA NTRSDate not supplied

Reconstruction of Thermal Protection System Aeroheating using a Green’s Function Approach

Inverse heat transfer (IHT) techniques are often used to reconstruct the surface heating conditions on spacecraft thermal protection systems (TPS) during atmospheric entry. Current IHT techniques for entry spacecraft applications, however, demand substantial computational resources, and are impractical for analyses such as uncertainty quantification and real-time health monitoring. In this paper, a Green’s function sensor fusion approach is used to reconstruct the TPS surface aeroheating conditions on experimental spaceflight and ground test systems from collocated temperature and heat flux sensors embedded in the TPS. The algorithm leverages Green’s functions to model the heat conduction within the spacecraft TPS and stabilizes the recovery of the surface heating condition using the direct heat flux sensor measurement. The algorithm is validated using arc-jet ground test data and applied to the reconstruction of the Mars 2020 backshell heating during Martian atmospheric entry. The performance of the algorithm is benchmarked against a current state-of-the-art IHT framework, FIAT_Opt. The Green’s function-based reconstruction algorithm recovers the net hot-wall heat flux absorbed by the TPS and the incident heat flux from the atmospheric entry environment in close agreement with FIAT_Opt. Notably, computation of the surface heating condition is completed in three orders of magnitude less time with the Green’s function sensor fusion approach using a consumer-grade PC, versus with FIAT_Opt running on a high performance computer cluster. The efficiency of the algorithm is leveraged to compute the uncertainty contributions of input parameters to the total uncertainty in reconstructed Mars 2020 backshell heating for the full atmospheric entry heat pulse. The sensitivity analysis uncovers that, at different times throughout the entry heat pulse, uncertainties in the TPS specific heat, thermal conductivity, and emissivity are all dominant drivers of the reconstruction uncertainty. These results demonstrate Green’s functions and sensor-fusion techniques as promising IHT approaches to reconstruct atmospheric entry environments from TPS-embedded measurements, and highlight how these techniques may give access to post-flight analyses previously hindered by the prohibitive cost of current methods.

Kenneth McAfee↗
NASA NTRS2005

Autonomic Cluster Management System (ACMS): A Demonstration of Autonomic Principles at Work

Cluster computing, whereby a large number of simple processors or nodes are combined together to apparently function as a single powerful computer, has emerged as a research area in its own right. The approach offers a relatively inexpensive means of achieving significant computational capabilities for high-performance computing applications, while simultaneously affording the ability to. increase that capability simply by adding more (inexpensive) processors. However, the task of manually managing and con.guring a cluster quickly becomes impossible as the cluster grows in size. Autonomic computing is a relatively new approach to managing complex systems that can potentially solve many of the problems inherent in cluster management. We describe the development of a prototype Automatic Cluster Management System (ACMS) that exploits autonomic properties in automating cluster management.

Baldassari, James D.↗
DOE OSTI2024

Containers on Switches: A Cluster School Experience

Network switches, such as those from Arista and Mellanox, often have underutilized computational resources in the form of built-in processors and memory. By leveraging these untapped resources, we can optimize functionality and efficiency of computational cluster networks. Our research focuses on deploying containers directly onto these switches to execute various auxiliary tasks ranging from metric logging to system-wide management via post-boot configuration. By doing so, we can significantly enchance the capabilities of the cluster without the need for additional dedicated hardware. Our research involved five distinct scenarios where switch utilization could have a profound impact on HPC Clusters: run cloud-init services via link-local connection; configuring a Telegraf container to export metrics; deploying a caching proxy; creating a reconfigurable IPv6 DHCP/DNS provider for VLAN; and implementing a client detection with Magellan discovery. These scenarios were containerized with podman and docker, and tested both physically on the switch virtually on a QEMU VM both running SONiC OS. Testing and findings indicate that network switches can indeed be used for these scenarios. They offer a wide range of possibilities beyond these applications. They run as expected as containers on the switches, and although there were some minor issues, work-arounds were implemented. Overall, this is a positive result that can be further explored with more scenarios.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

A Study of Parallel Scalability and Dynamic Workload Balancing in GlennICE

The Glenn Icing Computational Environment (GlennICE) is a computational tool designed to calculate ice growth on complex three-dimensional geometries. It utilizes user-supplied computational fluid dynamics solutions for the geometry of interest. Key developments include advancements in convergence of collection efficiency, trajectory optimization, and refinement methodology. These improvements have significantly enhanced GlennICE’s efficiency for practical engineering applications. A recent study focused on benchmarking GlennICE’s scalability in a parallel environment using static scheduling. Findings indicated a potential twofold increase in efficiency through workload balance enhancements. This paper presents an analysis of the solver’s new workload balancing improvements, incorporating shared memory and dynamic scheduling routines. Results demonstrate a highly efficient and consistent algorithm across high-performance computing clusters.

Computational Icing↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2019

A cloud computing approach to massively-parallel distributed stochastic optimization with application to geomechanical reservoir characterization

Many search strategies used to optimize parameters during hydrologic model calibration are inherently limited in that simulations must be performed sequentially in order for the algorithm to converge. As a result, the overall time required to reach a solution increases linearly with the number of steps necessary. In contrast, sampling strategies that attempt to approximate the posterior distribution of the parameters do not require the model simulations to be run in sequence and therefore invite the use of a massively parallel approach across heterogeneous computing environments. We apply a cloud computing methodology to this problem where a head node running on Amazon Web Services continuously produces candidate models that are distributed to arbitrary compute environments to run the simulation, in our case Clemson’s Palmetto cluster computer. Thus the time required to obtain a sufficient number of samples to produce an adequate approximation of the posterior distribution of the parameters is limited by the number of available compute sites. Monte Carlo sampling of the posterior space would be duplicative and inefficient, thus we combine Latin hypercube methods with genetic algorithms to ensure that no large regions of the parameter space are left unexplored, and that high probability regions are investigated thoroughly. We independently assess the fit to different types of data sets without the need to construct a single objective function. Sets of Pareto optimal models are particularly valuable when the quality and information content of data is poorly known or conceptual model errors exist in our simulation. We illustrate our approach for a problem where in situ strain data are used to calibrate the parameters of an oil reservoir in the Avant oil field in Oklahoma. We use well injection and shut-in to stress a deep (500m) confined formation, and observe the geomechanical response using borehole strainmeters installed in the shallow subsurface (30m). These data were then used to calibrate the poroelastic properties of the reservoir and caprock along with the geometry of an elliptic channel associated with the formation. Using our approach we are able to identify issues with model error as well as substantial tradeoffs that occur within the parameters.

inversion, stochastic, geomechanics, poroelastic, ↗
DOE OSTI2022

Elastic Resource Management for Deep Learning Applications in a Container Cluster

The increasing demand for learning from massive datasets is restructuring our economy. Effective learning, however, involves nontrivial computing resources. Most businesses utilize commercial infrastructure providers (e.g., AWS) to host their computing clusters in the cloud, where various jobs compete for available resources. While cloud resource management is a fruitful research field that has made many advances in production, such as Kubernetes and YARN, few efforts have been invested to further optimize the system performance, especially for deep learning (DL) training jobs in a container cluster. This work introduces FlowCon, a system that is able to monitor the individual evaluation functions of DL jobs at runtime, and thus to make placement decisions on resource allocations elastically. Here, we present a detailed design and implementation of FlowCon and conduct intensive experiments over various DL models. The results demonstrate that FlowCon significantly improves DL job completion time and resource utilization efficiency, compared to default systems. According to the results, FlowCon is able to improve the completion time by up to 68.8% and meanwhile, reduce the makespan by 18.0%, in the presence of various DL job workloads.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1997

WDM Component Requirements for Bit-Parallel Fiber Optic Computer Networks

The device specifications for an all optical bit-parallel WDM single fiber link for the cluster computer network community are intended for dissemination to the opto-electronic device research community to stimulate synergy between the two, ultimately leading to early availability of new devices to the computer network researchers.

wavelength↗
DOE OSTI2025

Predicting runtime and resource utilization of jobs on integrated cloud and HPC systems

Recent advances in virtualization technologies used in cloud computing offer performance that closely approaches bare-metal levels. Combined with specialized instance types and high-speed networking services for cluster computing, cloud platforms have become a compelling option for high-performance computing (HPC). However, most current batch job schedulers in HPC systems are designed for homogeneous clusters and make decisions based on limited information about jobs and system status. Scientists typically submit computational jobs to these schedulers with a requested runtime that is often over- or under-estimated. More accurate runtime predictions can help schedulers make better decisions and reduce job turnaround times. Here, they can also support decisions about migrating jobs to the cloud to avoid long queue wait times in HPC systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

DICER: Data Intensive Computing Environment and Runtime for Evaluating Unprecedented Scale of Geospatial-Temporal Human Mobility Data

With the significant increase in sources and volume of human mobility data through commercial data vendors as well as microsimulation of cities, the scale of geospatial-temporal data to analyze and assess for mobility characterization has grown to the level of Big Data. There are mobility related commercial organizations deploying scalable computing, but often the system architecture, workflow, and intermediate processing components are not fully disclosed in relevant scope. Current research literature has a notable lack of studies demonstrating architectures and workflows for human mobility analytics that are implemented on a TeraByte scale of geospatial-temporal data. In this context, this paper presents a hyperscale-level system solution named DICER (Data Intensive Computing Environment and Runtime) for processing and analytics of geospatial-temporal data at big data scale. Although the cluster computing architecture of DICER with Apache Spark job running on Kubernetes cluster is not new, there are innovations in the workflow, hierarchical processing logic, and a wide range of intermediate preprocessing and mobility metrics calculation. We have performed case studies to validate the effectiveness of DICER system solution by performing detailed analytics and assessment of human mobility microsimulation output at three different scopes and scale, including a usecase with 16.97 TeraByte and 259.2 Billion rows of data. In addition, we have presented another case study of utilizing DICER to perform the same mobility processing and comparative analytics on large-scale commercially available geospatial-temporal data. All these case studies validate the efficiency and usefulness of DICER in computing population mobility characteristics from geospatial-temporal trajectory data at an unprecedented scale (not only just data volume, but also combination of: number of user entities, temporal frequency, spatial resolution, data duration).

De, Debraj↗