Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “cluster computer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Fast Fault Injection and Sensitivity Analysis for Collective Communications, In: 2015 IEEE International Conference on Cluster Computing

The collective communication operations, which are widely used in parallel applications for global communication and synchronization are critical for application's performance and scalability. However, how faulty collective communications impact the application and how errors propagate between the application processes is largely unexplored. One of the critical reasons for this situation is the lack of fast evaluation method to investigate the impacts of faulty collective operations. The traditional random fault injection methods relying on a large amount of fault injection tests to ensure statistical significance require a significant amount of resources and time. These methods result in prohibitive evaluation cost when applied to the collectives. In this paper, we introduce a novel tool named Fast Fault Injection and Sensitivity Analysis Tool (FastFIT) to conduct fast fault injection and characterize the application sensitivity to faulty collectives. The tool achieves fast exploration by reducing the exploration space and predicting the application sensitivity using Machine Learning (ML) techniques. A basis for these techniques are implicit correlations between MPI semantics, application context, critical application features, and application responses to faulty collective communications. The experimental results show that our approach reduces the fault injection points and tests by 97% for representative benchmarks (NAS Parallel Benchmarks (NPB)) and a realistic application (Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS)) on a production supercomputer. Further, we statistically generalize the application sensitivity to faulty collective communications for these workloads, and present correlation between application features and the sensitivity.

Computer Science↗
DOE OSTI2015

TRIO: Burst Buffer Based I/O Orchestration, In: 2015 IEEE International Conference on Cluster Computing

The growing computing power on leadership HPC systems is often accompanied by ever-escalating failure rates. Checkpointing is a common defensive mechanism used by scientific applications for failure recovery. However, directly writing the large and bursty checkpointing dataset to parallel file systems can incur significant I/O contention on storage servers. Such contention in turn degrades bandwidth utilization of storage servers and prolongs the average job I/O time of concurrent applications. Recently burst buffers have been proposed as an intermediate layer to absorb the bursty I/O traffic from compute nodes to storage backend. But an I/O orchestration mechanism is still desirable to efficiently move checkpointing data from burst buffers to storage backend. In this paper, we propose a burst buffer based I/O orchestration framework, named TRIO, to intercept and reshape the bursty writes for better sequential write traffic to storage servers. Meanwhile, TRIO coordinates the flushing orders among concurrent burst buffers to alleviate the contention on storage server. Our experimental results demonstrated that TRIO could efficiently utilize storage bandwidth and reduce the average job I/O time by 37% on average for data-intensive applications in typical checkpointing scenarios.

Computer Science↗
DOE OSTI2015

An Approach to Selecting Thread + Process Mixes for Hybrid MPI + OpenMP Applications, In: 2015 IEEE International Conference on Cluster Computing

Hybrid MPI + OpenMP is a popular means of programming modern machines that feature substantial parallelism both off-node and on-node. Determining the right mix of the two programming models to use, however, is not as straightforward as simply using exclusively OpenMP on-node and limiting MPI to only inter-node communication. We present a step-by-step methodology to help make the decision of which mix of the two programming models to use. It starts with an estimate of the performance of a generic hybrid application on a given machine and incorporates additional available information about the specific application and the machine to provide guidance for selecting effective mixes of MPI processes and OpenMP threads to use when running that application on the machine in question. We validate our approach on four different applications on an IBM Blue Gene/Q, a Cray XK7, and a Cray XC30.

Computer Science↗
DOE OSTI2018

High Performance Computing Cluster Engineering Academy

Collection of tools and code for evaluation testing of HPC orchestration solutions by HPCCEA summer students. These scripts and configuration files were written by HPCCEA summer students as part of their summer internship where they explore alternative solutions to our current HPC cluster management software. These scripts/configuration files range from using configuration management tools (Puppet, Chef, Ansible, Fabric, etc.) to scripting languages (Phython, BASH, Perl, etc.). Each year, new students will be added to the project and expand on previous years work.

Bennett, ThomasG↗
DOE OSTI2022

DRAS: Deep Reinforcement Learning for Cluster Scheduling in High Performance Computing

Cluster schedulers are crucial in high-performance computing (HPC). They determine when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with specific HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling. An efficient training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a specific scheduling objective given by the system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. We implement DRAS into a HPC scheduling platform called CQGym. CQGym provides a common platform allowing users to flexibly evaluate DRAS and other scheduling methods such as heuristic and optimization methods. Furthermore, the experiments using CQGym with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 50%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Proposal for grid computing for nuclear applications

The use of computer clusters for computational sciences including computational physics is vital as it provides computing power to crunch big numbers at a faster rate. In compute intensive applications that requires high resolution such as Monte Carlo simulation, the use of computer clusters in a grid form that supplies computational power to any nodes within the grid that needs computing power, has now become a necessity. In this paper, we described how the clusters running on a specific application could use resources within the grid, to run the applications to speed up the computing process.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

Cluster Node Computing for Target Generation Systems in Aircraft Simulations

Target generation systems provide the positions of aircraft in air traffic simulations. As the scope of the simulation domains expand, there is a need to develop systems that can provide position reports for thousands of aircraft simultaneously and at update rates that support out-the-window visualization. This paper discusses the motivation and reasoning behind investigating development of a next generation target generator through distributed computing using clustered node processing and how a target generation system benefit future research that utilizes human-in-the-loop simulations.

air traffic simulations↗
NASA NTRS2019

Cluster Node Computing for Target Generation Systems in Aircraft Simulations

Target generation systems provide virtual position and state reports of aircraft for air traffic simulations. As the scope and scale of the simulation domains expand, there is a need to develop systems that can generate position reports for thousands of simulated aircraft simultaneously at high update rates that support out-the-window visualization. This paper discusses the motivation and reasoning behind investigating development of a next generation target generator through distributed computing using clustered node processing and the benefits such a target generation system has on future research that utilizes human-in-the-loop simulations.

Monheim, Spencer↗
DOE OSTI2016

An Innovative Approach to Bridge a Skill Gap and Grow a Workforce Pipeline: The Computer System, Cluster, and Networking Summer Institute

Sustainable and effective computing infrastructure depends critically on the skills and expertise of domain scientists and of committed and well-trained advanced computing professionals. But, in its ongoing High Performance Computing (HPC) work, Los Alamos National Laboratory noted a persistent shortage of well-prepared applicants, particularly for entry-level cluster administration, file systems administration, and high speed networking positions. Further, based upon recruiting efforts and interactions with universities graduating students in related majors of interest (e.g., computer science (CS)), there has been a long standing skillset gap, as focused training in HPC topics is typically lacking or absent in undergraduate and in even many graduate programs. Given that the effective operation and use of HPC systems requires specialized and often advanced training, that there is a recognized HPC skillset gap, and that there is intense global competition for computing and computational science talent, there is a long-standing and critical need for innovative approaches to help bridge the gap and create a well-prepared, next generation HPC workforce. Our paper places this need in the context of the HPC work and workforce requirements at Los Alamos National Laboratory (LANL) and presents one such innovative program conceived to address the need, bridge the gap, and grow an HPC workforce pipeline at LANL. The Computer System, Cluster, and Networking Summer Institute (CSCNSI) completed its 10th year in 2016. The story of the CSCNSI and its evolution is detailed below with a description of the design of its Boot Camp, and a summary of its success and some key factors that have enabled that success.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2004

NASA Multidimensional Stirling Convertor Code Developed

A high-efficiency Stirling Radioisotope Generator (SRG) for use on potential NASA Space Science missions is being developed by the Department of Energy, Lockheed Martin, Stirling Technology Company, and the NASA Glenn Research Center. These missions may include providing spacecraft onboard electric power for deep space missions or power for unmanned Mars rovers. Glenn is also developing advanced technology for Stirling convertors, aimed at substantially improving the specific power and efficiency of the convertor and the overall power system. Performance and mass improvement goals have been established for second- and third-generation Stirling radioisotope power systems. Multiple efforts are underway to achieve these goals, both in house at Glenn and under various grants and contracts. These efforts include the development of a multidimensional Stirling computational fluid dynamics code, high-temperature materials, advanced controllers, an end-to-end system dynamics model, low-vibration techniques, advanced regenerators, and a lightweight convertor. Under a NASA grant, Cleveland State University (CSU) and its subcontractors, the University of Minnesota (UMN) and Gedeon Associates, have developed a twodimensional computer simulation of a CSUmod Stirling convertor. The CFD-ACE commercial software developed by CFD Research Corp. of Huntsville, Alabama, is being used. The CSUmod is a scaled version of the Stirling Technology Demonstrator Convertor (TDC), which was designed and fabricated by the Stirling Technology Company and is being tested by NASA. The schematic illustrates the structure of this model. Modeled are the fluid-flow and heat-transfer phenomena that occur in the expansion space, the heater, the regenerator, the cooler, the compression space, the surrounding walls, and the moving piston and displacer. In addition, the overall heat transfer, the indicated power, and the efficiency can be calculated. The CSUmod model is being converted to a two-dimensional model of the TDC at NASA Glenn. Validation of the multidimensional Stirling code is an important part of the grant effort. UMN has been generating data in an oscillating-flow test facility using two different test sections: a 90 turn and a cooler/regenerator/heater test section. CSU has created computational fluid dynamics models of both these test sections and has been making comparisons with the data, then improving their models to improve the agreement with the test data. CSU has also been using data available in the literature for code validation. UMN is now preparing to begin fabrication of a new 180 turn test section that will be more representative of certain portions of the Stirling engine geometry. Simulations to almost periodic steady state with the two-dimensional CSUmod model indicate that, to reach periodic steady state on a single 2-GHz desktop computer, 75 to 100 complete simulation cycles would be required and between 1 and 2 months of computer time. Therefore, Glenn has purchased the first 8 computers, of a 64-computer cluster, to be run in parallel to accelerate the simulation. On the basis of CFD Research Corp.'s experience with running the parallelized version of CFD-ACE on their clusters, we estimate that the complete 64-computer cluster will reduce simulation computing time by a factor of about 40. Plans are to continue development of these multidimensional Stirling codes and to use them to study the fluid-flow and heat-transfer phenomena that occur inside Stirling convertors. This is expected to lead to improved thermodynamic loss understanding, onedimensional design and performance codes, and engine performance.

Tew, Roy C.↗