Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “High Performance Computing (HPC)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

High Performance Computing Metrics to Enable Application-Platform Communication

Sandia has invested heavily in scientific/engineering application development and in the research, development, and deployment of large scale HPC platforms to support the computational needs of these applications. As application developers continually expand the capabilities of their software and spend more time on performance tuning of applications for these platforms, HPC platform resources are at a premium as they are a heavily shared resource serving the varied needs of many users. To ensure that the HPC platform resources are being used effciently and perform as designed, it is necessary to obtain reliable data on resource utilization that will allow us to investigate the occurrence, severity, and causes of performance-affecting contention between applications. The work presented in this paper was an initial step to determine if resource contention can be understood and minimized through monitoring, modeling, planning and infrastructure. This paper describes the set of metric definitions, identified in this research, that can be used as meaningful and potentially actionable indicators of performance-affecting contention between applications. These metrics were verified using the observed slowdown of IOR, IMB, and CTH in operating scenarios that forced contention. This paper also describes system/application monitoring activities that are critical to distilling vast amounts of data into quantities that hold the key to understanding for an application's performance under production conditions and that will ultimately aid in Sandia's efforts to succeed in extreme-scale computing.

97 MATHEMATICS AND COMPUTING↗

High-resolution hourly surrogate modeling framework for physics-based large-scale building stock modeling

Surrogate modeling can play a key role in reducing high computational burdens for large-scale physics-based modeling and uncertainty quantification. With the rapid development of large-scale building stock energy modeling, surrogate modeling has also begun to be widely applied in this field; however, most existing surrogate models lack hourly time resolution for regional-scale modeling, which is essential for understanding building demand profiles and grid impacts. Further, there is generally a lack of necessary data and feature engineering frameworks specific to building modeling for efficiently managing large datasets and complex computations. Furthermore, this paper proposes a modeling framework for large-scale (city-/region-scale), high-resolution, high-fidelity surrogate building stock energy models. Our developed framework consists of six modules: (1) building stock energy modeling (ComStock TM and ResStock TM ), (2) data engineering for large simulation data, (3) high performance computing workflow, (4) feature engineering, (5) machine learning model development, and (6) model performance evaluation. Two case studies apply the developed framework in both residential and commercial building stock analysis to demonstrate its computational efficiency and surrogate modeling accuracies. Results show that surrogate models, when efficiently trained using the HPC workflow module, reach a high level of modeling accuracy for two case studies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

HPC Institutional Computing Project: W15_lesreactiveflow KIVA-hpFE Development: A Robust and Accurate Engine Modeling Software

KIVA-hpFE is a high performance computer software for solving the physics of multi-species and multiphase turbulent reactive flow in complex geometries having immersed moving parts. The code is written in Fortran 90/95 and can be used on any computer platform with any popular complier. The code is in two versions, a serial version and a parallel version utilizing MPICH2 type Message Passing Interface (MPI or Intel MPI) for solving distributed domains. The parallel version is at least 30x faster than the serial version and much faster than our previous generation of parallel engine modeling software, by many factors. The 5th generation algorithm construction is a Galerkin type Finite Element Method (FEM) solving conservative momentum, species, and energy transport equations along with two-equation turbulent model k-ω Reynolds Averaged Navier-Stokes (RANS) model and a Vreman type dynamic Large Eddy Simulation (LES) method. The LES method is capable modeling transitional flow from laminar to fully turbulent; therefore, this LES method does not require special hybrid or blending to walls. The FEM projection method also uses a Petrov-Galerkin (P-G) stabilization along with pressure stabilization. We employ hierarchical basis sets, constructed on the fly with enrichment in areas associated with relatively larger error as determined by error estimation methods. In addition, when not using the hp-adaptive module, the code employs Lagrangian basis or shape functions. The shape functions are constructed for hexahedral, prismatic and tetrahedral elements. The software is designed to solve many types of reactive flow problems, from burners to internal combustion engines and turbines. In addition, the formulation allows for direct integration of solid bodies (conjugate heat transfer), as in heat transfer through housings, parts, cylinders. It can also easily be extended to stress modeling of solids, used in fluid structure interactions problems, solidification, porous media modeling and magneto hydrodynamics.

97 MATHEMATICS AND COMPUTING↗

Computational design and analysis of modular cells for large libraries of exchangeable product synthesis modules

Microbial metabolism can be harnessed to produce a large library of useful chemicals from renewable resources such as plant biomass. However, it is laborious and expensive to create microbial biocatalysts to produce each new product. To tackle this challenge, we have recently developed modular cell (ModCell) design principles that enable rapid generation of production strains by assembling a modular (chassis) cell with exchangeable production modules to achieve overproduction of target molecules. Previous computational ModCell design methods are limited to analyze small libraries of around 20 products. In this study, we developed a new computational method, named ModCell-HPC, that can design modular cells for large libraries with hundreds of products with a highly-parallel and multi-objective evolutionary algorithm and enable us to elucidate modular design properties. We demonstrated ModCell-HPC to design Escherichia coli modular cells towards a library of 161 endogenous production modules. From these simulations, we identified E. coli modular cells with few genetic manipulations that can produce dozens of molecules in a growth-coupled manner with different types of fermentable sugars. These designs revealed key genetic manipulations at the chassis and module levels to accomplish versatile modular cells, involving not only in the removal of major by-products but also modification of branch points in the central metabolism. We further found that the effect of various sugar degradation on redox metabolism results in lower compatibility between a modular cell and production modules for growth on pentoses than hexoses. To better characterize the degree of compatibility, we developed a method to calculate the minimal set cover, identifying that only three modular cells are all needed to couple with up 85 compatible production modules. By determining the unknown compatibility contribution metric, we further elucidated the design features that allow an existing modular cell to be re-purposed towards production of new molecules. Altogether, ModCell-HPC is a useful tool for understanding modularity of biological systems and guiding more efficient and generalizable design of modular cells that help reduce research and development cost in biocatalysis.

59 BASIC BIOLOGICAL SCIENCES↗

Tigres Workflow Library: Supporting Scientific Pipelines on HPC Systems

The growth in scientific data volumes has resulted in the need for new tools that enable users to operate on and analyze data on large-scale resources. In the last decade, a number of scientific workflow tools have emerged. These tools often target distributed environments, and often need expert help to compose and execute the workflows. Data-intensive workflows are often ad-hoc, they involve an iterative development process that includes users composing and testing their workflows on desktops, and scaling up to larger systems. In this paper, we present the design and implementation of Tigres, a workflow library that supports the iterative workflow development cycle of data-intensive workflows. Tigres provides an application programming interface to a set of programming templates i.e., sequence, parallel, split, merge, that can be used to compose and execute computational and data pipelines. We discuss the results of our evaluation of scientific and synthetic workflows showing Tigres performs with minimal template overheads (mean of 13 seconds over all experiments). We also discuss various factors (e.g., I/O performance, execution mechanisms) that affect the performance of scientific workflows on HPC systems.

97 MATHEMATICS AND COMPUTING↗

Reproducibility Practice in High-Performance Computing: Community Survey Results

The integrity of science and engineering research is grounded in assumptions of rigor and transparency on the part of those engaging in such research. HPC community effort to strengthen rigor and transparency take the form of reproducibility efforts. In a recent survey of the SC conference community, we collected information about the SC Reproducibility Initiative activities. Here, we present the survey results in this paper. Results show that the reproducibility initiative activities have contributed to higher levels of awareness on the part of SC conference technical program participants, and hint at contributing to greater scientific impact for the published papers of the SC conference series. Stringent point-of-manuscript-submission verification is problematic for reasons we point out, as are inherent difficulties of computational reproducibility in HPC. Future efforts should better decouple the community educational goals from goals that specifically strengthen a research work's potential for long-term impact through reuse 5-10 years down the road.

97 MATHEMATICS AND COMPUTING↗

ARES v1.x - Performance Portable Tool to Simulate Supernovae based on Parthenon Framework

Historically, codes for simulating supernovae (such as Arepo, FLASH or LEAFS) have been at the forefront of scientific high-performance computing to the immense computational resources required for full 3D simulations. However, given the shift towards heterogenous HPC architectures, many current-generation codes are at the risk of losing their competitiveness as they are only designed to run on homogeneous CPU-only systems. There exist several efforts to enable these codes for GPU’s, however, these efforts only consider specific architectures or vendors (e.g., implement only CUDA or HIP), limiting themselves to a small range of exascale computing systems. Frameworks such as Kokkos aim to provide a framework which is agnostic of the targeted architecture, enabling the development of performant and portable code. In the Ares code, we develop a performance portable tool to simulate supernovae based on the Parthenon Framework, which in turn uses Kokkos in the background. Here, the Parthenon Framework provides an interface to the underlying mesh-refinement routines, which form the backbone of our code. In addition, we incorporate the already existing Singularity-EOS toolkit to provide us with various equations of state, primarily the Helmholtz equation of state. We also include the JINA Reaclib as a basis for our nuclear network solver. Finally, we implement a gravity solver to complete the required physics. This setup will provide us with a minimal code base to simulate supernova in a similar style to the tried-and-tested Arepo code, but in a futureproof performance portable framework.

Lim, Hyun↗

Co-design in ACES and exascale.

The Alliance for Computing at the Extreme Scale (ACES) is a Los Alamos and Sandia collaboration encompassing not only HPC procurements and operations, but also computer science and architecture research and development. One area of focus within ACES relates to the critical technology developments for future high performance computing systems and the applications that would run on them, and ACES is heavily involved in the proposed DOE Exascale Initiative. The proposed Exascale Initiative emphasizes the need for co-design, which is the three-way collaborative and concurrent design of HPC hardware, software, and the applications themselves. Transformational changes will occur not only in HPC hardware, but also in the applications space, and taken together these will require transformation changes in the overall software layers supporting the programming models, tools, runtimes, file systems, and operating systems. Co-design involving all three areas of hardware, software, and applications will be the key to success. This talk will outline key aspects of the Exascale Initiative and its emphasis on co-design. It will provide some examples from LANL & Sandia experiences in co-design including aspects of the innovative Roadrunner architecture and software, a ACES-Cray project studying advanced interconnects within Cray, and Sandia's work in computer system simulators and mini-applications.

99 GENERAL AND MISCELLANEOUS↗

Delta: Data Reduction for Integrated Application Workflows

Integrated Application Workflows (IAWs) run multiple simulation workflow components concurrently on an HPC resource connecting these components using compute area resources and compensating for any performance or data processing rate mismatches. These IAWs require high frequency and high volume data transfers between compute nodes and staging area nodes during the lifetime of a large parallel computation. The available network band-width between the two areas may not be enough to efficiently support the data movement. As the processing power available to compute resources increases, the requirements for this data transfer will become more difficult to satisfy and perhaps will not be satisfiable at all since network capabilities are not expanding at a comparable rate. Furthermore, energy consumption in HPC environments is expected to grow by an order of magnitude as exascale systems become a reality. The energy cost of moving large amounts of data frequently will contribute to this issue. It is necessary to reduce the volume of data without reducing the quality of data when it is being processed and analyzed. Delta resolves the issue by addressing the lifetime data transfer operations. Delta removes subsequent identical copies of already transmitted data during transfers and restores those copies once the data has reached the destination. Delta is able to identify duplicated information and determine the most space efficient way to represent it. Initial tests show about 50% reduction in data movement while maintaining the same data quality and transmission frequency.

97 MATHEMATICS AND COMPUTING↗

Reliable High Performance Peta- and Exa-Scale Computing

As supercomputers become larger and more powerful, they are growing increasingly complex. This is reflected both in the exponentially increasing numbers of components in HPC systems (LLNL is currently installing the 1.6 million core Sequoia system) as well as the wide variety of software and hardware components that a typical system includes. At this scale it becomes infeasible to make each component sufficiently reliable to prevent regular faults somewhere in the system or to account for all possible cross-component interactions. The resulting faults and instability cause HPC applications to crash, perform sub-optimally or even produce erroneous results. As supercomputers continue to approach Exascale performance and full system reliability becomes prohibitively expensive, we will require novel techniques to bridge the gap between the lower reliability provided by hardware systems and users unchanging need for consistent performance and reliable results. Previous research on HPC system reliability has developed various techniques for tolerating and detecting various types of faults. However, these techniques have seen very limited real applicability because of our poor understanding of how real systems are affected by complex faults such as soft fault-induced bit flips or performance degradations. Prior work on such techniques has had very limited practical utility because it has generally focused on analyzing the behavior of entire software/hardware systems both during normal operation and in the face of faults. Because such behaviors are extremely complex, such studies have only produced coarse behavioral models of limited sets of software/hardware system stacks. Since this provides little insight into the many different system stacks and applications used in practice, this work has had little real-world impact. My project addresses this problem by developing a modular methodology to analyze the behavior of applications and systems during both normal and faulty operation. By synthesizing models of individual components into a whole-system behavior models my work is making it possible to automatically understand the behavior of arbitrary real-world systems to enable them to tolerate a wide range of system faults. My project is following a multi-pronged research strategy. Section II discusses my work on modeling the behavior of existing applications and systems. Section II.A discusses resilience in the face of soft faults and Section II.B looks at techniques to tolerate performance faults. Finally Section III presents an alternative approach that studies how a system should be designed from the ground up to make resilience natural and easy.

97 MATHEMATICS AND COMPUTING↗

Comprehensive Measurement and Analysis of the User-Perceived I/O Performance in a Production Leadership-Class Storage System

With the increase of the scale and intensity of the parallel I/O workloads generated by those scientific applications running on high performance computing facilities, understanding the I/O dynamics, especially the root cause of the I/O performance variability and degradation in HPC environment, have become extremely critical to the HPC community. In this paper, we run extensive I/O measuring tests on a production leadership-class storage system to capture the performance variabilities of large-scale parallel I/O. Analyzing these results and its statistic correlation revealed some valuable insights into the characteristics of the storage system and the root cause of I/O performance variability. Further, we leverage these findings and propose an I/O middleware design refactoring which can improve the performance of the parallel I/O by optimizing the data striping and placement. Our preliminary evaluation results demonstrate the proposed approach can reduce the average per-process write latency by at least 80% and the maximum per-process write latency by at least 20%.

Wan, Lipeng↗

The Locus Algorithm IV: Performance metrics of a grid computing system used to create catalogues of optimised pointings

This paper discusses the requirements for and performance metrics of the the Grid Computing system used to implement the Locus Algorithm to identify optimum pointings for differential photometry of 61,662,376 stars and 23,779 quasars. Initial operational tests indicated a need for a software system to analyse the data and a High Performance Computing system to run that software in a scalable manner. Practical assessments of the performance of the software in a serial computing environment were used to provide a benchmark against which the performance metrics of the HPC solution could be compared, as well as to indicate any bottlenecks in performance. These performance metrics indicated a distinct split in the performance dictated more by differences in the input data than by differences in the design of the systems used. This indicates a need for experimental analysis of system performance, and suggests that algorithmic complexity analyses may lead to incorrect or naive conclusions, especially in systems with high data I/O overhead such as grid computing. Further, it implies that systems which reduce or eliminate this bottleneck such as in-memory processing could lead to a substantial increase in performance.

Creaner, Oisín↗

Blockchain for High Performance Data Integrity and Provenance

Describes the development of a test bed to evaluate the the feasibility of different Blockchain mechanisms for determining the integrity of data computed on a high performance computer while limiting the degradation of performance.

99 GENERAL AND MISCELLANEOUS↗

An Optimizing Compiler for Petascale I/O on Leadership Class Architectures

In high-performance computing systems, parallel I/O architectures usually have very complex hierarchies with multiple layers that collectively constitute an I/O stack, including high-level I/O libraries such as PnetCDF and HDF5, I/O middleware such as MPI-IO, and parallel file systems such as PVFS and Lustre. Our project explored automated instrumentation and compiler support for I/O intensive applications. Our project made significant progress towards understanding the complex I/O hierarchies of high-performance storage systems (including storage caches, HDDs, and SSDs), and designing and implementing state-of-the-art compiler/runtime system technology that targets I/O intensive HPC applications that target leadership class machine. This final report summarizes the major achievements of the project and also points out promising future directions.

97 MATHEMATICS AND COMPUTING↗