Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “High Performance Computing (HPC)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

Yearly Report for the Period Jan-Dec 2012: IC PROJECT W12c_earthuq - “High-Performance Computing for Uncertainty Quantification of 3D Earth Models”

This proposal was written to be a new project leveraging on computational achievements of the two IC11 proposals “3D Geophysical Modeling Validation” and “Time-Reversal Imaging of tremor sources”. W12c_earthuq involves additional codes/tools other than SPECFEM3D for data processing and planned a strong effort to move toward modeling at much larger scales than the aforementioned projects. W12c_earthuq goal was to use extensive HPC resources at LANL to assess the uncertainty of high resolution 3D geophysical models. Validation of these models as planned in W11_geophys is the first step in a full assessment of uncertainty. So, the scientific and programmatic impacts of this project are the same as W11_geophys. The main effort related to w12c_earthuq is the preparation to transition to large runs with a resolution down to 2s. We already report the work done in the report for w11_geophys. Such resolution requires meshes with 10 billions elements with a size of about 5km and a few thousands of compute nodes. First tests on mustang have revealed that the I/O part of the code need to be reviewed because it opens too many files in its current version.

58 GEOSCIENCES↗

Exploratory Visual Analysis of Anomalous Runtime Behavior in Streaming High Performance Computing Applications

Online analysis of runtime behavior is essential for performance tuning in streaming scientific workflows. Integration of anomaly detection and visualization is necessary to support human-centered analysis, such as verification of candidate anomalies utilizing domain knowledge. In this work, we propose an efficient and scalable visual analytics system for online performance analysis of scientific workflows toward the exascale scenario. Here, our approach uses a call stack tree representation to encode the structural and temporal information of the function executions. Based on the call stack tree features (e.g., execution time of the root function or vector representation of the tree structure), we employ online anomaly detection approaches to identify candidate anomalous function executions. We also present a set of visualization tools for verification and exploration in a level-of-detailed manner. General information, such as distribution of execution times, are provided in an overview visualization. The detailed structure (e.g., function invocation relations) and the temporal information (e.g., message communication) of the execution call stack of interest are also visualized. The usability and efficiency of our methods are verified in a real-world HPC application.

97 MATHEMATICS AND COMPUTING↗

Online Diagnosis of Performance Variation in HPC Systems Using Machine Learning

As the size and complexity of HPC systems grow in line with advancements in hardware and software technology, HPC systems increasingly suffer from performance variation due to shared resource contention as well as software- and hardware-related problems. Such performance variations can lead to failures and inefficiencies, and are among the main challenges in system resiliency. To minimize the impact of performance variation, one must quickly and accurately detect and diagnose the anomalies that cause the variation and take mitigating actions. However, it is difficult to identify anomalies based on the voluminous, high-dimensional, and noisy data collected by system monitoring infrastructures. This paper presents a novel machine learning based framework to automatically diagnose performance anomalies at runtime. Our framework leverages historical resource usage data to extract signatures of previously-observed anomalies. We first convert the collected time series data into easy-to-compute statistical features. We then identify the features that are required to detect anomalies, and extract the signatures of these anomalies. At runtime, we use these signatures to diagnose anomalies with negligible overhead. Here, we evaluate our framework using experiments on a real-world HPC supercomputer and demonstrate that our approach successfully identifies 98% of injected anomalies and consistently outperforms existing anomaly diagnosis techniques.

97 MATHEMATICS AND COMPUTING↗

Navigating Exascale Operational Data Analytics: From Inundation to Insight

In this paper, we address the challenges in achieving sustainable data-driven efficiency by providing a detailed exploration of the end-to-end operational data analytics (ODA) framework that evolved through two generations of supercomputer systems at the Oak Ridge Leadership Computing Facility (OLCF). This framework addresses large data streams ingested from heavily instrumented HPC environment that accumulates multi-terabytes per day. We outline the multifaceted data life cycle across HPC procurement, operations, and research & development, identifying key obstacles and design decisions that shape effective strategies in building and supporting data pipelines end-to-end. By sharing key insights and lessons learned from our experience, we offer recommendations for the HPC community on enabling sustainable operational data analytics and beyond. Our contributions aim to bridge the gap between potential and real benefits of operational data, guiding future efforts towards integrated and sustainable operational intelligence in high-performance computing environments.

Shin, Woong↗

2016 High Performance Computing Annual Report

Sandia National Laboratories’ California site is celebrating its 60th anniversary (1956 to 2016), and high performance computing has been a key enabler for its scientists and engineers throughout much of its history. Since its founding, Sandia California has helped pioneer the use of HPC platforms including hosting Sandia’s first Cray-1 supercomputer in the 1970s and supporting development of scalable cluster computing platforms to create a new paradigm for cost-effective supercomputing in the 1990s. Recent decades of investment in creation of scalable application frameworks for scientific computing have also enabled new generations of modeling and simulation codes. These resources have facilitated computational analysis of complex phenomena in diverse applications spanning national defense, energy, and homeland security. Today, Sandia California researchers work with partners in academia, industry, and national labs to evolve the state-of-the-art in HPC, modeling, and data analysis (including foundational capabilities for exascale computing platforms) and apply them in transformational ways. Research efforts include mitigating the effects of silent hardware failures that can jeopardize the results of large-scale computations, developing exascale-capable asynchronous task-parallel programming models and runtime systems, formulating new techniques to better explore and analyze extreme-scale data sets, and increasing our understanding of materials and chemical sciences which has applications spanning nuclear weapons stockpile stewardship to more efficient automobile engines. The following section highlights some of these research and applications projects and further illustrates the breadth of our HPC capabilities.

97 MATHEMATICS AND COMPUTING↗

Distributing Power Grid State Estimation on HPC Clusters A System Architecture Prototype

The future power grid is expected to further expand with highly distributed energy sources and smart loads. The increased size and complexity lead to increased burden on existing computational resources in energy control centers. Thus the need to perform real-time assessment on such systems entails efficient means to distribute centralized functions such as state estimation in the power system. In this paper, we present our early prototype of a system architecture that connects distributed state estimators individually running parallel programs to solve non-linear estimation procedure. The prototype consists of a middleware and data processing toolkits that allows data exchange in the distributed state estimation. We build a test case based on the IEEE 118 bus system and partition the state estimation of the whole system model to available HPC clusters. The measurement from the testbed demonstrates the low overhead of our solution.

Distributed architecture↗

Hpc Natural Language Understanding (nlu) Dataset

This code provides natural language annotation and labels for multiple activities in high performance. The purpose is to enable machine learning for natural language processing algorithms with high performance computing systems.

Biggs, BrandonS↗

...And Eat it Too: High Read Performance in Write-Optimized HPC I/O Middleware File Formats

As HPC applications run on increasingly high process counts on larger and larger machines, both the frequency of checkpoints needed for fault tolerance and the resolution and size of Data Analysis Dumps are expected to increase proportionally. In order to maintain an acceptable ratio of time spent performing useful computation work to time spent performing I/O, write bandwidth to the underlying storage system must increase proportionally to this increase in the checkpoint and computation size. Unfortunately, popular scientific self-describing file formats such as netCDF and HDF5 are designed with a focus on portability and flexibility. Extra care and careful crafting of the output structure and API calls is required to optimize for write performance using these APIs. To provide sufficient write bandwidth to continue to support the demands of scientific applications, the HPC community has developed a number of I/O middleware layers, that structure output into write-optimized file formats. However, the obvious concern with any write optimized file format would be a corresponding penalty on reads. In the log-structured filesystem, for example, a file generated by random writes could be written efficiently, but reading the file back sequentially later would result in very poor performance. Simulation results require efficient read-back for visualization and analytics, and though most checkpoint files are never used, the efficiency of a restart is very important in the face of inevitable failures. The utility of write speed improving middleware would be greatly diminished if it sacrificed acceptable read performance. In this paper we examine the read performance of two write-optimized middleware layers on large parallel machines and compare it to reading data natively in popular file formats. The two I/O middleware layer examined in this paper are the Adaptable IO System (ADIOS), a library-based approach developed at Oak Ridge National Laboratory to provide a high-level IO API that can be used in place of netCDF or HDF5 to do much more aggressive write-behind and efficient reordering of data locations within the file; and the Parallel Log-structured Filesystem (PLFS), a stackable FUSE filesystem approach developed at Los Alamos National Laboratory that decouples concurrent writes to improve the speed of checkpoints. Since ADIOS is an I/O componentization that affords selection of different I/O methods at or during runtime, through a single API, users can have access to MPI-IO, Posix-IO, HDF5, netCDF, and staging methods. The ADIOS BP file format is a new log-file format that has a superset of the features of both HDF5 and netCDF, but is designed to be portable and flexible while being optimized for writing. PLFS, on the other hand, is mounted as a stackable filesystem on top of an existing parallel filesystem. Reads or writes to the PLFS filesystem are transparently translated into operations on per-process log files stored in the underlying parallel filesystem. Since PLFS performs this translation without application modification, users can write in HDF5, netCDF, or app-specific file formats and PLFS will store the writes in a set of efficiently written log-formatted files, while presenting the user with a logical 'flat' file on reads. Despite their different approaches, the commonality behind both of these middleware systems is that they both write to a log file format. As shown in previous publications, writes are fully optimized in both systems, sometimes resulting in 100x improvements over writing data in popular file formats. But as mentioned above, writing is only one part of the story. In this paper we examine the read performance of our middleware layers on large parallel machines and compare these to reading data either natively or from other popular file formats. We compare the reading performance in two different scenarios: (1) Reading back restarts from the same number of processors as wrote the data and (2) Reading back restart data from a different number of processors. We observe that not only can write-optimized I/O middleware be built to not penalize read speeds, but for important workloads, techniques that improve write performance can, perhaps counterintuitively, improve read speeds over reading from a contiguously organized file format. In the remainder of this paper, we investigate this further through case studies of PLFS and ADIOS on simulation checkpoint restarts.

99 GENERAL AND MISCELLANEOUS↗

Performance Characterization of De Novo Genome Assembly on Leading Parallel Systems.

De novo genome assembly is one of the most important and challenging computational problems in modern genomics; further, it shares algorithms and communication patterns important to other graph analytic and irregular applications. Unlike simulations, it has no floating point arithmetic and is dominated by small memory transactions within and between computing nodes. In this work, we focus on the highly scalable HipMer assembler and identify the dominant algorithms and communication patterns, also using microbenchmarks to capture the workload. We evaluate HipMer on a variety of platforms from the latest HPC systems to ethernet clusters. HipMer performs well on all single node systems, including the Xeon Phi manycore architecture. Given large enough problems, it also demonstrates excellent scaling across nodes in an HPC system, but requires a high speed network with low overhead and high injection rates. Our results shed light on the architectural features that are most important for achieving good parallel efficiency on this and related problems.

Ellis, Marquita↗

OpenCHAMI Developer Summit [Slides]

The mission of the OpenCHAMI consortium is to steward the collaborative development and continuous evolution of cloud-like software to manage High Performance Computing capacity regardless of the size or deployment platform. We are guided by the operators and practitioners who use modern tooling and concepts to address the needs of classical HPC applications and the growing AI/ML and Data Science community that wish to leverage HPC capacity within their own workflows, to meet their needs with their own tools.

97 MATHEMATICS AND COMPUTING↗

Towards a supported common NEAMS software stack

The NEAMS IPSC's are developing multidimensional, multiphysics, multiscale simulation codes based on first principles that will be capable of predicting all aspects of current and future nuclear reactor systems. These new breeds of simulation codes will include rigorous verification, validation and uncertainty quantification checks to quantify the accuracy and quality of the simulation results. The resulting NEAMS IPSC simulation codes will be an invaluable tool in designing the next generation of Nuclear Reactors and also contribute to a more speedy process in the acquisition of licenses from the NRC for new Reactor designs. Due to the high resolution of the models, the complexity of the physics and the added computational resources to quantify the accuracy/quality of the results, the NEAMS IPSC codes will require large HPC resources to carry out the production simulation runs.

97 MATHEMATICS AND COMPUTING↗

Lamellar: A Rust-based Asynchronous Tasking and PGAS Runtime for High Performance Computing

Cybersecurity is one of the largest concerns in modern computing, impacting and dictating how governments, private corporations, and individuals interact with and live in an increasingly digital world. The NSA has recently released a memo [ 1] on “Software Memory Safety” where they highlight that both Microsoft and Google have stated around 70% of software vulnerabilities were due to memory safety issues. Although languages such as C and C++ provide freedom and flexibility with memory management, guaran- teeing safety falls mostly on the developer. The NSA recommends using “memory safe” languages whenever possible. In this paper we introduce Lamellar, an asynchronous tasking and PGAS HPC runtime written in Rust, one such "memory safe" language. We describe the entire Lamellar stack, from network interfaces to high- level abstractions such as distributed LamellarArrays and Active Messages. We conclude by showing comparable performance to legacy PGAS runtimes (e.g. OpenSHMEM) on a subset of the BALE kernel suite while maintaining strong memory safety principles.

HPC Software Systems, Rust Programming Language, P↗

On-demand data analytics in HPC environments at leadership computing facilities: Challenges and experiences

Scaling-up scientific data analysis and machine learning algorithms for data-driven discovery is a grand challenge that we face today. Despite the growing need for analysis from science domains that are generating `Big Data' from instruments and simulations, building high-performance analytical workflows of data-intensive algorithms have been daunting because: (i) the `Big Data' hardware and software architecture landscape is constantly evolving, (ii) newer architectures impose new programming models, and (iii) data-parallel kernels of analysis algorithms and their performance facets on different architectures are poorly understood. To address these problems, we have: (i) identified scalable data-parallel kernels of popular data analysis algorithms, (ii) implemented `Mini-Apps' of those kernels using different programming models (e.g. Map Reduce, MPI, etc.), (iii) benchmarked and validated the performance of the kernels in diverse architectures. In this paper, we discuss two of those Mini-Apps and show the execution of principal component analysis built as a workflow of the Mini-Apps. We show that Mini-Apps enable scientists to (i) write domain-specific data analysis code that scales on most HPC hardware and (ii) and offers the ability (most times with over a 10x speed-up) to analyze data sizes 100 times the size of what off-the-shelf desktop/workstations of today can handle.

Harney, John↗

Correction to Reconciling Work Functions and Adsorption Enthalpies for Implicit Solvent Models: A Pt (111)/Water Interface Case Study

This errata is to correct the acknowledgment in the original published article. G.B. (partial support), V.-A.G., M.-T.N., and R.R. were supported by the U.S. Department of Energy, Office of Science, Basic Energy Sciences, Chemical Sciences, Geosciences, and Biosciences Division. Computer resources were provided by Research Computing at Pacific Northwest National Laboratory and the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility operated under Contract No. DE-AC02-05CH11231. PNNL is operated by Battelle for the U.S. Department of Energy under Contract DE-AC05-76RL01830. G.B. acknowledges the EPSRC for partial support in PhD funding. The authors acknowledge the use of the IRIDIS High Performance Computing Facility (IRIDIS 5) and associated support services at the University of Southampton in the completion of this work. We are grateful to the UK Materials and Molecular Modeling Hub (Thomas HPC) for computational resources, which is partially funded by EPSRC (EP/P020194/1).

continuum solvation, metal/water interfaces, ab in↗

Kokkos Programming Model and Library

High performance computing is in the midst of a many-core revolution driven by rapidly evolving microchips with diverse and heterogeneous architectures, increasing concurrency (thread count), and decreasing memory per thread. Our HPC applications face a major challenge to effectively use these microchips with a single code base that is performance portable across these evolving architectures. Kokkos was developed to enable on-chip concurrency and manage architecture-dependent constraints on memory access patterns. The goal is to maximize the amount of application code that is portable across diverse architectures and still obtain the same (or nearly) performance as a variant of that code written specifically for that architecture. The Kokkos team has ongoing R&D co-design collaborations with Sandia's and vendors' microchip and system engineers to rapidly incorporate evolving computer architecture features into Kokkos. The team is also providing leadership for DOE laboratories' participation on the ISO/C++ language standard committee to subsuming Kokkos features into a future standard.

97 MATHEMATICS AND COMPUTING↗

NLR HPC Facility Power Usage Effectiveness (PUE) Data

Timeseries of Energy Systems Integration Facility (ESIF) Data Center Power Usage Effectiveness (PUE) Data provided in Parquet and compressed CSV formats Power Metrics Timeseries Fields: ts: Timestamp cooling_kw: Cooling (kilowatts) - Captures the power used by fans and pipe trace heaters associated with outdoor cooling equipment. The dedicated tower filter pump power is also captured as cooling load. energy_reuse: Energy Reuse Effectiveness hvac_kw: Heating, ventilation, and air conditioning (kilowatts) - Captures fan walls, fan coils that support the data center electrical rooms, and the make-up air unit. it_power_kw: IT equipment (kilowatts) - Captures power used by the IT equipment on the data center floor. plug_and_light_kw: Lights and utility plugs (kilowatts) - Captures power associated with the data center and dedicated mechanical room. The crank-case heater for the emergency standby generator is also captured as light and plug load. pue: Power Usage Effectiveness pump_kw: Pumps (kilowatts) - Captures power from pumps that move water in the data center Energy Recover Water loop and the Tower Water loops, and also captures power used by the boost pumps that circulate water through the fan walls. Note: The tower filter pump runs constantly to filter water from the data center cooling tower system, so 2.67 kilowatts are attributed to this pump and that is not reflected in this data field. day: Day of month Outside Weather Station Timeseries Fields: ts: Timestamp outside_air_humidity: Outside air humidity - Relative humidity percent outside_air_temp: Outside air temperature - Degrees Fahrenheit day: Day of month More detail: High-Performance Computing Data Center Power Usage Effectiveness

97 MATHEMATICS AND COMPUTING↗