SEARCH · Engineering Papers

Results for “Software Stack”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2012

Irregular Applications: Architectures & Algorithms

Irregular applications are characterized by irregular data structures, control and communication patterns. Novel irregular high performance applications which deal with large data sets and require have recently appeared. Unfortunately, current high performance systems and software infrastructures executes irregular algorithms poorly. Only coordinated efforts by end user, area specialists and computer scientists that consider both the architecture and the software stack may be able to provide solutions to the challenges of modern irregular applications.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2014

Fault Tolerance for OpenSHMEM. In: PGAS '14 Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models. Article No. 23

On today's supercomputing systems, faults are becoming a norm rather than an exception. Given the complexity required for achieving expected scalability and performance on future systems, this situation is expected to become worse. The systems are expected to function in a nearly constant presence of faults. To be productive on these systems, programming models will require both hardware and software to be resilient to faults. With the growing importance of PGAS programming model and OpenSHMEM, as a part of HPC software stack, a lack of a fault tolerance model may become a liability for its users. Towards this end, in this paper, we discuss the viability of using checkpoint/restart as a fault-tolerance method for OpenSHMEM, propose a selective checkpoint/restart fault-tolerance model, and discuss challenges associated with implementing the proposed model.

Hao, Pengfei↗

DOE OSTI2015

Barrier elision for production parallel programs

Large scientific code bases are often composed of several layers of runtime libraries, implemented in multiple programming languages. In such situation, programmers often choose conservative synchronization patterns leading to suboptimal performance. In this paper, we present context-sensitive dynamic optimizations that elide barriers redundant during the program execution. In our technique, we perform data race detection alongside the program to identify redundant barriers in their calling contexts; after an initial learning, we start eliding all future instances of barriers occurring in the same calling context. We present an automatic on-the-fly optimization and a multi-pass guided optimization. We apply our techniques to NWChem - a 6 million line computational chemistry code written in C/C++/Fortran that uses several runtime libraries such as Global Arrays, ComEx, DMAPP, and MPI. Our technique elides a surprisingly high fraction of barriers (as many as 63%) in production runs. This redundancy elimination translates to application speedups as high as 14% on 2048 cores. Our techniques also provided valuable insight about the application behavior, later used by NWChem developers. Overall, we demonstrate the value of holistic context-sensitive analyses that consider the domain science in conjunction with the associated runtime software stack.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Optimizing I/O Performance of HPC Applications with Autotuning

Parallel Input output is an essential component of modern high-performance computing (HPC). Obtaining good I/O performance for a broad range of applications on diverse HPC platforms is a major challenge, in part, because of complex inter dependencies between I/O middleware and hardware. The parallel file system and I/O middleware layers all offer optimization parameters that can, in theory, result in better I/O performance. Unfortunately, the right combination of parameters is highly dependent on the application, HPC platform, problem size, and concurrency. Scientific application developers do not have the time or expertise to take on the substantial burden of identifying good parameters for each problem configuration. They resort to using system defaults, a choice that frequently results in poor I/O performance. We expect this problem to be compounded on exascale-class machines, which will likely have a deeper software stack with hierarchically arranged hardware resources. We present as a solution to this problem an autotuning system for optimizing I/O performance, I/O performance modeling, I/O tuning, and I/O patterns. We demonstrate the value of this framework across several HPC platforms and applications at scale.

Behzad, Babak↗

DOE OSTI2022

Lifting and Dropping VMs to Dynamically Transition Between Time- and Space-sharing for Large-Scale HPC Systems

As HPC environments increasingly integrate with edge based systems, system architectures will need to handle a broader class of workloads and scheduling requirements. One result of this shift will be the need to simultaneously support bulk-synchronous parallel (BSP) and on-demand service based applications on the same infrastructure. This in turn will require that future resource management approaches utilize both space-shared as well as time-shared resource scheduling strategies. In this work we introduce the concept of "VM-lifting'' (and its inverse "VM-Dropping'') which allows dynamically switching an HPC workload between space-shared and time-shared scheduling regimes. Our work targets co-kernel based HPC system software environments, in which multiple specialized OS kernels execute natively on dedicated physical resource partitions inside a single compute node. With VM-lifting, a native co-kernel can be migrated at runtime to and from locally hosted Virtual Machine Environments due to changing scheduling requirements of the node. This allows an HPC node to be dynamically (re-)configured as either a time-shared Infrastructure-as-a-Service (IaaS) resource or a dedicated space shared resource based on the current workload demands. We have implemented this approach in the context of the Hobbes Exascale System Software stack and have demonstrated that a node can be reconfigured with minimal impact on the running applications.

Gordon, Nick↗

DOE OSTI2025

Lowering and Runtime Support for Fortran’s Multi-Image Parallel Features using LLVM Flang, PRIF, and Caffeine

This paper provides an overview of the multi-image parallel features in Fortran 2023 and their implementation in the LLVM flang compiler and the Caffeine parallel runtime library. The features of interest support a Single-Program, Multiple-Data (SPMD) programming model based on executing multiple “images”, each of which is a program instance. The features also support a Partitioned Global Address Space (PGAS) in the form of “coarray” distributed data structures. The paper discusses the lowering of multi-image features to the Parallel Runtime Interface for Fortran (PRIF) and the implementation of PRIF in the Caffeine parallel runtime library. This paper also provides an early view into the design of a new multi-image dialect of the LLVM Multi-Level Intermediate Representation (MLIR). We describe validation and testing of the resulting software stack, and demonstrate that performance compares favorably to another open-source compiler and runtime library: GNU Compiler Collection (GCC) gfortran and OpenCoarrays, respectively.

Bonachea, Dan↗

DOE OSTI2019

Sierra Center of Excellence: Lessons Learned

The introduction of heterogeneous computing via GPUs from the Sierra architecture represented a significant shift in direction for computational science at Lawrence Livermore National Laboratory (LLNL), and therefore required significant preparation. Over the last five years, the Sierra Center of Excellence (CoE) has brought employees with specific expertise from IBM and NVIDIA together with LLNL in a concentrated effort to prepare applications, system software, and tools for the Sierra supercomputer. This article shares the process we applied for the CoE and documents lessons learned during the collaboration, with the hope that others will be able to learn from both our success and intermediate setbacks. We describe what we have found to work for the management of such a collaboration and best practices for algorithms and source code, system configuration and software stack, tools, and application performance.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

Multi-Purpose, Application-Centric, Scalable I/O Proxy Application

MACSio is a Multi-purpose, Application-Centric, Scalable I/O proxy application. It is designed to support a number of goals with respect to parallel I/O performance testing and benchmarking including the ability to test and compare various I/O libraries and I/O paradigms, to predict scalable performance of real applications and to help identify where improvements in I/O performance can be made within the HPC I/O software stack.

Miller, M.C.↗

DOE OSTI2015

Caliper Context Annotation Library

To understand the performance of parallel programs, developers need to be able to relate performance measurement data with context information, such as the call path / line numbers or iteration numbers where measurements were taken. Caliper provides a generic way to specify and collect multi-dimensional context information across the software stack, and provide ti to third-party measurement tools or write it into a file or database in the form of context streams.

ECP↗

DOE OSTI2017

hpcswtest

The HPC Software stack testing framework

Berg, Paul↗

DOE OSTI2018

lalibe v1.0

This is top-level lattice QCD code developed by employees at LBNL and LLNL. It currently compiles on top of the USQCD Chroma software stack. This code contains high-performance routines that are used in our research, and we intend to release this code publicly to ensure reproducibility and to meet our obligations for complying with the DOE Digital Data rules.

Walker-Loud, Andre↗

DOE OSTI2018

Sonar

Sonar contains software for managing a HPC performance-monitoring software stack. This includes streaming data into a distributed Kafka message queue, storing/querying from a distributed Cassandra database, and front-end tools for secure access, visualization, and analytics of facility-wide performance data.

Gimenez, AlfredoA.↗

DOE OSTI2020

DiHydrogen

DiHydrogen is the second version of the Hydrogen fork of the well-known distributed linear algebra library, Elemental. DiHydrogen is a GPU-accelerated distributed multilinear algebra interface with a particular emphasis on the needs of the scalable distributed deep learning training and inference. DiHydrogen is part of the Livermore Big Artificial Neural Network (LBANN) software stack.

Maruyama, Naoya↗

DOE OSTI2021

Architecture for Quantum Time-dependent Circuits (ArQTIC) v1.0

ArQTiC is a domain-specific full-stack software package built for the dynamic simulations of materials on quantum computers. Its main contributions include providing a software library for high-level programming of such simulations on quantum computers, and providing post-processing capabilities that allow users to more efficiently analyze results from the quantum computer. Paired with the power to optimize and execute quantum circuits, ArQTiC opens the field of dynamic materials simulations on quantum computers to a broader community of scientists from a wider range of domain sciences, paving the way for accelerated progress towards physical quantum supremacy.

Bassman, Lindsay↗

DOE OSTI2022

Spack-Manager

SAND2022-3282 O Spack-Manager is a lightweight wrapper for streamlining the software development and deployment cycle using Spack. Spack-Manager creates a uniform framework for developing, testing, and deploying specific software stacks for projects across multiple platforms while utilizing Spack as much as possible. It organizes Spack extensions, machine configurations, project customizations, and tools for features that are mostly project specific. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Sakievich, Philip↗

DOE OSTI2021

NSU Hack-a-Thon Workshop v.2.0

SAND2021-6722 O This is a fitness tracking mobile application and server created to train college-level students to build full stack software. The front end is written using an open-source framework called Flutter. The API server is written using an open-source python framework called Flask. The app functions as a workout tracker and is intentionally partially functional for the purpose of training students during the live workshops hosted by Sandia's HR recruiting teams. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Carpenter, Logan↗

DOE OSTI2019

pnnl/ARTS

Abstract RunTime System (ARTS) aims to provide different levels of abstraction that trade-off performance and programmability and helps to connect upper layers of the software stack with the different computational substrates.

Manzano Franco, Joseph↗

DOE OSTI2024

WETO Stack [SWR-24-81]

The WETO Stack software is a collection of data, data analysis scripts, and website content produced through the Holistic Modeling Project’s Portfolio Coordination task. The purpose of the software is to characterize the collection of wind energy software projects supported by the U.S. Department of Energy’s Wind Energy Technologies Office. The included data contains characteristics of the actively funded software projects, and the data analysis scripts provide insights into the breadth, depth, and maturity of the collection of software projects. The website content presents the results of the analyses as well as reports from workshops and other events. It can be viewed at https://nrel.github.io/WETOStack/index.html.

Mudafort, Rafael↗