Ionkov, Latchesar | Engineering Papers

DOE OSTI2024

The ECP SICM project: Managing complex memory hierarchies for exascale applications

The Exascale Computing Project (ECP)’s Simplified Interface to Complex Memories (SICM) effort focuses on developing universal interfaces for discovering, managing, and sharing data across complex memory hierarchies. These facilitate the exploitation of emerging memory technologies and support precise control over their various trade-offs such as high-bandwidth versus low-latency, persistent versus ephemeral, high-capacity versus low-capacity, and near-CPU versus near-GPU. SICM comprises three interrelated components: a low-level interface, a high-level interface, and a persistent-heap interface. The low-level SICM interface is intended for system and run-time developers as well as expert application developers who prefer full control of the memory objects used within their application. The high-level SICM interface builds upon the low-level interface, employing application-level profiling and analysis to optimize data management for complex memory hierarchies. The persistent-heap interface provides applications with a persistent memory allocator that can allocate custom C++ data structures in both block-storage and byte-addressable persistent memories.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

UNITY: Unified Memory and File Space

This paper describes the vision for UNITY, a new high-performance computing focused data storage abstraction that places the entire memory hierarchy, including both traditionally separated memoryand file-based data storage, into one storage continuum. Through the use of a novel API and a set of services centered around a smart runtime system, UNITY is able to provide a number of valuable and interesting benefits. The unified storage space provides a scalable and resilient data environment that dynamically manages the mapping of data onto available resources based on multiple factors, including desired persistence and energy budget considerations. By eliminating the need for high-performance computing domain scientists to develop architecture-dependent optimizations for rapidly evolving data storage technologies, UNITY addresses both ease-of-use and performance.

Otstott, Douglas↗

DOE OSTI2017

Simplified Interface to Complex Memory Hierarchies 1.x

Memory systems are expected to get evermore complicated in the coming years, and it isn't clear exactly what form that complexity will take. On the software side, a simple, flexible way of identifying and working with memory pools is needed. Additionally, most developers seek code portability and do not want to learn the intricacies of complex memory. Hence, we believe that a library for interacting with complex memory systems should expose two kinds of abstraction: First, a low-level, mechanism-based interface designed for the runtime or advanced user that wants complete control, with its focus on simplified representation but with all decisions left to the caller. Second, a high-level, policy-based interface designed for ease of use for the application developer, in which we aim for best-practice decisions based on application intent. We have developed such a library, called SICM: Simplified Interface to Complex Memory.

Lang, Michael↗

DOE OSTI2017

Simplified Interface to Complex Memory Hierarchies 1.x

Memory systems are expected to get evermore complicated in the coming years, and it isn't clear exactly what form that complexity will take. On the software side, a simple, flexible way of identifying and working with memory pools is needed. Additionally, most developers seek code portability and do not want to learn the intricacies of complex memory. Hence, we believe that a library for interacting with complex memory systems should expose two kinds of abstraction: First, a low-level, mechanism-based interface designed for the runtime or advanced user that wants complete control, with its focus on simplified representation but with all decisions left to the caller. Second, a high-level, policy-based interface designed for ease of use for the application developer, in which we aim for best-practice decisions based on application intent. We have developed such a library, called SICM: Simplified Interface to Complex Memory.

Lang, Michael↗

DOE OSTI2017

UNITY: Unified Memory and File Space. In: ROSS '17 Proceedings of the 7th International Workshop on Runtime and Operating Systems for Supercomputers ROSS 2017, Article No. 6

This paper describes the vision for UNITY, a new high-performance computing focused data storage abstraction that places the entire memory hierarchy, including both traditionally separated memory-and file-based data storage, into one storage continuum. Through the use of a novel API and a set of services centered around a smart runtime system, UNITY is able to provide a number of valuable and interesting benefits. The unified storage space provides a scalable and resilient data environment that dynamically manages the mapping of data onto available resources based on multiple factors, including desired persistence and energy budget considerations. By eliminating the need for high-performance computing domain scientists to develop architecture-dependent optimizations for rapidly evolving data storage technologies, UNITY addresses both ease-of-use and performance.

Jones, Terry↗