Datacenter Explorer
An Unreal Engine plugin for visualizing and interacting with hierarchical datacenter infrastructure through JSON configurations.
Greenwood, Scott↗
Engineering topics
Bouknight, Sedrick
Publications and source records attributed to Bouknight, Sedrick.
A Digital Twin Framework for Liquid-cooled Supercomputers as Demonstrated at Exascale
We present ExaDigiT, an open-source framework for developing comprehensive digital twins of liquid-cooled supercomputers. It integrates three main modules: (1) a resource allocator and power simulator, (2) a transient thermo-fluidic cooling model, and (3) an augmented reality model of the supercomputer and central energy plant. The framework enables the study of "what-if" scenarios, system optimizations, and virtual prototyping of future systems. Using Frontier as a case study, we demonstrate the framework's capabilities by replaying six months of system telemetry for systematic verification and validation. Such a comprehensive analysis of a liquid-cooled exascale supercomputer is the first of its kind. ExaDigiT elucidates complex transient cooling system dynamics, runs synthetic or real workloads, and predicts energy losses due to rectification and voltage conversion. Throughout our paper, we present lessons learned to benefit HPC practitioners developing similar digital twins. We envision the digital twin will be a key enabler for sustainable, energy-efficient supercomputing.
Brewer, Wes↗
Visualizing an Exascale Data Center Digital Twin: Considerations, Challenges and Opportunities
Digital twins are an excellent tool to model, visualize, and simulate complex systems, to understand and optimize their operation. In this work, we present the technical challenges of real-time visualization of a digital twin of the Frontier supercomputer.We show the initial prototype and current state of the twin and highlight technical design challenges of visualizing such a large High Performance Computing (HPC) system. The goal is to understand the use of augmented reality as a primary way to extract information and collaborate on digital twins of complex systems. This leverages the spatio-temporal aspect of a 3D representation of a digital twin, with the ability to view historical and real-time telemetry, triggering simulations of a system state and viewing the results, which can be augmented via dashboards for details. Finally, we discuss considerations and opportunities for augmented reality of digital twins of large-scale, parallel computers.
Maiterth, Matthias↗
Dynamic Modeling of Power Conversion Stages for an Exascale Supercomputer
In this paper a power conversion and energy consumption model for an exascale supercomputer is investigated. Power consumption, energy loss and efficiency are derived for the 27.2 MW liquid-cooled, centralized, High Performance Computing (HPC) power system, which is supplied directly from the 480 V three-phase mains. Two energy conversion stages are analyzed, measured and modeled. The model is developed in order to be adapted and implemented in a digital twin platform utilizing a Resource Allocator and Power Simulator (RAPS) module. RAPS enables estimation of potential energy savings in the direct AC power supply architecture via both conventional rectifier load sharing (commonly used in HPC systems), as well as smart rectifier load sharing. Moreover, besides the direct AC supply architecture analysis, the full direct DC supply architecture with with 1 kV DC bus were also studied. Comparison of 10 hour time frame operation of the system, with direct 480 V AC voltage supply with conventional and smart load sharing and medium dc voltage supply were done. For the direct AC supply architecture, with conventional and smart load sharing the predicted power loss was approximately 840 kW and 820 kW, respectively and the predicted total system efficiency was 92.87% and 93.05%, respectively. For the direct DC supply architecture with the 1000 V DC supply bus power loss was approximately 340 kW and the predicted total system efficiency was 97.02%.
Wojda, Rafal↗
exadigitUE5
This project provides the AR/VR interface to ORNL's exascale digital twin. The main functionality is implemented using Unreal Engine 5.1 for Desktop or Microsoft Hololens2 based visualization and interation with the system. The digital twin provides data ingestion from telemetry, as well as triggering and interacting with simulations developed for the wider ExaDigiT project at ORNL, as well as for the LUMI system at CSC and other CrayEX Supercomputers. For the overarching project, see ExaDigiT at https://exadigit.github.io, with the code repositories at https://code.ornl.gov/exadigit.
Maiterth, Matthias [Oak Ridge National Laboratory ↗