Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “SUMMIT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Full Modeling and parameter compression methods in configuration space for DESI 2024 and beyond

In the contemporary era of high-precision spectroscopic surveys, led by projects like DESI, there is an increasing demand for optimizing the extraction of cosmological information from clustering data. This work conducts a thorough comparison of various methodologies for modeling the full shape of the two-point statistics in configuration space. We investigate the performance of both direct fits (Full Modeling) and the parameter compression approaches (ShapeFit and Standard). We utilize the ABACUS-SUMMIT simulations, tailored to exceed DESI's precision requirements. Particularly, we fit the two-point statistics of three distinct tracers (LRG, ELG, and QSO), by employing a Gaussian Streaming Model in tandem with Convolution Lagrangian Perturbation Theory and Effective Field Theory. We explore methodological setup variations, including the range of scales, the set of galaxy bias parameters, the inclusion of the hexadecapole, as well as model extensions encompassing varying ns and allowing for w 0 w a CDM dark energy model. Throughout these varied explorations, while precision levels fluctuate and certain configurations exhibit tighter parameter constraints, our pipeline consistently recovers the parameter values of the mocks within 1σ in all cases for a 1-year DESI volume. Additionally, we compare the performance of configuration space analysis with its Fourier space counterpart using three models: PyBird, FOLPS and velocileptors, presented in companion papers. We find good agreement with the results from all these models.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Organizing Large Data Sets for Efficient Analyses on HPC Systems

Upcoming exascale applications could introduce significant data management challenges due to their large sizes, dynamic work distribution, and involvement of accelerators such as graphical processing units, GPUs. In this work, we explore the performance of reading and writing operations involving one such scientific application on two different supercomputers. Our tests showed that the Adaptable Input and Output System, ADIOS, was able to achieve speeds over 1TB/s, a significant fraction of the peak I/O performance on Summit. We also demonstrated the querying functionality in ADIOS could effectively support common selective data analysis operations, such as conditional histograms. In tests, this query mechanism was able to reduce the execution time by a factor of five. More importantly, ADIOS data management framework allows us to achieve these performance improvements with only a minimal amount of coding effort.

Gu, Junmin↗
DOE OSTI2022

The MURAVES muon telescope: a low power consumption muon tracker for muon radiography applications

Muon Radiography or muography is based on the measurement of the absorption or scattering of cosmic muons, as they pass through the interior of large scale bodies, In particular, absorption muography has been applied to investigate the presence of hidden cavities inside the pyramids or underground, as well as the interior of volcanoes’ edifices. The MURAVES project has the challenging aim of investigating the density distribution inside the summit of Mt. Vesuvius. The information, together with that coming from gravimetric measurements, is useful as input to models, to predict how an eruption may develop. The MURAVES apparatus is a robust and low power consumption muon telescope consisting of an array of three identical and independent muon trackers, which provide in a modular way a total sensitive area of three square meters. Each tracker consists of four doublets of planes of plastic scintillator bars with orthogonal orientation, optically coupled to Silicon photomultipliers for the readout of the signal. The muon telescope has been installed on the slope of the volcano and has collected a first set of data, which are being analyzed.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Supporting multiple hardware architectures at CMS: the integration and validation of POWER9

Computing resources in the Worldwide LHC Computing Grid (WLCG) have been based entirely on the x86 architecture for more than two decades. In the near future, however, heterogeneous non-x86 resources, such as ARM, POWER and Risc-V, will become a substantial fraction of the resources that will be provided to the LHC experiments, due to their presence in existing and planned world-class HPC installations. The CMS experiment, one of the four large detectors at the LHC, has started to prepare for this situation, with the CMS software stack (CMSSW) already compiled for multiple architectures. In order to allow for a production use, the tools for workload management and job distribution need to be extended to be able to exploit heterogeneous architectures. Profiting from the opportunity to exploit the first sizable IBM Power9 allocation available on Marconi100 HPC system at CINECA, CMS developed all the needed modifications to the CMS workload management system. After a successful proof of concept, a full physics validation has been performed in order to bring the system in production. The experiences are of very high value, when it comes to commissioning of the similar (even larger) Summit HPC system at Oak Ridge, where CMS is also expecting a resource allocation. Moreover the compute power of those systems is being provided also via GPUs and this represents an extremely valuable opportunity to exploit the offloading capability already implemented in CMSSW. The status of the current integration including the exploitation of the GPUs, the results of the validation as well as the future plans will be shown and discussed.

Boccali, Tommaso [INFN, Pisa]↗
DOE OSTI2020

Muon radiography applied to volcanoes imaging: the MURAVES experiment at Mt. Vesuvius

Muon radiography is a technique based on the measurement of absorption profiles of muons as they pass through matter. This measurement allows to obtain an image of the inner structure of large volume objects and is suitable to be applied in several fields, such as volcanology, archaeology and civil engeneering. One of the main applications concerns the study of volcanic structures; indeed it is possible to use this technique to measure the mass distribution inside the edifice of a volcano providing useful information to better understand the possible eruption mechanisms. The MURAVES (MUon RAdiography of VESuvius) project aims to the study of the summital cone of Mt. Vesuvius near Naples in Italy, one of the most dangerous active volcanoes in the world. The MURAVES apparatus is a modular, robust muon hodoscope system with a low power consumption, optimized to be used in inhospitable environments like the surroundings of volcanoes. The complete detection system is an array of identical tracking modules, each with an area of 1 m2, based on the use of plasic scintillators. The technologies, the status and the data analysis strategy of the experiment will be presented in this paper.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Establishing the quantum supremacy frontier with a 281 Pflop/s simulation

Noisy intermediate-scale quantum (NISQ) computers are entering an era in which they can perform computational tasks beyond the capabilities of the most powerful classical computers, thereby achieving 'quantum supremacy', a major milestone in quantum computing. NISQ supremacy requires comparison with a state-of-the-art classical simulator. In this paper, we report HPC simulations of hard random quantum circuits (RQC), which have been recently used as a benchmark for the first experimental demonstration of quantum supremacy, sustaining an average performance of 281 Pflop/s (true single precision) on Summit, currently the fastest supercomputer in the world. These simulations were carried out using qFlex, a tensor-network-based classical high-performance simulator of RQCs. Our results show an advantage of many orders of magnitude in energy consumption of NISQ devices over classical supercomputers. In addition, we propose a standard benchmark for NISQ computers based on qFlex.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Scaling neural simulations in STACS

Abstract As modern neuroscience tools acquire more details about the brain, the need to move towards biological-scale neural simulations continues to grow. However, effective simulations at scale remain a challenge. Beyond just the tooling required to enable parallel execution, there is also the unique structure of the synaptic interconnectivity, which is globally sparse but has relatively high connection density and non-local interactions per neuron. There are also various practicalities to consider in high performance computing applications, such as the need for serializing neural networks to support potentially long-running simulations that require checkpoint-restart. Although acceleration on neuromorphic hardware is also a possibility, development in this space can be difficult as hardware support tends to vary between platforms and software support for larger scale models also tends to be limited. In this paper, we focus our attention on Simulation Tool for Asynchronous Cortical Streams (STACS), a spiking neural network simulator that leverages the Charm++ parallel programming framework, with the goal of supporting biological-scale simulations as well as interoperability between platforms. Central to these goals is the implementation of scalable data structures suitable for efficiently distributing a network across parallel partitions. Here, we discuss a straightforward extension of a parallel data format with a history of use in graph partitioners, which also serves as a portable intermediate representation for different neuromorphic backends. We perform scaling studies on the Summit supercomputer, examining the capabilities of STACS in terms of network build and storage, partitioning, and execution. We highlight how a suitably partitioned, spatially dependent synaptic structure introduces a communication workload well-suited to the multicast communication supported by Charm++. We evaluate the strong and weak scaling behavior for networks on the order of millions of neurons and billions of synapses, and show that STACS achieves competitive levels of parallel efficiency.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

High-quality genome of the basidiomycete yeast Dioszegia hungarica PDD-24b-2 isolated from cloud water

The genome of the basidiomycete yeast Dioszegia hungarica strain PDD-24b-2 isolated from cloud water at the summit of puy de $D\hat{o}me$ (France) was sequenced using a hybrid PacBio and Illumina sequencing strategy. The obtained assembled genome of 20.98 Mb and a GC content of 57% is structured in 16 large-scale contigs ranging from 90 kb to 5.56Mb, and another 27.2 kb contig representing the complete circular mitochondrial genome. In total, 8,234 proteins were predicted from the genome sequence. The mitochondrial genome shows 16.2% cgu codon usage for arginine but has no canonical cognate tRNA to translate this codon. Detected transposable element (TE)-related sequences account for about 0.63% of the assembled genome. A dataset of 2,068 hand-picked public environmental metagenomes, representing over 20 Tbp of raw reads, was probed for D. hungarica related ITS sequences, and revealed worldwide distribution of this species, particularly in aerial habitats. Growth experiments suggested a psychrophilic phenotype and the ability to disperse by producing ballistospores. The high-quality assembled genome obtained for this D. hungarica strain will help investigate the behavior and ecological functions of this species in the environment.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Global adjoint tomography—model GLAD-M25

SUMMARY Building on global adjoint tomography model GLAD-M15, we present transversely isotropic global model GLAD-M25, which is the result of 10 quasi-Newton tomographic iterations with an earthquake database consisting of 1480 events in the magnitude range 5.5 ≤ Mw ≤ 7.2, an almost sixfold increase over the first-generation model. We calculated fully 3-D synthetic seismograms with a shortest period of 17 s based on a GPU-accelerated spectral-element wave propagation solver which accommodates effects due to 3-D anelastic crust and mantle structure, topography and bathymetry, the ocean load, ellipticity, rotation and self-gravitation. We used an adjoint-state method to calculate Fréchet derivatives in 3-D anelastic Earth models facilitated by a parsimonious storage algorithm. The simulations were performed on the Cray XK7 ‘Titan’ and the IBM Power 9 ‘Summit’ at the Oak Ridge Leadership Computing Facility. We quantitatively evaluated GLAD-M25 by assessing misfit reductions and traveltime anomaly histograms in 12 measurement categories. We performed similar assessments for a held-out data set consisting of 360 earthquakes, with results comparable to the actual inversion. We highlight the new model for a variety of plumes and subduction zones.

58 GEOSCIENCES↗
DOE OSTI2021

The ABACUS cosmological N -body code

We present abacus, a fast and accurate cosmological N-body code based on a new method for calculating the gravitational potential from a static multipole mesh. The method analytically separates the near- and far-field forces, reducing the former to direct 1/r 2 summation and the latter to a discrete convolution over multipoles. The method achieves 70 million particle updates per second per node of the Summit supercomputer, while maintaining a median fractional force error of 10 -5 . We express the simulation time-step as an event-driven ‘pipeline’, incorporating asynchronous events such as completion of co-processor work, input/output, and network communication. abacus has been used to produce the largest suite of N-body simulations to date, the abacussummit suite of 60 trillion particles, incorporating on-the-fly halo finding. abacus enables the production of mock catalogues of the volume and resolution required by the coming generation of cosmological surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

ABACUSSUMMIT : a massive set of high-accuracy, high-resolution N -body simulations

We present the public data release of the AbacusSummit cosmological N-body simulation suite, produced with the Abacus N-body code on the Summit supercomputer of the Oak Ridge Leadership Computing Facility. Abacus achieves $\mathcal {O}(10^{-5})$ median fractional force error at superlative speeds, calculating 70M particle updates per second per node at early times, and 45M particle updates per second per node at late times. The simulation suite totals roughly 60 trillion particles, the core of which is a set of 139 simulations with particle mass $2\times 10^{9}\, h^{-1}\, \mathrm{M}_\odot$ in box size $2\, h^{-1}\, \mathrm{Gpc}$. The suite spans 97 cosmological models, including Planck 2018, previous flagship simulation cosmologies, and a linear derivative and cosmic emulator grid. A subsuite of 1883 boxes of size $500\, h^{-1}\, \mathrm{Mpc}$ is available for covariance estimation. AbacusSummit data products span 33 epochs from z = 8 to 0.1 and include light cones, full particle snapshots, halo catalogues, and particle subsets sampled consistently across redshift. AbacusSummit is the largest high-accuracy cosmological N-body data set produced to date.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

The short ionizing photon mean free path at z = 6 in Cosmic Dawn III, a new fully coupled radiation-hydrodynamical simulation of the Epoch of Reionization

ABSTRACT Recent determinations of the mean free path of ionizing photons (mfp) in the intergalactic medium (IGM) at z = 6 are lower than many theoretical predictions. In order to gain insight, we investigate the evolution of the mfp in our new massive fully coupled radiation-hydrodynamics cosmological simulation of reionization: Cosmic Dawn III (CoDa III). CoDa III’s scale ($\rm 94^3 \, cMpc^3$) and resolution ($\rm 8192^3$ grid) make it particularly suitable to study the IGM during reionization. The simulation was performed with ramses-cudaton on Summit, and used 13 1072 processors coupled to 24 576 GPUs, making it the largest reionization simulation, and largest ever ramses simulation. A superior agreement with global constraints on reionization is obtained in CoDa III over Cosmic Dawn II (CoDa II), especially for the evolution of the neutral hydrogen fraction and the cosmic photoionization rate, thanks to an improved calibration, later end of reionization (z = 5.6), and higher spatial resolution. Analysing the mfp, we find that CoDa III reproduces the most recent observations very well, from z = 6 to z = 4.6. We show that the distribution of the mfp in CoDa III is bimodal, with short (neutral) and long (ionized) mfp modes, due to the patchiness of reionization and the coexistence of neutral versus ionized regions during reionization. The neutral mode peaks at sub-kpc to kpc scales of mfp, while the ionized mode peak evolves from 0.1 Mpc h−1 at z = 7 to ∼10 Mpc h−1 at z = 5.2. Computing the mfp as the average of the ionized mode provides the best match to the recent observational determinations. The distribution reduces to a single neutral (ionized) mode at z > 13 (z < 5).

Lewis, Joseph S. W. (ORCID:0000000179178474)↗
DOE OSTI2020

MARBLE: A Multi-GPU Aware Job Scheduler for Deep Learning on HPC Systems

Deep learning (DL) has become a key tool for solving complex scientific problems. However, managing the multi-dimensional large-scale data associated with DL, especially atop extant multiple graphics processing units (GPUs) in modern supercomputers poses significant challenges. Moreover, the latest high-performance computing (HPC) architectures bring different performance trends in training throughput compared to the existing studies. Existing DL optimizations such as larger batch size and GPU locality-aware scheduling have little effect on improving DL training throughput performance due to fast CPU-to-GPU connections. Additionally, DL training on multiple GPUs scales sublinearly. Thus, simply adding more GPUs to a system is ineffective. To this end, we design MARBLE, a first-of-its-kind job scheduler, which considers the non-linear scalability of GPUs at the intra-node level to schedule an appropriate number of GPUs per node for a job. By sharing the GPU resources on a node with multiple DL jobs, MARBLE avoids low GPU utilization in current multi-GPU DL training on HPC systems. Our comprehensive evaluation in the Summit supercomputer shows that MARBLE is able to improve DL training performance by up to 48.3% compared to the popular Platform Load Sharing Facility (LSF) scheduler. Compared to the state-of-the-art of DL scheduler, Optimus, MARBLE reduces the job completion time by up to 47%.

Han, Jingoo↗
DOE OSTI2021

Battle of the Defaults: Extracting Performance Characteristics of HDF5 under Production Load

Popular parallel I/O libraries, such as HDF5, provide tuning parameters to obtain superior performance. However, the selection of effective parameters on production systems is complex due to the interdependence of I/O software and file system layers. Hence, application developers typically use the default parameters and often experience poor I/O performance. This work conducts a benchmarking-based analysis on the HDF5 behaviors with a wide variety of I/O patterns to extract performance characteristics under the production workload. To make the analysis well controlled, we exercise I/O benchmarks on POSIX-IO, MPI-IO, and HDF5 using the same I/O patterns and in the same jobs. To address high performance variability in production environments, we repeat the benchmarks across I/O patterns, storage devices, and time intervals. Based on the results, we identified consistent HDF5 behaviors that appropriate configurations and operations on dataset layout and file-metadata placement can improve performance significantly. We apply our findings and evaluate the tuned I/O library on two supercomputers: Summit and Cori. The results show that our tuned parameters can achieve more than 10× I/O performance speedup than that with default parameters on both systems, suggesting the effectiveness, stability, and generality of our solution.

Xie, Bing↗
DOE OSTI2021

The Challenge of Disproportionate Importance of Temporal Features in Predicting HPC Power Consumption

In this work, we demonstrate the challenges in predicting HPC cluster power consumption in the face of significant temporal skew in power consumption behavioral patterns. Predicting large power swings that extend several megawatts has significant operational value for HPC centers, however, prediction is challenging due to the relative rarity of such events and also due to the abrupt or disjoint deviation from the average power consumption levels. To study the impact of this challenge, we have trained a recurrent neural network (RNN) as a reasonably sophisticated model to predict power consumption of the one-year worth of node power consumption data from the Summit supercomputer located in the Oak Ridge Leadership Computing Facility. By studying the prediction results, we have found that although simple usage of RNN models can provide good results on average power consumption levels, it would fail at predicting the power swings that have more operational value. With such results, we discuss potential next steps in addressing such issues aiming towards a robust usage of power prediction techniques in HPC operations.

Li, Chengcheng↗
DOE OSTI2022

Hybrid Analysis of Fusion Data for Online Understanding of Complex Science on Extreme Scale Computers

The current practice for fusion scientists running first principle simulations on high performance computing plat-forms is to either run their simulations and output their data for post-hoc analysis, or to place in situ analytics into their code. In this paper we examine a complex workflow using XGC fusions simulation run on the Oak Ridge Leadership Computing Facility's supercomputer Summit, which also involve three anal-yses as part of the results necessary for scientific discovery. We discuss the challenges faced when implementing these algorithms and present an original hybrid staging technique to help enable the physicists to make discoveries during the execution of the simulation. By creating this infrastructure, we can examine complicated physics results, which may not have been possible without the infrastructure. For example, our work enables the online visualization of turbulent homoclinic tangle around the magnetic X-point, breaking the last confinement surface. This visualization could help fusion scientists to better understand and improve the turbulence spread of plasma exhaust heat, which is crucial toward realizing plasmas beyond the currently accessible physics regimes of present-day tokamak reactors. The physics of turbulent homoclinic tangle will be reported in a future physics publication, by utilizing the original online analysis/visualization framework presented in this paper.

Suchyta, Eric↗
DOE OSTI2022

Hvac: Removing I/O Bottleneck for Large-Scale Deep Learning Applications

Scientific communities are increasingly adopting deep learning (DL) models in their applications to accelerate scientific discovery processes. However, with rapid growth in the computing capabilities of HPC supercomputers, large-scale DL applications have to spend a significant portion of training time performing I/O to a parallel storage system. Previous research works have investigated optimization techniques such as prefetching and caching. Unfortunately, there exist non-trivial challenges to adopting the existing solutions on HPC supercomputers for large-scale DL training applications, which include non-performance and/or failures at extreme scale, lack of portability and generality in design, complex deployment methodology, and being limited to a specific application or dataset. To address these challenges, we propose High-Velocity AI Cache (HVAC), a distributed read-cache layer that targets and fully exploits the node-local storage or near node-local storage technology. HVAC seamlessly accelerates read I/O by aggregating node-local or near node-local storage, avoiding metadata lookups and file locking while preserving portability in the application code. We deploy and evaluate HVAC on 1,024 nodes (with over 6000 NVIDIA V100 GPUS) of the Summit supercomputer. In particular, we evaluate the scalability, efficiency, accuracy, and load distribution of HVAC compared to GPFS and XFS-on-NVMe. With four different DL applications, we observe an average 25 % performance improvement atop GPFS and 9% drop against XFS-on-NVMe, which scale linearly and are considered the performance upper bound. We envision HVAC as an important caching library for upcoming HPC supercomputers such as Frontier.

Khan, Awais↗
DOE OSTI2022

Running Ensemble Workflows at Extreme Scale: Lessons Learned and Path Forward

The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.

Mehta, Kshitij↗