Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Cluster computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Studying CPU and memory utilization of applications on Fujitsu A64FX and Nvidia Grace Superchip

ARM-based manycore CPU architectures are well-positioned to provide the rising memory throughput requirements of modern data intensive scientific applications in High Performance Computing (HPC). The Fujitsu A64FX CPU platform is based on the ARM v8.2A architecture, and is the processor of the flagship Japanese supercomputer - "Fugaku", which was previously ranked as the #1 supercomputer in the world according to the Top500 list. The Nvidia Grace superchip features 144 Neoverse V2 cores based on the ARMv9 architecture with 4x128b SVE2, providing exceptional computational power. The chip supports up to 480GB of memory, making it ideal for AI, machine learning, and scientific computing workloads. In this paper, we conduct a thorough performance exploration of a variety of parallel bandwidth-sensitive benchmarks and applications compiled with the native Fujitsu compiler on a Fugaku A64FX compute node and ARM (LLVM) Compiler on an NVIDIA Grace superchip compute node, engaging all the computational cores per cluster using OpenMP multithreading (assuming the cores can drive the available bandwidth). Our ultimate goals are to study the resource utilization of scientific applications and benchmarks on A64FX and Grace superchip, considering graph application scenarios ( GAP Benchmark suite) and eleven appli- cation proxies from the Rodinia heterogeneous benchmark suite (considering domains such as Data Mining, Bioinformatics, Fluid Dynamics, Pattern Recognition, etc.). Through exhaustive performance monitoring, we quantify the resource utilization of diverse OpenMP-based HPC applications on both the Fujitsu A64FX and the Nvidia Grace Superchip platforms.

benchmarking, Performance Analysis, High performan↗
DOE OSTI2022

Cosmological inference from an emulator based halo model. II. Joint analysis of galaxy-galaxy weak lensing and galaxy clustering from HSC-Y1 and SDSS

Here, we present high-fidelity cosmology results from a blinded joint analysis of galaxy-galaxy weak lensing (ΔΣ) and projected galaxy clustering (w p ) measured from the Hyper Suprime-Cam Year-1 (HSC-Y1) data and spectroscopic Sloan Digital Sky Survey (SDSS) galaxy catalogs in the redshift range 0.15 < z < 0.7. We define luminosity-limited samples of SDSS galaxies to serve as the tracers of w p in three spectroscopic redshift bins, and as the lens samples for ΔΣ. For the ΔΣ measurements, we select a single sample of 4×10 6 source galaxies over 140 deg 2 from HSC-Y1 with photometric redshifts (photo z) greater than 0.75, enabling a better handle of photo- z errors by comparing the ΔΣ amplitudes for the three lens redshift bins. The deep, high-quality HSC-Y1 data enable significant detections of the ΔΣ signals, with integrated signal-to-noise ratio S/N ~ 15 in the range 3 ≤ R/[h –1 Mpc] ≤ 30 for the three lens samples, despite the small area coverage. For cosmological parameter inference, we use an input galaxy-halo connection model built on the dark emulator package (which uses an ensemble set of high-resolution N-body simulations and enables fast, accurate computation of the clustering observables) with a halo occupation distribution that includes nuisance parameters to marginalize over modeling uncertainties. We model the ΔΣ and wp measurements on scales from R≃3 and 2h –1 Mpc , respectively, up to 30 h –1 Mpc (therefore excluding the baryon acoustic oscillations information) assuming a flat Λ CDM cosmology, marginalizing over about 20 nuisance parameters and demonstrating the robustness of our results to them. With various tests using mock catalogs described in Miyatake et al. [preceding paper, Phys. Rev. D 106, 083519 (2022)], we show that any bias in the clustering amplitude S 8 ≡ σ 8 (Ω m /0.3) 0.5 due to uncertainties in the galaxy-halo connection is less than ~ 50 % of the statistical uncertainty of S 8 , unless the assembly biaseffect is unexpectedly large. Our best-fit models have S 8 = 0.795$_{-0.042}^{+0.049}$ (mode and 68% credible interval) for the flat Λ CDM model; we find tighter constraints on the quantity S 8 (α = 0.17)≡σ 8 (Ω m /0.3) 0.17 = 0.745$_{-0.031}^{+0.039}$.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1978

Predicting the workload of a computer system

A technique based on clustering is described for characterizing the current workload of a computer system and for predicting future workload. The technique was applied to predicting the load for an installation operating two machines processing about 20,000 job steps per month.

Agrawala, A. K.↗
DOE OSTI2026

Concordant Mode Approach (CMA): Vibrational Analysis of New and Upgraded Intermolecular Benchmarks for Noncovalent Bonding

The Concordant Mode Approach (CMA) is a novel method that offers tremendous potential for increasing the system size and the level of theory attainable in quantum chemical computations of molecular vibrational frequencies. To investigate the extension of CMA to intermolecular vibrations, computations with coupled cluster singles and doubles with perturbative triples theory [CCSD(T)] using two augmented correlation-consistent polarized-valence triple-ζ basis sets (aug-cc-pVTZ or h-aug-ccpVTZ) were performed on 17 prototypical loosely bound complexes of hydrogen-bonded, dispersion, and mixed character. These Level A results provide new and upgraded benchmarks for noncovalent bonding and a severe test for CMA vibrational analyses. The Level A target frequencies were recovered remarkably well using second-order Møller−Plesset perturbation theory (MP2) with h-aug-cc-pVTZ for generating the underlying (Level B) normal modes of the CMA scheme. Employing this Level B within the lowest-rung CMA-0A method reproduces the 435 benchmark frequencies with a mean absolute error (MAE) of 0.23 cm −1 and a corresponding standard deviation (σ) of 0.84 cm −1 ; strikingly, the corresponding subset of 106 interfragment frequencies exhibits MAE = 0.34 cm −1 and σ = 0.90 cm −1 . Subsequent application of the higher-rung CMA-2A scheme eliminates all outliers and reduces the overall MAE to a minuscule 0.08 cm −1 with the inclusion of only 3.0% of the off-diagonal couplings not accounted for by CMA-0A. Accordingly, the highly efficient CMA methodology proves to be robust even for vibrations on flat potential energy surfaces.

Aromatic compounds↗
DOE OSTI2024

VIBES: a workflow for annotating and visualizing viral sequences integrated into bacterial genomes

Abstract Bacteriophages are viruses that infect bacteria. Many bacteriophages integrate their genomes into the bacterial chromosome and become prophages. Prophages may substantially burden or benefit host bacteria fitness, acting in some cases as parasites and in others as mutualists. Some prophages have been demonstrated to increase host virulence. The increasing ease of bacterial genome sequencing provides an opportunity to deeply explore prophage prevalence and insertion sites. Here we present VIBES (Viral Integrations in Bacterial genomES), a workflow intended to automate prophage annotation in complete bacterial genome sequences. VIBES provides additional context to prophage annotations by annotating bacterial genes and viral proteins in user-provided bacterial and viral genomes. The VIBES pipeline is implemented as a Nextflow-driven workflow, providing a simple, unified interface for execution on local, cluster and cloud computing environments. For each step of the pipeline, a container including all necessary software dependencies is provided. VIBES produces results in simple tab-separated format and generates intuitive and interactive visualizations for data exploration. Despite VIBES’s primary emphasis on prophage annotation, its generic alignment-based design allows it to be deployed as a general-purpose sequence similarity search manager. We demonstrate the utility of the VIBES prophage annotation workflow by searching for 178 Pf phage genomes across 1072 Pseudomonas spp. genomes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

A Cast of Thousands: How the IDEAS Productivity Project Has Advanced Software Productivity and Sustainability

Computational and data-enabled science and engineering are revolutionizing advances throughout science and society, at all scales of computing. For example, teams in the U.S. Department of Energy’s Exascale Computing Project have been tackling new frontiers in modeling, simulation, and analysis by exploiting unprecedented exascale computing capabilities—building an advanced software ecosystem that supports next-generation applications and addresses disruptive changes in computer architectures. However, concerns are growing about the productivity of the developers of scientific software. Members of the Interoperable Design of Extreme-scale Application Software project serve as catalysts to address these challenges through fostering software communities, incubating and curating methodologies and resources, and disseminating knowledge to advance developer productivity and software sustainability. This article discusses how these synergistic activities are advancing scientific discovery—mitigating technical risks by building a firmer foundation for reproducible, sustainable science at all scales of computing, from laptops to clusters to exascale and beyond.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

panhandle

A project to provide user activity monitoring for High Performance Computing systems and clusters. The goal is to provide effective user activity monitoring with minimal performance impact on the host running this service.

McGee, David [@LANL @USMC @DoD]↗
DOE OSTI2024

Using containers to speed up development, to run integration tests and to teach about distributed systems

GlideinWMS is a workload manager provisioning resources for many experiments including CMS and DUNE. The software is distributed both as native packages and specialized production containers. Following an approach used in other communities like web development we built our workspaces, system-like containers to ease development and testing. Developers can change the source tree or check out a different branch and quickly reconfigure the services to see the effect of their changes. In this paper, we’ll talk about what differentiates workspaces from other containers. We’ll describe our base system composed of three containers. A one-node cluster including a compute element and a batch system. A GlideinWMS Factory controlling pilot jobs. And a scheduler and Frontend, to submit jobs and provision resources. Additional containers can be used for optional components. This system can easily run on a laptop and we’ll share our evaluation of different container runtimes, with an eye for ease of use and performance. Finally, we’ll talk about our experience as developers and with students. The GlideinWMS workspaces are easily integrated with IDEs like VS Code, simplifying debugging and allowing development and testing of the system also when offline. They simplified the training and onboarding of new team members and Summer interns. And they were useful in workshops where students could have first-hand experience with the mechanisms and components that, in production, run millions of jobs.

Mambelli, Marco↗
DOE OSTI2025

Using Containers to Speed Up Development, to Run Integration Tests and to Teach About Distributed Systems

GlideinWMS is a workload manager provisioning resources for many experiments, including CMS and DUNE. The software is distributed both as native packages and specialized production containers. Following an approach used in other communities like web development, we built our workspaces, system-like containers to ease development and testing. Developers can change the source tree or check out a different branch and quickly reconfigure the services to see the effect of their changes. In this paper, we will talk about what differentiates workspaces from other containers. We will describe our base system, composed of three containers: a one-node cluster including a compute element and a batch system, a GlideinWMS Factory controlling pilot jobs, and a scheduler and Frontend to submit jobs and provision resources. Additional containers can be used for optional components. This system can easily run on a laptop, and we will share our evaluation of different container runtimes, with an eye for ease of use and performance. Finally, we will talk about our experience as developers and with students. The GlideinWMS workspaces are easily integrated with IDEs like VS Code, simplifying debugging and allowing development and testing of the system even when offline. They simplified the training and onboarding of new team members and summer interns. And they were useful in workshops where students could have first-hand experience with the mechanisms and components that, in production, run millions of jobs.

Mambelli, Marco [Fermilab] (ORCID:0000000294892681↗
DOE OSTI2025

A new biogeochemical modelling framework (FLaMe-v1.0) for lake methane emissions on the regional scale: development and application to the European domain

This study presents a new physical-biogeochemical modelling framework for simulating lake methane (CH 4 ) emissions at regional scales. The new model, FLaMe-v1.0 (Fluxes of Lake Methane), rests on an innovative, computationally efficient lake clustering approach that enables the simulation of CH 4 emissions across a large number of lakes. Building on the Canadian Small Lake Model (CSLM) that simulates the lake physics, we develop a suite of biogeochemical modules to simulate transient dynamics of organic Carbon (C), Oxygen (O 2 ), and CH 4 . We first test the performance of FLaMe-v1.0 by analyzing physical and biogeochemical processes in two theoretical lakes with characteristics that can be considered representative for many lakes (an oligotrophic, deep lake driven by cold climate versus a eutrophic, shallow lake driven by warm climate). Next, we evaluate the model by comparing simulated and observed timeseries of CH 4 emissions in four well-surveyed lakes. We then apply FLaMe-v1.0 at the European scale to evaluate simulated diffusive and ebullitive lake CH 4 fluxes against in-situ measurements in both boreal and central European regions. Finally, we provide a first assessment of the spatio-temporal variability in CH 4 emissions from European lakes with a surface area comprised between 0.1–1000 km 2 (n= 108 407, total area = 1.33 × 105 km 2 ), indicating a total emission of 0.97 ± 0.23 Tg CH 4 yr −1 , with the uncertainty constrained by combining FLaMe-v1.0 and machine learning techniques. Moreover, 30 % and 70 % of these CH 4 emissions are through diffusive and ebullitive pathways, respectively. Annually averaged CH 4 emission rates per unit lake area during 2010–2016 have a South-to-North decreasing gradient, resulting in a mean over the European domain as 7.39 g CH 4 m −2 yr −1 . Our simulations reveal a strong seasonality (with ice-blocking effects accounted for) in European lake CH 4 emissions, with nearly ten times higher emissions during late summer than during winter. This pronounced seasonal variation highlights the importance of accounting for the sub-annual variability in CH 4 emissions to accurately constrain regional CH 4 budgets. In the future, FLaMe-v1.0 could be embedded into Earth System Models to investigate the feedback between climate warming and global lake CH 4 emissions.

Maisonnier, Manon [Free Univ. of Brussels (Belgium↗
DOE OSTI2025

precipbestats (c0)

Best estimates of precipitation from ARM instruments derived through clustering and other computational techniques.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

precipbetseries (c1)

Best estimates of precipitation from ARM instruments derived through clustering and other computational techniques.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

precipbetseries (c0)

Best estimates of precipitation from ARM instruments derived through clustering and other computational techniques.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1981

An algorithm for spatial heirarchy clustering

A method for utilizing both spectral and spatial redundancy in compacting and preclassifying images is presented. In multispectral satellite images, a high correlation exists between neighboring image points which tend to occupy dense and restricted regions of the feature space. The image is divided into windows of the same size where the clustering is made. The classes obtained in several neighboring windows are clustered, and then again successively clustered until only one region corresponding to the whole image is obtained. By employing this algorithm only a few points are considered in each clustering, thus reducing computational effort. The method is illustrated as applied to LANDSAT images.

Dejesusparada, N.↗
NASA NTRS1984

A unified N-body and statistical treatment of stellar dynamics. I - The hybrid code. II - Applications to globular cluster cores

A unified N-body and statistical treatment of stellar dynamics is developed and applied to the late stages of core collapse and early stages of post collapse evolution in globular clusters. A 'hybrid' computer code is joined to a direct N-body code which is used to calculate exactly the behavior of particles in the inner spatial region, and the combination is used to follow particles statistically in the outer spatial region. A transition zone allows the exchange of particles and energy between the two regions. The main application results include: formation of a hard central binary system, reversal of core collapse and expansion due to the heat input from this binary, ejection of the binary from the core, and recollapse of the core; density profiles that form a one-parameter sequence during the core oscillations; and indications that these oscillations will eventually cease.

Mcmillan, S. L. W.↗
NASA NTRS1995

Analysis of developing laminar flows in circular pipes using a higher-order finite-difference technique

A higher-order finite-difference technique is developed to calculate the developing-flow field of steady incompressible laminar flows in the entrance regions of circular pipes. Navier-Stokes equations governing the motion of such a flow field are solved by using this new finite-difference scheme. This new technique can increase the accuracy of the finite-difference approximation, while also providing the option of using unevenly spaced clustered nodes for computation such that relatively fine grids can be adopted for regions with large velocity gradients. The velocity profile at the entrance of the pipe is assumed to be uniform for the computation. The velocity distribution and the surface pressure drop of the developing flow then are calculated and compared to existing experimental measurements reported in the literature. Computational results obtained are found to be in good agreement with existing experimental correlations and therefore, the reliability of the new technique has been successfully tested.

Gladden, Herbert J.↗
NASA NTRS1994

High Performance Distributed Computing in a Supercomputer Environment: Computational Services and Applications Issues

This tutorial proposes to be a practical guide for the uninitiated to the main topics and themes of high-performance computing (HPC), with particular emphasis to distributed computing. The intent is first to provide some guidance and directions in the rapidly increasing field of scientific computing using both massively parallel and traditional supercomputers. Because of their considerable potential computational power, loosely or tightly coupled clusters of workstations are increasingly considered as a third alternative to both the more conventional supercomputers based on a small number of powerful vector processors, as well as high massively parallel processors. Even though many research issues concerning the effective use of workstation clusters and their integration into a large scale production facility are still unresolved, such clusters are already used for production computing. In this tutorial we will utilize the unique experience made at the NAS facility at NASA Ames Research Center. Over the last five years at NAS massively parallel supercomputers such as the Connection Machines CM-2 and CM-5 from Thinking Machines Corporation and the iPSC/860 (Touchstone Gamma Machine) and Paragon Machines from Intel were used in a production supercomputer center alongside with traditional vector supercomputers such as the Cray Y-MP and C90.

Kramer, Williams T. C.↗