SEARCH · Engineering Papers

Results for “distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Parallel quantum computing simulations via quantum accelerator platform virtualization

Quantum circuit execution is a central task in quantum computation. Due to inherent quantum-mechanical constraints, quantum computing workflows often involve a considerable number of independent measurements over a large set of slightly different quantum circuits. Here we discuss a simple model for parallelizing such quantum circuit executions that is based on introducing a large array of virtual quantum processing units (mapped to HPC nodes in our case) as a parallel quantum computing platform. Implemented within the XACC framework, the model can readily take advantage of its backend-agnostic features, enabling parallel quantum computing/simulation over any target backend supported by XACC. We illustrate the performance of this approach by demonstrating strong scaling in two pertinent domain science problems, namely in computing the gradients for the multi-contracted variational quantum eigensolver and in data-driven quantum circuit learning, where we vary the number of qubits and the number of circuit layers. Here, the latter simulation leverages the cuQuantum library to run efficiently on GPU-accelerated HPC platforms.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Reaching new peaks for the future of the CMS HTCondor Global Pool

The CMS experiment at CERN employs a distributed computing infrastructure to satisfy its data processing and simulation needs. The CMS Submission Infrastructure team manages a dynamic HTCondor pool, aggregating mainly Grid clusters worldwide, but also HPC, Cloud and opportunistic resources. This CMS Global Pool, which currently involves over 70 computing sites worldwide and peaks at 350k CPU cores, is employed to successfully manage the simultaneous execution of up to 150k tasks. While the present infrastructure is sufficient to harness the current computing power scales, CMS latest estimates predict a noticeable expansion in the amount of CPU that will be required in order to cope with the massive data increase of the High-Luminosity LHC (HL-LHC) era, planned to start in 2027. This contribution presents the latest results of the CMS Submission Infrastructure team in exploring and expanding the scalability reach of our Global Pool, in order to preventively detect and overcome any barriers in relation to the HL-LHC goals, while maintaining high effciency in our workload scheduling and resource utilization.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

ATLAS Data Analysis using a Parallel Workflow on Distributed Cloud-based Services with GPUs

A new type of parallel workflow is developed for the ATLAS experiment at the Large Hadron Collider, that makes use of distributed computing combined with a cloud-based infrastructure. This has been developed for a specific type of analysis using ATLAS data, one popularly referred to as Simulation-Based Inference (SBI). The JAX library is used for the parts of the workflow to compute gradients as well as accelerate program execution using just-in-time compilation, which becomes essential in a full SBI analysis and can also offer significant speed-ups in more traditional types of analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

DIRAC current, upcoming and planned capabilities and technologies

DIRAC is the interware for building and operating large scale distributed computing systems. It is adopted by multiple collaborations from various scientific domains for implementing their computing models. DIRAC provides a framework and a rich set of ready-to-use services for Workload, Data and Production Management tasks of small, medium and large scientific communities having different computing requirements. The base functionality can be easily extended by custom components supporting community specific workflows. DIRAC is at the same time an aging project, and a new DiracX project is taking shape for replacing DIRAC in the long term. This contribution will highlight DIRAC’s current, upcoming and planned capabilities and technologies, and how the transition to DiracX will take place. Examples include, but are not limited to, adoption of security tokens and interactions with Identity Provider services, integration of Clouds and High Performance Computers, interface with Rucio, improved monitoring and deployment procedures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

The ATLAS Workflow Management System Evolution in the LHC Run3 and towards the High-Luminosity LHC era

The ATLAS experiment has 18+ years of experience using workload management systems to deploy and develop workflows to process and to simulate data on the distributed computing infrastructure. Simulation, processing and analysis of LHC experiment data require the coordinated work of heterogeneous computing resources. In particular, the ATLAS experiment utilizes the resources of 250 computing centers worldwide, the power of supercomputing centres, and national, academic and commercial cloud computing resources. In this contribution, we present new techniques for cost-effectively improving efficiency introduced in workflow management system software. The evolution from a mesh framework to new types of computing facilities such as cloud and HPCs is described, as well as new types of production and analysis workflows.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

ATLAS HL-LHC Demonstrators with Data Carousel: Dataon-Demand and Tape Smart Writing

The High Luminosity upgrade to the LHC (HL-LHC) is expected to deliver scientific data at the multi-exabyte scale. To tackle this unprecedented data storage challenge, the ATLAS experiment initiated the Data Carousel project in 2018. Data Carousel is a tape-driven workflow in which bulk production campaigns with input data resident on tape are executed by staging and promptly processing a sliding window to disk buffer such that only a small fraction of inputs are pinned on disk at any one time. Put in ATLAS production before Run3, Data Carousel continues to be our focus for seeking new opportunities in disk space savings, and enhancing tape usage throughout the ATLAS Distributed Computing (ADC) environment. These efforts are highlighted by two recent ATLAS HL-LHC demonstrator projects: data-on-demand and tape smart writing. In this paper, we will discuss the recent studies and outcomes from these projects. The research was conducted together with site experts at CERN and Tier-1 centers.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2020

Gauges, loops, and polynomials for partition functions of graphical models

Graphical models represent multivariate and generally not normalized probability distributions. Computing the normalization factor, called the partition function, is the main inference challenge relevant to multiple statistical and optimization applications. The problem is #P-hard that is of an exponential complexity with respect to the number of variables. Here, aimed at approximating the partition function, we consider multi-graph models where binary variables and multivariable factors are associated with edges and nodes, respectively, of an undirected multi-graph. We suggest a new methodology for analysis and computations that combines the Gauge function technique from Chertkov and Chernyak with the technique developed in Anari and Oveis Gharan 2017 arXiv:1702.02937; Gurvits 2011 arXiv:1106.2844; Straszak and Vishnoi 2017 55th Annual Allerton Conf. on Communication, Control, and Computing, based on the recent progress in the field of real stable polynomials. We show that the Gauge function, representing a single-out term in a finite sum expression for the partition function which achieves extremum at the so-called belief-propagation gauge, has a natural polynomial representation in terms of gauges/variables associated with edges of the multi-graph. Moreover, Gauge function can be used to recover the partition function through a sequence of transformations allowing appealing algebraic and graphical interpretations. Algebraically, one step in the sequence consists of the application of a differential operator over gauges associated with an edge. Graphically, the sequence is interpreted as a repetitive elimination/contraction of edges resulting in multi-graph models on decreasing in size (number of edges) graphs with the same partition function as in the original multi-graph model. Even though the complexity of computing factors in the sequence of the derived multi-graph models and respective Gauge functions grow exponentially with the number of eliminated edges, polynomials associated with the new factors remain bi-stable if the original factors have this property. Moreover, we show that BP estimations in the sequence do not decrease, each low-bounding the partition function.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

A catalogue of Locus Algorithm pointings for optimal differential photometry for 23 779 quasars

ABSTRACT This paper presents a catalogue of optimized pointings for differential photometry of 23 779 quasars extracted from the Sloan Digital Sky Survey (SDSS) Catalogue and a Score for each indicating the quality of the Field of View (FoV) associated with that pointing. Observation of millimagnitude variability on a time-scale of minutes typically requires differential observations with reference to an ensemble of reference stars. For optimal performance, these reference stars should have similar colour and magnitude to the target quasar. In addition, the greatest quantity and quality of suitable reference stars may be found by using a telescope pointing which offsets the target object from the centre of the FoV. By comparing each quasar with the stars which appear close to it on the sky in the SDSS Catalogue, an optimum pointing can be calculated, and a figure of merit, referred to as the ‘Score’ is calculated for that pointing. Highly flexible software has been developed to enable this process to be automated and implemented in a distributed computing paradigm, which enables the creation of catalogues of pointings given a set of input targets. Applying this technique to a sample of 40 000 targets from the fourth SDSS quasar catalogue resulted in the production of pointings and Scores for 23 779 quasars based on their magnitudes in the SDSS r-band. This catalogue is a useful resource for observers planning differential photometry studies and surveys of quasars to select those which have many suitable celestial neighbours for differential photometry.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Microscopic optical potentials for medium-mass isotopes derived at the first order of Watson multiple-scattering theory

We perform a first-principles calculation of optical potentials for nucleon elastic scattering off medium-mass isotopes. Fully based on a saturating chiral Hamiltonian, the optical potentials are derived by folding nuclear density distributions computed with self-consistent Green's function theory with a nucleon-nucleon t matrix computed with a consistent chiral interaction. The dependence on the folding interaction as well as the convergence of the target densities are investigated. Numerical results are presented and discussed for differential cross sections and analyzing powers, with focus on elastic proton scattering off calcium and nickel isotopes. Our optical potentials generally show a remarkable agreement with the available experimental data for laboratory energies in the range 65–200 MeV. We study the evolution of the scattering observables with increasing proton-neutron asymmetry by computing theoretical predictions of the cross section and analyzing power over the calcium and nickel isotopic chains. Published by the American Physical Society 2024

Physics↗

DOE OSTI2024

Comparing pion production in transport simulations of heavy-ion collisions at 270⁢𝐴 MeV under controlled conditions

Within the TMEP, we present a detailed study of the performance of different transport models in Sn+Sn collisions at 270A MeV, and put particular emphasis on the production of pions and Delta resonances, which have been used as probes of the nuclear symmetry energy. We prescribe a common and rather simple physics model, and follow in detail the results of 4 BUU models and 6 QMD models. The nucleonic evolution of the collision and the nucleonic observables in these codes do not completely converge, but the differences among the codes can be understood as being due to several reasons: the basic differences between BUU and QMD models in the representation of the phase-space distributions, computational differences in the mean-field evaluation, and differences in the adopted strategies for the Pauli blocking in the collision integrals. For pionic observables, we find that a higher maximum density leads to an enhanced pion yield and a reduced pi- pi+ yield ratio, while a more effective Pauli blocking generally leads to a slightly suppressed pion yield and an enhanced pi- pi+ yield ratio. We specifically investigate the effect of the Coulomb force, and find that it increases the total pi- pi+ yield ratio but reduces the ratio at high pion energies, although differences in its implementations do not have a dominating role in the differences among the codes. Taking into account only the results of codes that strictly follow the homework specifications, we find a convergence of the codes in the final charged pion yield ratio to a 1 sigma deviation of about 5%. Furthermore, the uncertainty is expected to be reduced to about 1.6% if the same or similar strategies and ingredients, i.e., an improved Pauli blocking and calculation of the non-linear term in the mean-field potential, are similarly used in all codes.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques

Large volumes of data generated by scientific simulations, genome sequencing, and other applications need to be moved among clusters for data collection/analysis. Data compression techniques have effectively reduced data storage and transfer costs. However, users' requirements on interactively controlling both data quality and compression ratios are non-trivial to fulfill. Here, we propose a novel Compression-as-a-Service (CaaS) platform called Ocelot with four important contributions: (1) It offers real-time visualization, interactive compression, and transfer of scientific datasets. (2) It incorporates new strategies for compressing diverse types of datasets more effectively than traditional methods. (3) It provides an effective method for estimating the compression ratio and execution time of compression tasks. (4) Experiments on multiple real-world datasets on geographically distributed computers show that Ocelot can significantly improve data transfer efficiency with a performance gain of more than 10x in computing clusters with relatively slow networks.

compression as a service (CaaS)↗

DOE OSTI2023

Artificial Intelligence for a Resilient and Flexible Power Grid

Recent widespread and extreme natural disasters as well as the drive towards clean sustainable energy sources necessitate a transformational operational and technological approaches to improve power grid resilience. Convergence of artificial intelligence (AI), distributed computing, and connectivity technologies enables these new approaches to deliver a sustainable electric grid. In particular, AI-based techniques can enable proactive decision-making by processing massive amounts of data to deliver intelligence at various levels of the system, edge devices to control room. In this chapter, we will explore the potential applications AI-based technology will unlock and challenges to address to achieve a sustainable, flexible power grid.

Omitaomu, Femi↗

DOE OSTI2024

Operational experience and R&D results using the Google Cloud for High-Energy Physics in the ATLAS experiment

The ATLAS experiment at CERN relies on a Worldwide Distributed Computing Grid infrastructure to support its physics program at the Large Hadron Collider. ATLAS has integrated cloud computing resources to complement its Grid infrastructure and conducted an R&D program on Google Cloud Platform. These initiatives leverage key features of commercial cloud providers: lightweight configuration and operation, elasticity and availability of diverse infrastructures. Here this paper examines the seamless integration of cloud computing services as a conventional Grid site within the ATLAS workflow management and data management systems, while also offering new setups for interactive, parallel analysis. It underscores pivotal results that enhance the on-site computing model and outlines several R&D projects that have benefited from large-scale, elastic resource provisioning models. Furthermore, this study discusses the impact of cloud-enabled R&D projects in three domains: accelerators and AI/ML, ARM CPUs and columnar data analysis techniques.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

MTIP Single Particle Imaging (Spinifel) v0.1.0

Performs Multi-Tiered Iterative Phasing for Single Particle Imaging. The Spinifel code uses distributed computing (using MPI and Legion) to spread work across many compute nodes. On-node computation is accelerated using GPGPUs (using CuPy, and CUDA).

Blaschke, Johannes↗

DOE OSTI2022

RLC4CLR (Reinforcement Learning Controller for Critical Load Restoration Problems)

RLC4CLR demonstrates using a reinforcement learning controller (RLC) to solve a critical load restoration (CLR) problem, which improves the grid resilience after a substation outage event. RLC4CLR consists of two parts. (1) RL environment: This environment encapsulates the CLR problem to be solved and provides interfacing functions to follow the standard OpenAI Gym format. A power system simulator, i.e., OpenDSS, is included to provide the power flow solution. Controller inputs and outputs (RL state and action) as well as the reward are defined in this environment as well. In summary, the RL environment is the problem formulation from which the RL agent can learn. (2) RL training script: The training script enables the RL agent to learn its control policy by interacting with the RL environment. For RL training, an open-sourced RL library, i.e., RLlib, is leveraged which is based on a distributed computing framework (Ray). The training script is designed to be able to be run on both local machine or the NREL HPC system. Other components of RLC4CLR include input data, e.g., grid model (standard IEEE test feeders), and other files used for results analysis.

Zhang, Xiangyu↗

DOE OSTI2023

mkite

mkite is a distributed computing platform for materials simulation. mkite is built with the server-client pattern, decoupling production databases from client runners. When used in combination with message brokers, mkite enables any available client to perform calculations without prior hardware specification on the server side. Furthermore, the software enables the creation of complex workflows with multiple inputs and branches, facilitating the exploration of combinatorial chemical spaces. The mkite suite provides recipes and tools to interact with package such as VASP, but is extensible to any other simulation package. Finally, mkite helps keeping the provenance of calculations in a SQL database. A complete description of the software is available at https://arxiv.org/abs/2301.08841.

Schwalbe Koda, Daniel↗

DOE OSTI2024

msdlive-cli-distro

MSD-LIVE, the MultiSector Dynamics – Living, Intuitive, Value-adding, Environment, is a flexible and scalable data and code management system combined with a distributed computational platform that will enable MSD researchers to document and archive their data, run their models and analysis tools, and share their data, software, and multi-model workflows within a robust Community of Practice. MSD-LIVE will facilitate a new open, collaborative, resource-rich, technology-facilitated, community-driven way of doing MSD research.

Lansing, Carina↗

DOE OSTI2025

torc (Torc Workflow Management System) [SWR-24-127]

This software package orchestrates execution of a workflow of jobs on distributed computing resources. It is optimized for use on HPCs with Slurm, but also can be used in the cloud and on local computers. Please refer to the documentation at https://nrel.github.io/torc

Thom, Daniel [National Renewable Energy Laboratory↗