SEARCH · Engineering Papers

Results for “Distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2015

Influence of molecular clustering on the interpretation of diffractograms of hydrocarbon films from tokamak T-10

The influence of molecular clustering on the formerly suggested interpretation of diffraction patterns of hydrocarbon films formed in the vacuum vessel of the tokamak T-10 is analyzed numerically. The simulation of clustering of simple hydrocarbon molecules C(D, H){sub 4}, C{sub 2}(D, H){sub 4}, and C{sub 6}(D, H){sub 6} and molecules composed of curved graphene (fullerenes and toroidal nanotubes) is carried out with the rigid body molecular dynamics method. It is shown that formerly neglected atomic correlations C–C and C–D(H) in the amorphous hydrocarbon component decrease the calculated values of the scattered intensity in the range of scattering vector modulus 5 < q < 20 nm{sup –1} because of homogenization of scatters on the spatial scale of ∼1 nm. The allowance for these correlations does not change the diffraction patterns in the range q > 20 nm{sup –1}. The results suggest the necessity to introduce to the procedure of determining the structural content of the films, similar to those from the tokamak T-10, the clusters formed by the van der Waals adhesion of hydrocarbon molecules to “graphene” nanoparticles. This simplifies the mathematical optimization to the former level of complexity—but for an extended ensemble of objects—and makes it possible to calculate the diffraction patterns of these objects using the distributed computing resources. A modified algorithm of structural content identification on the basis of joint X-ray and neutron diffractometry is suggested.

36 MATERIALS SCIENCE↗

DOE OSTI2016

INTEGRATION OF PANDA WORKLOAD MANAGEMENT SYSTEM WITH SUPERCOMPUTERS

Abstract The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the funda- mental nature of matter and the basic forces that shape our universe, and were recently credited for the dis- covery of a Higgs boson. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 140 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data cen- ters are physically scattered all over the world. While PanDA currently uses more than 250000 cores with a peak performance of 0.3+ petaFLOPS, next LHC data taking runs will require more resources than Grid computing can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, Europe and Russia (in particular with Titan supercomputer at Oak Ridge Leadership Com- puting Facility (OLCF), Supercomputer at the National Research Center Kurchatov Institute , IT4 in Ostrava, and others). The current approach utilizes a modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single- threaded workloads in parallel on Titan s multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms. We will present our current accom- plishments in running PanDA WMS at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facility s infrastructure for High Energy and Nuclear Physics, as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

De, K↗

DOE OSTI2023

Artificial Intelligence for a Resilient and Flexible Power Grid

Recent widespread and extreme natural disasters as well as the drive towards clean sustainable energy sources necessitate a transformational operational and technological approaches to improve power grid resilience. Convergence of artificial intelligence (AI), distributed computing, and connectivity technologies enables these new approaches to deliver a sustainable electric grid. In particular, AI-based techniques can enable proactive decision-making by processing massive amounts of data to deliver intelligence at various levels of the system, edge devices to control room. In this chapter, we will explore the potential applications AI-based technology will unlock and challenges to address to achieve a sustainable, flexible power grid.

Omitaomu, Femi↗

DOE OSTI2024

Operational experience and R&D results using the Google Cloud for High-Energy Physics in the ATLAS experiment

The ATLAS experiment at CERN relies on a Worldwide Distributed Computing Grid infrastructure to support its physics program at the Large Hadron Collider. ATLAS has integrated cloud computing resources to complement its Grid infrastructure and conducted an R&D program on Google Cloud Platform. These initiatives leverage key features of commercial cloud providers: lightweight configuration and operation, elasticity and availability of diverse infrastructures. Here this paper examines the seamless integration of cloud computing services as a conventional Grid site within the ATLAS workflow management and data management systems, while also offering new setups for interactive, parallel analysis. It underscores pivotal results that enhance the on-site computing model and outlines several R&D projects that have benefited from large-scale, elastic resource provisioning models. Furthermore, this study discusses the impact of cloud-enabled R&D projects in three domains: accelerators and AI/ML, ARM CPUs and columnar data analysis techniques.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2016

Are Lock-Free Concurrent Algorithms Practically Wait-Free?

Lock-free concurrent algorithms guarantee that some concurrent operation will always make progress in a finite number of steps. Yet programmers prefer to treat concurrent code as if it were wait-free, guaranteeing that all operations always make progress. Unfortunately, designing wait-free algorithms is generally a very complex task, and the resulting algorithms are not always efficient. Although obtaining efficient wait-free algorithms has been a long-time goal for the theory community, most nonblocking commercial code is only lock-free. This article suggests a simple solution to this problem. We show that for a large class of lock-free algorithms, under scheduling conditions that approximate those found in commercial hardware architectures, lock-free algorithms behave as if they are wait-free. In other words, programmers can continue to design simple lock-free algorithms instead of complex wait-free ones, and in practice, they will get wait-free progress. Our main contribution is a new way of analyzing a general class of lock-free algorithms under a stochastic scheduler. Our analysis relates the individual performance of processes to the global performance of the system using Markov chain lifting between a complex per-process chain and a simpler system progress chain. We show that lock-free algorithms are not only wait-free with probability 1 but that in fact a general subset of lock-free algorithms can be closely bounded in terms of the average number of steps required until an operation completes. Here, to the best of our knowledge, this is the first attempt to analyze progress conditions, typically stated in relation to a worst-case adversary, in a stochastic model capturing their expected asymptotic behavior.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

Dimensionality Reduction Library v 0.2

Dimensionality Reduction Library is a C++ library for dimensionality reduction. In the context of this library, dimensionality reduction is considered to consist of 1)estimation of the intrinsic dimensionality using sampled data, 2) Finding maps that reduce the dimensionality of data (forward map) or increase the dimensionality of data (reverse map) and 3) mapping arbitray coordinates to high and low dimensionalities. The library is intended to provide a consistent interface to multiple dimensionality reduction algorithms with an efficient C++ interface that runs efficiently on multicore architectures. A few routines have been optimized with an option for GPU acceleration or distributed computation. Currently the library offers intrinsic dimensionality estimation using point-PCA reconstruction error and/ residual variance. The following dimensionality reduction methods have been implemented: Principal Component Analysis Multidimensional Scaling Locally Linear Embedding IsoMap Autoencoder Neutral Networks An executable is also supplied that can be built to allow for command-line access to the library routines. A description for an application of the library for molecular structure analysis has been published.

BROWN, WILLIAM↗

DOE OSTI2018

DBSchema_DMS

PRISM Data Management System database schema. This is a collection of DDL files that define the database schema used by the Data Management System (DMS) component of the Pan-Omics Research Information Storage and Management system (PRISM) at the Pacific Northwest National Laboratory (PNNL). PRISM incorporates a diverse set of analysis tools and allows a wide range of operations to be incorporated, leveraing a state machine that is accessible to independent, distributed computational nodes. The system has scaled well as data volume has increased since 2002, while allowing adaptability for incorporating new and improved data analysis tools for more effective proteomics research.

Monroe, Matthew↗

DOE OSTI2016

OpenStudio® Server [SWR-07-40]

OpenStudio® Server is a web application and distributed computing tool, which is the backbone of the OpenStudio Analysis Framework (OSAF). It is intended to make parametric analysis of building energy models accessible to architects, engineers, and designers via the OpenStudio PAT GUI or the OpenStudio Analysis Gem. OpenStudio Server analyses are defined by PAT projects or OSA's. Each analysis may include many OpenStudio simulations, as determined by project configuration.

Long, Nicholas↗

DOE OSTI2018

OTM-MPI (Open Traffic Models - MPI)

This software is an extension to OTM (Open Traffic Models), which is an open-source traffic simulator maintained at the Institute of Transportation Studies at U.C. Berkeley. OTM-MPI provides functionality for running OTM simulations in distributed computing environments. Its functionality includes min-cut division of traffic networks (using METIS), deployment of sub-networks on multiple compute nodes, and inter-node MPI communications.

Ugirumurera, Juliette↗

DOE OSTI2021

MTIP Single Particle Imaging (Spinifel) v0.1.0

Performs Multi-Tiered Iterative Phasing for Single Particle Imaging. The Spinifel code uses distributed computing (using MPI and Legion) to spread work across many compute nodes. On-node computation is accelerated using GPGPUs (using CuPy, and CUDA).

Blaschke, Johannes↗

DOE OSTI2022

RLC4CLR (Reinforcement Learning Controller for Critical Load Restoration Problems)

RLC4CLR demonstrates using a reinforcement learning controller (RLC) to solve a critical load restoration (CLR) problem, which improves the grid resilience after a substation outage event. RLC4CLR consists of two parts. (1) RL environment: This environment encapsulates the CLR problem to be solved and provides interfacing functions to follow the standard OpenAI Gym format. A power system simulator, i.e., OpenDSS, is included to provide the power flow solution. Controller inputs and outputs (RL state and action) as well as the reward are defined in this environment as well. In summary, the RL environment is the problem formulation from which the RL agent can learn. (2) RL training script: The training script enables the RL agent to learn its control policy by interacting with the RL environment. For RL training, an open-sourced RL library, i.e., RLlib, is leveraged which is based on a distributed computing framework (Ray). The training script is designed to be able to be run on both local machine or the NREL HPC system. Other components of RLC4CLR include input data, e.g., grid model (standard IEEE test feeders), and other files used for results analysis.

Zhang, Xiangyu↗

DOE OSTI2023

mkite

mkite is a distributed computing platform for materials simulation. mkite is built with the server-client pattern, decoupling production databases from client runners. When used in combination with message brokers, mkite enables any available client to perform calculations without prior hardware specification on the server side. Furthermore, the software enables the creation of complex workflows with multiple inputs and branches, facilitating the exploration of combinatorial chemical spaces. The mkite suite provides recipes and tools to interact with package such as VASP, but is extensible to any other simulation package. Finally, mkite helps keeping the provenance of calculations in a SQL database. A complete description of the software is available at https://arxiv.org/abs/2301.08841.

Schwalbe Koda, Daniel↗

DOE OSTI2024

msdlive-cli-distro

MSD-LIVE, the MultiSector Dynamics – Living, Intuitive, Value-adding, Environment, is a flexible and scalable data and code management system combined with a distributed computational platform that will enable MSD researchers to document and archive their data, run their models and analysis tools, and share their data, software, and multi-model workflows within a robust Community of Practice. MSD-LIVE will facilitate a new open, collaborative, resource-rich, technology-facilitated, community-driven way of doing MSD research.

Lansing, Carina↗

DOE OSTI2025

torc (Torc Workflow Management System) [SWR-24-127]

This software package orchestrates execution of a workflow of jobs on distributed computing resources. It is optimized for use on HPCs with Slurm, but also can be used in the cloud and on local computers. Please refer to the documentation at https://nrel.github.io/torc

Thom, Daniel [National Renewable Energy Laboratory↗

DOE OSTI2025

Cadmus v1.0

This software package is for distributed computation of persistent (co)homology. It is primarily intended for Topological Data Analysis (TDA) audience and scientists who use TDA methods and apply them to datasets that are too big to handle on a single machine. There are very few codes available for this; Cadmus outperforms DIPHA, if the user is interested in cohomology. The accompanying paper explaining the algorithm was accepted to ALENEX 26.

Nigmetov, Arnur [Lawrence Berkeley National Labora↗

DOE OSTI2026

Detector Interface for Streaming, Control, and Open-source integration (DISCO) v1.0.0

This suite consists of a multi-package ecosystem featuring detector emulators, EPICS areaDetector drivers, and remote server frameworks designed for the Advanced Light Source (ALS). Engineered for high-bandwidth devices—including VFCCD, Timepix3, Timepix4, and related pixel detectors—the software simulates hardware, wraps vendor SDKs into remote-callable servers, and integrates with open-source control systems. Key Capabilities: Distributed SDK Architecture: Server packages wrap hardware-specific SDKs, allowing areaDetector drivers to execute remote framework calls. This isolates proprietary libraries from the EPICS IOC, enhancing stability and enabling distributed computing across beamline networks. Device Support: Custom drivers for VFCCD, the Timepix family, and similar sensors optimize the data path from hardware control to high-speed transport. Full-Stack Emulation: Sophisticated emulator packages allow end-to-end pipeline testing and software development without requiring physical hardware or beam time. Integrated Workflows: Supports high-bandwidth streaming for real-time analysis and robust, metadata-rich file-based workflows (e.g., HDF5/NeXus). By standardizing interfaces across heterogeneous hardware, this suite reduces technical debt. It provides the ALS with a scalable, open-source solution to manage massive data rates within a unified control environment.

Mahl, Johannes [Lawrence Berkeley National Laborat↗

DOE OSTI2024

Designing and prototyping extensions to the Message Passing Interface in MPICH

As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. Here, in this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI’s progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages

The rapid evolution of large language models (LLMs) has opened new possibilities for automating various tasks in software development. This paper evaluates the capabilities of the LLaMA 2-70B model in automating these tasks for scientific applications written in commonly used programming languages. Using representative test problems, we assess the model's capacity to generate code, documentation, and unit tests, as well as its ability to translate existing code between commonly used programming languages. Our comprehensive analysis evaluates the compilation, runtime behavior, and correctness of the generated and translated code. Additionally, we assess the quality of automatically generated code, documentation, and unit tests. Here, our results indicate that while LLaMA 2-70B frequently generates syntactically correct and functional code for simpler numerical tasks, it encounters substantial difficulties with more complex, parallelized, or distributed computations, requiring considerable manual corrections. We identify key limitations and suggest areas for future improvements to better leverage AI-driven automation in scientific computing workflows.

97 MATHEMATICS AND COMPUTING↗