Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

OpenStudio® Server [SWR-07-40]

OpenStudio® Server is a web application and distributed computing tool, which is the backbone of the OpenStudio Analysis Framework (OSAF). It is intended to make parametric analysis of building energy models accessible to architects, engineers, and designers via the OpenStudio PAT GUI or the OpenStudio Analysis Gem. OpenStudio Server analyses are defined by PAT projects or OSA's. Each analysis may include many OpenStudio simulations, as determined by project configuration.

Long, Nicholas↗
DOE OSTI2018

OTM-MPI (Open Traffic Models - MPI)

This software is an extension to OTM (Open Traffic Models), which is an open-source traffic simulator maintained at the Institute of Transportation Studies at U.C. Berkeley. OTM-MPI provides functionality for running OTM simulations in distributed computing environments. Its functionality includes min-cut division of traffic networks (using METIS), deployment of sub-networks on multiple compute nodes, and inter-node MPI communications.

Ugirumurera, Juliette↗
DOE OSTI2021

MTIP Single Particle Imaging (Spinifel) v0.1.0

Performs Multi-Tiered Iterative Phasing for Single Particle Imaging. The Spinifel code uses distributed computing (using MPI and Legion) to spread work across many compute nodes. On-node computation is accelerated using GPGPUs (using CuPy, and CUDA).

Blaschke, Johannes↗
DOE OSTI2022

RLC4CLR (Reinforcement Learning Controller for Critical Load Restoration Problems)

RLC4CLR demonstrates using a reinforcement learning controller (RLC) to solve a critical load restoration (CLR) problem, which improves the grid resilience after a substation outage event. RLC4CLR consists of two parts. (1) RL environment: This environment encapsulates the CLR problem to be solved and provides interfacing functions to follow the standard OpenAI Gym format. A power system simulator, i.e., OpenDSS, is included to provide the power flow solution. Controller inputs and outputs (RL state and action) as well as the reward are defined in this environment as well. In summary, the RL environment is the problem formulation from which the RL agent can learn. (2) RL training script: The training script enables the RL agent to learn its control policy by interacting with the RL environment. For RL training, an open-sourced RL library, i.e., RLlib, is leveraged which is based on a distributed computing framework (Ray). The training script is designed to be able to be run on both local machine or the NREL HPC system. Other components of RLC4CLR include input data, e.g., grid model (standard IEEE test feeders), and other files used for results analysis.

Zhang, Xiangyu↗
DOE OSTI2023

mkite

mkite is a distributed computing platform for materials simulation. mkite is built with the server-client pattern, decoupling production databases from client runners. When used in combination with message brokers, mkite enables any available client to perform calculations without prior hardware specification on the server side. Furthermore, the software enables the creation of complex workflows with multiple inputs and branches, facilitating the exploration of combinatorial chemical spaces. The mkite suite provides recipes and tools to interact with package such as VASP, but is extensible to any other simulation package. Finally, mkite helps keeping the provenance of calculations in a SQL database. A complete description of the software is available at https://arxiv.org/abs/2301.08841.

Schwalbe Koda, Daniel↗
DOE OSTI2024

msdlive-cli-distro

MSD-LIVE, the MultiSector Dynamics – Living, Intuitive, Value-adding, Environment, is a flexible and scalable data and code management system combined with a distributed computational platform that will enable MSD researchers to document and archive their data, run their models and analysis tools, and share their data, software, and multi-model workflows within a robust Community of Practice. MSD-LIVE will facilitate a new open, collaborative, resource-rich, technology-facilitated, community-driven way of doing MSD research.

Lansing, Carina↗
DOE OSTI2025

torc (Torc Workflow Management System) [SWR-24-127]

This software package orchestrates execution of a workflow of jobs on distributed computing resources. It is optimized for use on HPCs with Slurm, but also can be used in the cloud and on local computers. Please refer to the documentation at https://nrel.github.io/torc

Thom, Daniel [National Renewable Energy Laboratory↗
DOE OSTI2025

Cadmus v1.0

This software package is for distributed computation of persistent (co)homology. It is primarily intended for Topological Data Analysis (TDA) audience and scientists who use TDA methods and apply them to datasets that are too big to handle on a single machine. There are very few codes available for this; Cadmus outperforms DIPHA, if the user is interested in cohomology. The accompanying paper explaining the algorithm was accepted to ALENEX 26.

Nigmetov, Arnur [Lawrence Berkeley National Labora↗
DOE OSTI2026

Detector Interface for Streaming, Control, and Open-source integration (DISCO) v1.0.0

This suite consists of a multi-package ecosystem featuring detector emulators, EPICS areaDetector drivers, and remote server frameworks designed for the Advanced Light Source (ALS). Engineered for high-bandwidth devices—including VFCCD, Timepix3, Timepix4, and related pixel detectors—the software simulates hardware, wraps vendor SDKs into remote-callable servers, and integrates with open-source control systems. Key Capabilities: Distributed SDK Architecture: Server packages wrap hardware-specific SDKs, allowing areaDetector drivers to execute remote framework calls. This isolates proprietary libraries from the EPICS IOC, enhancing stability and enabling distributed computing across beamline networks. Device Support: Custom drivers for VFCCD, the Timepix family, and similar sensors optimize the data path from hardware control to high-speed transport. Full-Stack Emulation: Sophisticated emulator packages allow end-to-end pipeline testing and software development without requiring physical hardware or beam time. Integrated Workflows: Supports high-bandwidth streaming for real-time analysis and robust, metadata-rich file-based workflows (e.g., HDF5/NeXus). By standardizing interfaces across heterogeneous hardware, this suite reduces technical debt. It provides the ALS with a scalable, open-source solution to manage massive data rates within a unified control environment.

Mahl, Johannes [Lawrence Berkeley National Laborat↗
DOE OSTI2024

Designing and prototyping extensions to the Message Passing Interface in MPICH

As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. Here, in this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI’s progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages

The rapid evolution of large language models (LLMs) has opened new possibilities for automating various tasks in software development. This paper evaluates the capabilities of the LLaMA 2-70B model in automating these tasks for scientific applications written in commonly used programming languages. Using representative test problems, we assess the model's capacity to generate code, documentation, and unit tests, as well as its ability to translate existing code between commonly used programming languages. Our comprehensive analysis evaluates the compilation, runtime behavior, and correctness of the generated and translated code. Additionally, we assess the quality of automatically generated code, documentation, and unit tests. Here, our results indicate that while LLaMA 2-70B frequently generates syntactically correct and functional code for simpler numerical tasks, it encounters substantial difficulties with more complex, parallelized, or distributed computations, requiring considerable manual corrections. We identify key limitations and suggest areas for future improvements to better leverage AI-driven automation in scientific computing workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Award ER25844: Minimizing System Noise Effects for Extreme-Scale Scientific Simulation Through Function Delegation

In software running on distributed computing clusters, time spent on communication between nodes in the cluster can be a significant portion of the overall computation time; background operating system tasks and other computational noise on the nodes of the system can have a significant impact on the amount of time this communication takes, especially on large systems. The research completed in this period has improved understanding of when such noise will have a significant impact. Specifically, it was demonstrated that not just noise on the nodes, but also noise on the network between nodes can have a significant impact on computation time. It was also demonstrated that noise patterns matter more than noise intensity: very regular noise can cause less disruption than lighter (on average) but less regular noise. It was also demonstrated that the effect of noise is more prominent as the speed of the network between nodes is increased. Furthermore, a tracing tool, Netgauge, was improved via our work, and a system simulator, LogGOPSim, was developed; they can be used by application developers to improve performance of their program and by system designers to mitigate the effects of noise by adjusting the noise characteristics of the operating system. Both have been made freely available as open source programs. In the course of developing these tools, we demonstrated weaknesses in existing methodologies for modeling communication, and we introduced a more detailed model, LogGOPS, for simulating systems. Not only were the deleterious effects of noise explored but we have also offered solutions. Our studies of simulations of system noise have led to specific recommendations on tuning systems to mitigate noise. We have also improved existing approaches to mitigating noise. Non-blocking collective communication avoids the effects of noise by letting communication continue simultaneously with computation (thus being non-blocking), so that the delays in communication introduced by noise have a smaller impact on overall computation time. Potentially, noise can be reduced much further by offloading communication tasks to a separate processing element than the operating system is using. We have improved our library LibNBC, which provides an implementation of non-blocking collectives, via this work. During this research, our proposal to include non-blocking collectives (which used LibNBC as a reference implementation) in the upcoming MPI-3 standard was accepted. As MPI is a ubiquitous and important standard for communication in parallel computing, this demonstrates a certain acceptance of the practicality and desirability of non-blocking collectives. Now that non-blocking collectives are a part of the standard we can expect to see optimized platform-specific implementations of non-blocking collectives. Also as part of this work we have also developed a language GOAL (Global Operation Assembly Language) that can be used as a starting point for defining languages to express optimized communication algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2013

Elementary Particle Physics (Final Report)

vATLAS and the LHC are delivering on the promise of discovery physics at the high energy frontier. Using 4.8 fb -1 of 2011 $\sqrt{s}$=7 TeV data and the first 5.8 fb -1 of 2012 $\sqrt{s}$=8 TeV data, ATLAS published the observation of a new particle with a mass of 126 GeV with a significance of 5.9σ that is compatible with a Standard Model (SM) Higgs. The LHC is outperforming initial projections for the 2012 run, and ATLAS is on track to integrate ~20 fb -1 of proton-proton collisions in 2012 before Long Shutdown 1 (LS1) begins in Spring 2013. University of Texas at Dallas (UTD) physicists will complete work on two ATLAS analyses this fall. The first is the search for the gauge bosons of a hypothesized dark sector. For 2011 data, UTD is responsible for the dark photon search in the electron-jet channel, and we are looking forward to an expanded leadership role in the dark photon search using the full 2012 data set. Our second analysis interest is the study of X/Y/Z exotic states having cc content, which builds on our experience in this field from the BABAR experiment After completing a measurement of the Xc(3872) production cross section this fall, we will search for evidence of the Z(4430)+ which is reported by Belle but not confirmed by BABAR. The UTD group has played a strong role in ATLAS operations, with group members serving as Pixel Run Coordinator, ATLAS Shift Leader, and Pixel/Inner Detector Shifter. For most of the current 3-year funding cycle, a group member coordinated the development of the Pixel DAQ code, and another continues to build and maintain the data quality monitoring (DQM) application that is used by the Inner Detector control room shifter. Additionally, members of our group take Pixel on-call expert shifts for DQM and DAQ. We led an optoboard lifetime study to assess concerns of premature on-detector VCSEL failure using the Pixel working prototype detector at CERN. Physicists based at UTD participated through Pixel Offline DQM and ATLAS Distributed Computing Operations Shifts (ADCoS) During Summer 2012, UTD joined upgrade activities in preparation for LS1. We took a major role in the testing of Electro-Readout (ER) Bundle testing for new Service Quarter Panels (nSQPs), and we developed two utilities to measure the timing jitter and bit error rate of the Pixel readout chain for use commissioning Pixel detector upgrades. During BABAR's heyday, the UTD group pioneered the use of e + e - annihilation events with hard Initial State Radiation (ISR) to study the charm threshold region, and we carried out the first BABAR double-cc analysis. Our most recent ISR paper, written in collaboration with A. Palano (Bari) is Exclusive Production of Ds + Ds - , D S *+ Ds - , and Ds *+ Ds *- via e+ e- Annihilation with Initial-State-Radiation was published in Physical Review D 82, 052004 (2010). Work continues on a study of ISR Λc + Λc - production, and a new search to establish and study double-ss production is starting.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2015

High Fidelity Simulations of Large-Scale Wireless Networks

The worldwide proliferation of wireless connected devices continues to accelerate. There are 10s of billions of wireless links across the planet with an additional explosion of new wireless usage anticipated as the Internet of Things develops. Wireless technologies do not only provide convenience for mobile applications, but are also extremely cost-effective to deploy. Thus, this trend towards wireless connectivity will only continue and Sandia must develop the necessary simulation technology to proactively analyze the associated emerging vulnerabilities. Wireless networks are marked by mobility and proximity-based connectivity. The de facto standard for exploratory studies of wireless networks is discrete event simulations (DES). However, the simulation of large-scale wireless networks is extremely difficult due to prohibitively large turnaround time. A path forward is to expedite simulations with parallel discrete event simulation (PDES) techniques. The mobility and distance-based connectivity associated with wireless simulations, however, typically doom PDES and fail to scale (e.g., OPNET and ns-3 simulators). We propose a PDES-based tool aimed at reducing the communication overhead between processors. The proposed solution will use light-weight processes to dynamically distribute computation workload while mitigating communication overhead associated with synchronizations. This work is vital to the analytics and validation capabilities of simulation and emulation at Sandia. We have years of experience in Sandia’s simulation and emulation projects (e.g., MINIMEGA and FIREWHEEL). Sandia’s current highly-regarded capabilities in large-scale emulations have focused on wired networks, where two assumptions prevent scalable wireless studies: (a) the connections between objects are mostly static and (b) the nodes have fixed locations.

Onunkwo, Uzoma↗
DOE OSTI2018

dV/dt - Accelerating the Rate of Progress towards Extreme Scale Collaborative Science

This report introduces publications that report the results of a project that aimed to design a computational framework that enables computational experimentation at scale while supporting the model of “submit locally, compute globally”. The project focuses on estimating application resource needs, finding the appropriate computing resources, acquiring those resources,deploying the applications and data on the resources, managing applications and resources during run.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Embedded 100 Gbps Photonic Components

This innovation to fiber optic component technology increases the performance, reduces the size and reduces the power consumption of optical communications within dense network systems, such as advanced distributed computing systems and data centers. VCSEL technology is enabling short-reach (< 100 m) and >100 Gbps optical interconnections over multi-mode fiber in commercial applications.

42 ENGINEERING↗
DOE OSTI2021

Exploring OpenSNAPI Use Cases and Evolving Requirements [Slides]

Emerging system architectures are rapidly transforming in order to meet shifting requirements. Motivated by expanding data volumes, energy efficiency concerns, and the omnipresent need to improve performance, architectures are increasingly adopting a data-centric approach. At the core of this concept is the goal of minimizing data motion and instead processing data in-situ to the greatest degree possible. Therefore, data-centric designs, in contrast to conventional CPU-centric models, typically distribute compute capabilities throughout the architecture. As part of this paradigm shift, a novel class of devices known as data processing units (DPUs), alongside CPUs and GPUs, are quickly forming a third pillar of data-centric systems. These devices, which include smart network adapters and switches, seek to offload computation on data at the network edge as well as in-flight within the network fabric. The Open Smart Network API (OpenSNAPI) project seeks to develop a unified API for DPU devices. In our previous talks, we introduced the OpenSNAPI project and detailed our investigations regarding the viability of offloading compute intensive kernels to BlueField DPUs. In contrast, in this talk we detail our efforts to offload application-level file I/O to the DPU. We also discuss plans and early efforts to explore in-network compute capabilities. Finally, we describe our observations with respect to the evolving design of OpenSNAPI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

User Manual - HydraGNN: Distributed PyTorch Implementation of Multi-Headed Graph Convolutional Neural Networks

This document serves as user manual for HydraGNN, a scalable graph neural network (GNN) architecture that allows for a simultaneous prediction of multiple target properties using multi-task learning (MTL). The HydraGNN architecture is constructed by successive superposition of three different sets of layers. The first set is made of message-passing layers to exchange information across nodes in the graph and use this to update the nodal features. The second set is made of global pooling layers that aggregate information from all the nodes in the graph and map it into a scalar, and is needed only for global target properties that are related to the entire graph. The third set of layers is dedicated to the implementation of MTL, which is enabled by forking of the architecture into separate heads, each one of them dedicated to the predictive task of one specific target property. Through an object-oriented programming paradigm, HydraGNN is templated over different message-passing policies, which allows for a user-friendly hyperparameter study to assess the sensitivity of the predictive performance of the HydraGNN architecture on a specific dataset with respect to the choice of the message-passing policy. The object-oriented paradigm used by HydraGNN also allows for a user-friendly inclusion of newly developed message passing policies within the existing framework. HydraGNN supports distributed computing capabilities for scalable data reading and scalable training on leadership-class supercomputers.

97 MATHEMATICS AND COMPUTING↗