Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scalable”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,225 records · Page 68
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

SpecSims: A Scalable Speculative Tree-based Simulation Cloning Framework for Finite Memory Machines

Simulation cloning is a technique in which cloned simulations whose state spaces differ partially from their parent simulation due to intervening events are spawned at runtime and concurrently advanced. It is a powerful method to carry out what-if analysis by speculatively exploring and evaluating the impact of various permutations of intervening cascade of events. Due to the exponential growth in the number of possible clones even for a small number of distinct intervening events, the practical efficacy of the approach is often severely limited by the maximum available memory of the computing host. In this paper, we introduce a novel speculative simulation cloning framework that executes a simulation cloning campaign capable of efficiently exploring an exponentially large space of clone simulations created by permutation of intervening events under a finite memory constraint. We provide a theoretical analysis of the runtime characteristics of our proposed approach and highlight its novel advantages such as memory-aware and as-long-as-needed execution. Furthermore, in support of our analytical findings and to demonstrate its practical feasibility, we implement a prototype of the cloning framework on a shared memory system and report its performance characteristics in the context of a heat diffusion simulation, and a power grid simulation subject to cascading disruptions from geomagnetic disturbances.

Simulation framework↗
DOE OSTI2025

A Digital Twin of Scalable Quantum Clouds

Quantum computing has emerged as a transformative technology capable of solving complex problems beyond the limit of classical systems. The rapid development of quantum processors has led to the proliferation of cloud-based quantum computing services offered by platforms such as IBM, Google, and Amazon. These platforms introduce unique challenges in resource allocation, job scheduling, and multi-device orchestration as quantum workloads become increasingly complex. In this work, we present a digital twin of quantum cloud infrastructures: a framework designed to model and simulate the behavior of real quantum cloud systems. Developed in Python using the SimPy discrete-event simulation library, the framework replicates key aspects of quantum cloud environments, including detailed quantum device modeling, job lifecycle management, and job fidelity. It incorporates noise-aware fidelity estimation, making it the first of its kind to simulate superconducting gate-based quantum cloud systems at an administrative level with job fidelity. We present use cases as proof of concept, demonstrating that our quantum cloud simulation framework can act as a digital twin of a quantum cloud and support the modeling and implementation of practical systems.

Luo, Waylon [Kent State University]↗
DOE OSTI2026

T-FSM: A Scalable Distributed Task-Based System for Frequent Subgraph Pattern Mining from a Big Graph

Finding frequent subgraph patterns in a big graph is an important problem with many applications such as classifying chemical compounds and building indexes to speed up graph queries. Since this problem is NP-hard, some recent parallel and distributed systems have been developed to accelerate the mining. However, they often have a huge memory cost, very long running time, suboptimal load balancing, poor scale-out capability, and possibly inaccurate results. In this article, we propose an efficient system called T-FSM for parallel mining of frequent subgraph patterns in a big graph. T-FSM supports a new anti-monotonic frequentness measure called Fraction-Score, which is more accurate than the widely used MNI measure. The execution engine of T-FSM supports both intra-machine parallelism and inter-machine parallelism. For intra-machine parallelism, T-FSM adopts a novel task-based execution model to ensure high multithreading concurrency, bounded memory consumption, and effective load balancing. For inter-machine parallelism, T-FSM ensures good scale-out performance with a lightweight pattern rebalancing approach that reduces workload skewness of pattern evaluations among machines. To avoid recomputing the contexts for migrated patterns, we design a novel context cache table to support concurrent and asynchronous requesting and caching of remote context data, which can timely evict and garbage collect used pattern contexts that are no longer needed to keep memory consumption bounded. Extensive experiments show that T-FSM is orders of magnitude faster than existing state-of-the-art parallel systems (more than 10×, 51×, 131×, 55× speedup over ScaleMine, DistGraph, Pangolin and Peregrine, respectively) and distributed systems (more than 42× and 88× over ScaleMine and DistGraph, respectively) for frequent subgraph pattern mining, and it scales out satisfactorily to 512 CPU cores on the Polaris supercomputer at Argonne National Laboratory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Scalable Federated Learning for Scientific Foundation Models on Leadership-Class Systems

Federated learning (FL) at leadership-class HPC systems remains largely unexplored, despite growing interest in deploying federated workflows on modern HPC systems. This paper provides the first system-level empirical characterization of federated fine-tuning of pretrained foundation models on an exascale supercomputer under a multi-node deployment. Using up to 96 concurrent FL clients deployed across Frontier nodes, we study the impact of client scale, model size, data heterogeneity, partial participation, and differential privacy on runtime, communication overhead, and convergence stability. Our results show that pretrained transformer models remain robust to heterogeneity, client dropout, and privacy noise, while system efficiency degrades rapidly with scale as synchronizat and orchestration dominate runtime. We further demonstrate that system-aware execution strategies, including intra-node aggregation and early aggregation, significantly reduce wall-clock time without degrading model quality. These findings establish a practical performance baseline and inform the design of communication-efficient FL systems on leadership-class HPC platforms.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗
DOE OSTI2018

Scalable molecular dynamics with NAMD on the Summit system

According to the article, NAnoscale Molecular Dynamics (NAMD) is a parallel molecular dynamics application that has been used to make breakthroughs in understanding the structure and dynamics of large biomolecular complexes, such as viruses like HIV and various types of influenza. State-of-the-art biomolecular simulations often require integration of billions of timesteps, computing all interatomic forces for each femtosecond timestep. Molecular dynamics simulation of large biomolecular systems and long-timescale biological phenomena requires tremendous computing power. NAMD harnesses the power of thousands of heterogeneous processors to meet this demand. In this paper, we present algorithmic improvements and performance optimizations that enable NAMD to achieve high performance on the IBM Newell platform (with IBM POWER9 processors and NVIDIA Volta V100 GPUs), which underpins the Oak Ridge National Laboratory's Summit and Lawrence Livermore National Laboratory's Sierra supercomputers. The Top-500 supercomputers November 2018 list shows Summit at the number one spot, with 200 petaflop/s peak performance, and Sierra second with 125 petaflop/s. Optimizations for NAMD on Summit include: data layout changes for GPU acceleration and CPU vectorization, improving GPU offload efficiency, increasing performance with Parallel Active Messaging Interface support in Charm++, improving efficiency of fast Fourier transform calculations, improving load balancing, enabling better CPU vectorization and cache performance, and providing an alternative thermostat through stochastic velocity rescaling. We also present performance scaling results on early Newell systems.

Computer Science↗
DOE OSTI2012

jShyLU Scalable Hybrid Preconditioner and Solver

ShyLU is numerical software to solve sparse linear systems of equations. ShyLU uses a hybrid direct-iterative Schur complement method, and may be used either as a preconditioner or as a solver. ShyLU is parallel and optimized for a single compute Solver node. ShyLU will be a package in the Trilinos software framework.

SciDAC↗
DOE OSTI2019

SAFIRE : Scalable and Accurate Fault Injection for Parallel Multi-threaded Applications

The software SAFIRE implements a fault injection tool to evaluate the impact of soft errors on parallel, multi-threaded applications. It implements instruction-level fault models that emulate soft errors by flipping bits in the register operands of executed machine instructions. SAFIRE is based on the LLVM compiler and utilizes novel techniques to optimize instruction instrumentation and fault injection. By detailed measurements, SAFIRE is multiple times faster than other state-of-the-art fault injection techniques, while having the same accuracy.

Georgakoudis, Giorgis↗
DOE OSTI2021

Scalable, Physical Effects Measureable Microgridfor Cyber Resilience Analysis

SPEMMCRA provides a low-cost simulated method to provide for testing of control theory and cybersecurity approaches to microgrid implementations. Specifically, this physics-based approach provides realistic responses to compare control strategies in simulation, but also to demonstrate actions that can or cannot be taken in taking cybersecurity actions . As communications interfaces or other actions are taken that affect the flow of data, SPEMMCRA can show whether the microgrid operation is adversely affected by what is intended to mitigate a cybersecurity event.

Rieger, CraigG↗
DOE OSTI2023

Scalable Object Store v. 1.0.0

SOS is software that stores data values in a set of structured flat files. The data is organized in the files based upon functions specified by the user in order to enable fast insertion and retrieval of data.

Tucker, Thomas↗
DOE OSTI2017

SSG (Scalable Service Groups)

SSG is a group membership microservice based on the Mercury RPC system. It provides mechanisms for bootstrapping sets of Mercury processes into logical groups and for managing the membership of these process groups over time. At a high-level, each group collectively maintains a group view, which is just a mapping from group member identifiers to Mercury address information. The initial group membership view is specified completely when the group is bootstrapped (created).

Carns, Phil↗
DOE OSTI2019

Scalable Reasoning System (SRS)

SRS is a web-based and mobile toolkit for information visualization and collaboration. It includes server-side components in the form of web services that provide analytical functions such as data clustering and temporal analysis. The SRS client is accessed via a web browser and uses asynchronous javascript and XML (AJAX) to interact with and visualize the results of these web services. Users of SRS can query data sources that are connected to the SRS backend, and query results are presented in the form of coordinated visual displays in a web browser that depict temporal, topical, and spatial relationships in the results. SRS also includes an early stage graphical reasoning diagram interface that helps analysts express and test hypotheses. This software was developed with funding from the Science and Technology Directorate of the U.S. Department of Homeland Security.

Pike, William↗
DOE OSTI2021

Enabling rapid COVID-19 small molecule drug design through scalable deep learning of generative models

We improved the quality and reduced the time to produce machine learned models for use in small molecule antiviral design. Our globally asynchronous multi-level parallel training approach strong scales to all of Sierra with up to 97.7% efficiency. We trained a novel, character-based Wasserstein autoencoder that produces a higher quality model trained on 1.613 billion compounds in 23 minutes while the previous state of the art takes a day on 1 million compounds. Reducing training time from a day to minutes shifts the model creation bottleneck from computer job turnaround time to human innovation time. Our implementation achieves 318 PFLOPs for 17.1% of half-precision peak. We will incorporate this model into our molecular design loop enabling the generation of more diverse compounds; searching for novel, candidate antiviral drugs improves and reduces the time to synthesize compounds to be tested in the lab.

Jacobs, Sam Ade↗
DOE OSTI2025

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗
DOE OSTI2023

Providing Geospatial Intelligence through a Scalable Imagery Pipeline

This chapter describes ORNL’s (Oak Ridge National Laboratory’s) contributions to imagery preprocessing for geospatial intelligence research and development (R&D) in four sections. First, we discuss challenges involved in building an effective imagery preprocessing workflow and the world-class high-performance computing (HPC) resources at ORNL available to process petabytes of imagery data. Second, we highlight how we developed imagery preprocessing tools over three decades while paving the way for our current cutting-edge machine learning and computer vision algorithms that are impacting humanitarian and disaster response efforts. Third, we discuss how PIPE modules work together to turn raw images into analysis-ready datasets. Fourth, we look toward the future and discuss planned advancements to PIPE and computing trends that will affect geospatial intelligence R&D.

Reith, Andrew↗