Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scalable computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

MIDAS2: Metagenomic Intra-species Diversity Analysis System

The Metagenomic Intra-Species Diversity Analysis System (MIDAS) is a scalable metagenomic pipeline that identifies single nucleotide variants (SNVs) and gene copy number variants in microbial populations. Here, we present MIDAS2, which addresses the computational challenges presented by increasingly large reference genome databases, while adding functionality for building custom databases and leveraging paired-end reads to improve SNV accuracy. This fast and scalable reengineering of the MIDAS pipeline enables thousands of metagenomic samples to be efficiently genotyped.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Characterization of bandwidth requirements of algorithms for extreme scale science

Technology trends have resulted in widely different rates of improvement in computational throughput as compared to data movement rates in computer systems. With future systems, the cost of data movement through the memory hierarchy is expected to become even more dominant relative to the cost of performing arithmetic operations, both in terms of time and energy. Therefore, optimizing data access costs will become ever more critical in the coming years. However, unlike the well understood model of computational complexity in terms of the number of elementary operations to execute an algorithm, the inherent data-movement complexity of computations is unknown for most algorithms. Given the crucial importance of optimizing data access costs in scalable parallel systems, it is of great interest to develop techniques for understanding the inherent data movement complexity of an algorithm. In this project, we focus on this fundamental and significant problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

PRIMA-X - Performance Retargeting of Instrumentation, Measurement, and Analysis Technologies for Exascale Computing

The PRIMA-X (Performance Retargeting of Instrumentation, Measurement, and Analysis Technologies for Exascale Computing) project is the successor of the DOE PRIMA (Performance Refactoring of Instrumentation, Measurement, and Analysis Technologies for Petascale Computing) project, which addressed the challenge of creating a core measurement infrastructure that would serve as a common platform for both integrating leading parallel performance systems (notably TAU and Scalasca) and developing next-generation scalable performance tools. The PRIMA-X project shifts the focus away from refactorization of robust performance tools towards a re-targeting of the parallel performance measurement and analysis architecture for extreme scales. The massive concurrency, asynchronous execution dynamics, hardware heterogeneity, and multi-objective prerequisites (performance, power, resilience) that identify exascale systems introduce fundamental constraints on the ability to carry forward existing performance methodologies. In particular, there must be a deemphasis of per-thread observation techniques to significantly reduce the otherwise unsustainable flood of redundant performance data. Instead, it will be necessary to assimilate multi-level resource observations into macroscopic performance views, from which resilient performance metrics can be attributed to the computational features of the application. This requires a scalable framework for node-level and system-wide monitoring and runtime analyses of dynamic performance information. Also, the interest in optimizing parallelism parameters with respect to performance and energy drives the integration of tool capabilities in the exascale environment further. Initially, PRIMA-X was a collaborative project between the University of Oregon (lead institution) and the German Research School for Simulation Sciences (GRS). Because Prof. Wolf, the PI at GRS, accepted a position as full professor at Technische Universitaet Darmstadt (TU Darmstadt) starting February 1st, 2015, the work of GRS was continued at TU Darmstadt. This report covers the work at TU Darmstadt after the transition. The first main accomplishment of TU Darmstadt is the development of different techniques to aggregate performance data on the level of threads. TU Darmstadt evaluated different schemes designed at GRS during the first phase of this project and integrated them into Score-P, a widely used performance measurement framework for HPC applications. The second main accomplishment is a substantial increase of Score-P’s scalability, achieved by improving its internal representation of the underlying system. Third, we extended our performance-modeling tool Extra-P to automatically create models with more than one input parameter (e.g., process count and input size), which was previously impossible. These models allow the performance behavior of an application to be extrapolated to larger machines. Specific applications of the new multi-parameter capability realized in this project include modeling the isoefficiency function of task-based applications and a novel co-design methodology. Further extensions of Extra-P comprise online modeling functionality, automatic refinement of the model search space, and segmented performance modeling. Further contributions include new locality metrics for both shared- and distributed-memory applications, a new way of measuring the impact of inter-application inference on the execution time of an application, and finally an automatic method that selects appropriate OpenMP constructs for the parallelization of sequential code and classifies the affected variables based on their sharing semantics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Advanced Dynamically Adaptive Algorithms for Stochastic Simulations on Extreme Scales

The focus of the project is the development of mathematical methods and high-performance com- putational tools for stochastic simulations, with a particular emphasis on computations on extreme scales. The core of the project revolves around the design of highly e cient and scalable numer- ical algorithms that can adaptively and accurately, in high dimensional spaces, resolve stochastic problems with limited smoothness, even containing discontinuities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

A fully coupled two-level Schwarz preconditioner based on smoothed aggregation for the transient multigroup neutron diffusion equations

The neutron multigroup diffusion equations is an important approximation of the neutron transport equation, and has been widely used for studying the motion of neutrons and their interactions with the stationary background materials. Solving the multigroup diffusion equations faces challenging because many variables are coupled together and a lot of computing resources are required. In this paper, we focus on the development of a scalable parallel algorithm framework for the system of equations arising from the discretization of the multigroup diffusion equations using a finite element method in space and using an implicit scheme in time. The parallel agorithm framweork consists of an inexact Jacobian-free Newton for the nonlinear equations, a Krylov subspace method for the Jacobian system and a parallel Schwarz preconditioner for the acceleration of the algorithm convergence. A coarse space based on smoothed aggregation is introduced to construct a two-level method to improve the parallel performance of the Schwarz preconditioner. We numerically shown that compared with the one-level method, the proposed two-level method works significantly better in terms of the number of linear iterations and the compute time for a system of equations with millions of unknowns and a supercomputer with thousands of processors.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

A Scalable Algorithm for Radiative Heat Transfer Using Reverse Monte Carlo Ray Tracing

Radiative heat transfer is an important mechanism in a class of challenging engineering and research problems. A direct all-to-all treatment of these problems is prohibitively expensive on large core counts due to pervasive all- to-all MPI communication. The massive heat transfer problem arising from the next generation of clean coal boilers being modeled by the Uintah framework has radiation as a dominant heat transfer mode. Reverse Monte Carlo ray tracing (RMCRT) can be used to solve for the radiative-flux divergence while accounting for the effects of participating media. The ray tracing approach used here replicates the geometry of the boiler on a multi-core node and then uses an all-to-all communication phase to distribute the results globally. The cost of this all-to-all is reduced by using an adaptive mesh approach in which a fine mesh is only used locally, and a coarse mesh is used elsewhere. Here, a model for communication and computation complexity is used to predict performance of this new method. We show this model is consistent with observed results and demonstrate excellent strong scaling to 262K cores on the DOE Titan system on problem sizes that were previously computationally intractable.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Graph states of atomic ensembles engineered by photon-mediated entanglement

Abstract Graph states are a broad family of entangled quantum states, each defined by a graph composed of edges representing the correlations between subsystems. Such states constitute versatile resources for quantum computation and quantum-enhanced measurement. Their generation and engineering require a high level of control over entanglement. Here we report on the generation of continuous-variable graph states of atomic spin ensembles, which form the nodes of the graph. We program the entanglement structure encoded in the graph edges by combining global photon-mediated interactions in an optical cavity with local spin rotations. By tuning the entanglement between two subsystems, we either localize correlations within each subsystem or enable Einstein–Podolsky–Rosen steering—a strong form of entanglement that enables the extraction of precise information from one subsystem through measurements on the other. We further engineer a four-mode square graph state, highlighting the flexibility of our approach. Our method is scalable to larger and more complex graphs, laying groundwork for measurement-based quantum computation and advanced protocols in quantum metrology.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2020

Scalability of high-performance PDE solvers

Performance tests and analyses are critical to effective high-performance computing software development and are central components in the design and implementation of computational algorithms for achieving faster simulations on existing and future computing architectures for large-scale application problems. In this article, we explore performance and space-time trade-offs for important compute-intensive kernels of large-scale numerical solvers for partial differential equations (PDEs) that govern a wide range of physical applications. We consider a sequence of PDE-motivated bake-off problems designed to establish best practices for efficient high-order simulations across a variety of codes and platforms. We measure peak performance (degrees of freedom per second) on a fixed number of nodes and identify effective code optimization strategies for each architecture. In addition to peak performance, we identify the minimum time to solution at 80% parallel efficiency. The performance analysis is based on spectral and p-type finite elements but is equally applicable to a broad spectrum of numerical PDE discretizations, including finite difference, finite volume, and h-type finite elements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A parallel hierarchical blocked adaptive cross approximation algorithm

This article presents a low-rank decomposition algorithm based on subsampling of matrix entries. The proposed algorithm first computes rank-revealing decompositions of submatrices with a blocked adaptive cross approximation (BACA) algorithm, and then applies a hierarchical merge operation via truncated singular value decompositions (H-BACA). The proposed algorithm significantly improves the convergence of the baseline ACA algorithm and achieves reduced computational complexity compared to the traditional decompositions such as rank-revealing QR. Numerical results demonstrate the efficiency, accuracy, and parallel scalability of the proposed algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Scalable stable comb-to-tone integrated RF photonic drive for superconducting qubits

The recent advent of quantum computing has the potential to overhaul security, communications, and scientific modeling. Superconducting qubits are a leading platform that is advancing noise-tolerant intermediate-scale quantum processors. The implementation requires scaling to large numbers of superconducting qubits, circuit depths, and gate speeds, wherein high-purity RF signal generation and effective cabling transport are desirable. Fiber photonic-enhanced RF signal generation has demonstrated the principle of addressing both signal generation and transport requirements, supporting intermediate qubit numbers and robust packaging efforts; however, fiber-based approaches to RF signal distribution are often bounded by their phase instability. Here, we present a silicon photonic integrated circuit-based version of a photonic-enhanced RF signal generator that demonstrates the requisite stability, as well as a path towards the necessary signal fidelity.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

ExaWorks software development kit: a robust and scalable collection of interoperable workflows technologies

Scientific discovery increasingly requires executing heterogeneous scientific workflows on high-performance computing (HPC) platforms. Heterogeneous workflows contain different types of tasks (e.g., simulation, analysis, and learning) that need to be mapped, scheduled, and launched on different computing. That requires a software stack that enables users to code their workflows and automate resource management and workflow execution. Currently, there are many workflow technologies with diverse levels of robustness and capabilities, and users face difficult choices of software that can effectively and efficiently support their use cases on HPC machines, especially when considering the latest exascale platforms. We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK). The SDK is a curated collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms. We present our experience with (1) curating those technologies, (2) integrating them to provide users with new capabilities, (3) developing a continuous integration platform to test the SDK on DOE HPC platforms, (4) designing a dashboard to publish the results of those tests, and (5) devising an innovative documentation platform to help users to use those technologies. Our experience details the requirements and the best practices needed to curate workflow technologies, and it also serves as a blueprint for the capabilities and services that DOE will have to offer to support a variety of scientific heterogeneous workflows on the newly available exascale HPC platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Asynchronous Checkpoint Migration with MRNet in the Scalable Checkpoint / Restart Library

Applications running on today's supercomputers tolerate failures by periodically saving their state in checkpoint files on stable storage, such as a parallel file system. Although this approach is simple, the overhead of writing the checkpoints can be prohibitive, especially for large-scale jobs. In this paper, we present initial results of an enhancement to our Scalable Checkpoint/Restart Library (SCR). We employ MRNet, a tree-based overlay network library, to transfer checkpoints from the compute nodes to the parallel file system asynchronously. This enhancement increases application efficiency by removing the need for an application to block while checkpoints are transferred to the parallel file system. We show that the integration of SCR with MRNet can reduce the time spent in I/O operations by as much as 15x. However, our experiments exposed new scalability issues with our initial implementation. We discuss the sources of the scalability problems and our plans to address them.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Freely scalable and reconfigurable optical hardware for deep learning

Abstract As deep neural network (DNN) models grow ever-larger, they can achieve higher accuracy and solve more complex problems. This trend has been enabled by an increase in available compute power; however, efforts to continue to scale electronic processors are impeded by the costs of communication, thermal management, power delivery and clocking. To improve scalability, we propose a digital optical neural network (DONN) with intralayer optical interconnects and reconfigurable input values. The path-length-independence of optical energy consumption enables information locality between a transmitter and a large number of arbitrarily arranged receivers, which allows greater flexibility in architecture design to circumvent scaling limitations. In a proof-of-concept experiment, we demonstrate optical multicast in the classification of 500 MNIST images with a 3-layer, fully-connected network. We also analyze the energy consumption of the DONN and find that digital optical data transfer is beneficial over electronics when the spacing of computational units is on the order of $$>10\,\upmu $$ > 10 μ m.

42 ENGINEERING↗
DOE OSTI2026

GPU-Accelerated Analytic Simulation of Sparse Ionization Signal Formation in Pixelated Projection Detector

This paper presents a GPU-accelerated simulation package, TRED, for next-generation neutrino detectors with pixelated charge readout, leveraging community-driven software ecosystems to ensure adaptability and extensibility. We introduce two generic contributions: (i) an effective-charge representation based on Gaussian quadrature rules, in which the linear- interpolation factors for the field response inside each voxel are absorbed into the effective charge, and (ii) a sparse, block- binned tensor representation that enables efficient FFT-based computation of induced signals on readout electrodes for sparsely activated detector volumes. The former captures structure inside a voxel without dense sampling, while the latter achieves low memory usage and scalable runtime, as demonstrated in bench- mark studies. The underlying data representation is applicable to large-scale detectors and to other computational problems involving sparse activity.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

A scalable exponential-DG approach for nonlinear conservation laws: With application to Burger and Euler equations

In this work, we propose an Exponential DG framework for partial differential equations. We decompose 7 governing equations into linear and nonlinear parts to which we apply the discontinuous Galerkin 8 (DG) spatial discretization. In particular, we construct the linear part using Jacobian that effectively 9 capture stiff characteristics in the system. The former is integrated analytically, whereas the latter 10 is approximated. This approach i) is stable with a large Courant number (Cr > 1); ii) supports 11 high-order solutions both in time and space; iii) is computationally favorable compared to IMEX 12 DG methods with no preconditioner; iv) becomes comparable to explicit RKDG methods on uniform 13 mesh and beneficial on non-uniform grid for Euler equations; v) is scalable in a modern massively 14 parallel computing architecture due to its explicit nature of exponential time integrators and com15 pact communication stencil of DG method. Numerical results demonstrate the performance of our 16 proposed methods through various examples. We also discuss the stability and convergence analysis 17 for our exponential DG scheme in the context of Burgers equation.

42 ENGINEERING↗
NASA NTRS2019

GeoNEX: A Cloud Gateway for Near Real-time Processing of Geostationary Satellite Products

The emergence of a new generation of geostationary satellite sensors provides land andatmosphere monitoring capabilities similar to MODIS and VIIRS with far greater temporal resolution (5-15 minutes). However, processing such large volume, highly dynamic datasets requires computing capabilities that (1) better support data access and knowledge discovery for scientists; (2) provide resources to enable real-time processing for emergency response (wildfire, smoke, dust, etc.); and (3) provide reliable and scalable services for the broader user community. This paper presents an implementation of GeoNEX (Geostationary NASA-NOAA Earth Exchange) services that integrate scientific algorithms with Amazon Web Services (AWS) to provide near realtime monitoring (~5 minute latency) capability in a hybrid cloud-computing environment. It offers a user-friendly, manageable and extendable interface and benefits from the scalability provided by Amazon Web Services. Four use cases are presented to illustrate how to (1) search and access geostationary data; (2) configure computing infrastructure to enable near real-time processing; (3) disseminate and utilize research results, visualizations, and animations to concurrent users; and (4) use a Jupyter Notebook-like interface for data exploration and rapid prototyping. As an example of (3), the Wildfire Automated Biomass Burning Algorithm (WF_ABBA) was implemented on GOES-16 and -17 data to produce an active fire map every 5 minutes over the conterminous US. Details of the implementation strategies, architectures, and challenges of the use cases are discussed.

GeoNEX↗
DOE OSTI2025

Quantum simulations of nuclear resonances with variational methods

Background: The many-body nature of nuclear physics problems poses significant computational challenges. These challenges become even more pronounced when studying the resonance states of nuclear systems, which are governed by the non-Hermitian Hamiltonian. Quantum computing, particularly for quantum many-body systems, offers a promising alternative, especially within the constraints of current noisy intermediate-scale quantum (NISQ) devices. Purpose: This work aims to simulate nuclear resonances using quantum algorithms by developing a variational framework compatible with non-Hermitian Hamiltonians and implementing it fully on a quantum simulator. Methods: We employ the complex scaling technique to extract resonance positions classically and adapt it for quantum simulations using a two-step algorithm. First, we transform the non-Hermitian Hamiltonian into a Hermitian form by using the energy variance as a cost function within a variational framework. Second, we perform 𝜃-trajectory calculations to determine optimal resonance positions in the complex energy plane. To address resource constraints on NISQ devices, we utilize Gray code (GC) encoding to reduce qubit requirements. Results: We first validate our approach using a schematic potential model that mimics a nuclear potential, successfully reproducing known resonance energies with high fidelity. We then extend the method to a more realistic 𝛼−𝛼 nuclear potential and compute the 𝐷- and 𝐺-wave resonance energies with a basis size of 𝑁=16, using only four qubits. The quantum simulation results closely match the classical values, demonstrating the feasibility of our approach. Conclusions: This study demonstrates, for the first time, that the complete 𝜃-trajectory method can be implemented on a quantum computer without relying on any classical input beyond the Hamiltonian. The results establish a scalable and efficient quantum framework for simulating resonance phenomena in nuclear systems. This work represents a significant step toward quantum simulations of open quantum systems and lays the foundation for future investigations into resonance structures in nuclear, atomic, and molecular physics.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Scalable Linked Dynamic Equilibrium (SLiDE)

SLiDE (Scalable Linked Dynamic Equilibrium) replicates the blueNOTE (National Open source Tools for general Equilibrium analysis) datastream from the Wisconsin National Data Consortium. From there, it produces a recursive dynamic computable general equilibrium model of the US states and eventually the global/national economies. The datastream operations have been set up to allow for scalability of both regions and sectors represented. The model has the capability to flexibly define functional forms similar to other general equilibrium model construction software.

Hughes, Caroline↗