Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software Stack”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

PymePix : a python library for SPIDR readout of Timepix3

PymePix is a new Python 3 library that provides control and acquisition for the Timepix3-SPIDR hardware. The rich set of data-structures and intuitive routines reduces time and coding effort to quickly configure, acquire, and visualize data from Timepix3. The highly extensible high-performance data-pipeline allows for alteration of the Timepix3 datastream into a form that is convinient for the user. This library is intended to be easily inserted into a standard scientific software stack as well as to allow for more direct interaction of Timepix3 with interactive flavors of Python. Included with the library are two example programs using PymePix: pymepix-acq is a command line control and acquisition program that can capture UDP packets and decode them into pixels and triggers. The second is pymepixviewer, an online control and data-acquisition program for general use, but with features geared toward mass-spectroscopy and ion imaging.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Atacama Cosmology Telescope: Combined kinematic and thermal Sunyaev-Zel’dovich measurements from BOSS CMASS and LOWZ halos

The scattering of cosmic microwave background (CMB) photons off the free-electron gas in galaxies and clusters leaves detectable imprints on high resolution CMB maps: the thermal and kinematic Sunyaev-Zel’dovich effects (tSZ and kSZ respectively). We use combined microwave maps from the Atacama Cosmology Telescope DR5 and Planck in combination with the CMASS (mean redshift $\langle z \rangle$ = 0.55 and host halo mass $\langle M_{vir} \rangle$ = 3 × 10 13 M ⊙ ) and LOWZ ( $\langle z \rangle$ = 0.31 , $\langle M_{vir} \rangle$ = 5 × 10 13 M ⊙ ) galaxy catalogs from the Baryon Oscillation Spectroscopic Survey (BOSS DR10 and DR12), to study the gas associated with these galaxy groups. Using individual reconstructed velocities, we perform a stacking analysis and reject the no-kSZ hypothesis at 6.5σ, the highest significance to date. This directly translates into a measurement of the electron number density profile, and thus of the gas density profile. Despite the limited signal to noise, the measurement shows at high significance that the gas density profile is more extended than the dark matter density profile, for any reasonable baryon abundance (formally > 90 σ for the cosmic baryon abundance). We simultaneously measure the tSZ signal, i.e., the electron thermal pressure profile of the same CMASS objects, and reject the no-tSZ hypothesis at 10σ. We combine tSZ and kSZ measurements to estimate the electron temperature to 20% precision in several aperture bins, and find it comparable to the virial temperature. In a companion paper, we analyze these measurements to constrain the gas thermodynamics and the properties of feedback inside galaxy groups. We present the corresponding LOWZ measurements in this paper, ruling out a null kSZ (tSZ) signal at 2.9 (13.9) σ , and leave their interpretation to future work. This paper and the companion paper demonstrate that current CMB experiments can detect and resolve gas profiles in low mass halos and at high redshifts, which are the most sensitive to feedback in galaxy formation and the most difficult to measure any other way. They will be a crucial input to cosmological hydrodynamical simulations, thus improving our understanding of galaxy formation. These precise gas profiles are already sufficient to reduce the main limiting theoretical systematic in galaxy-galaxy lensing: baryonic uncertainties. Future such measurements will thus unleash the statistical power of weak lensing from the Rubin, Euclid and Roman observatories. Finally, our stacking software ThumbStackis publicly available and directly applicable to future Simons Observatory and CMB-S4 data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Modeling Offshore Wind Farm Performance in Coastal Low-Level Jets Using Coupled Mesoscale-Microscale Large Eddy Simulations

Accurately predicting wind farm reliability under complex offshore atmospheric conditions remains a key challenge, particularly during noncanonical meteorological events such as coastal low-level jets (LLJs). LLJs, characterized by strong nonmonotonic vertical shear and directional veer, depart significantly from the simplified inflow assumptions embedded in conventional design standards, low-fidelity engineering models, and microscale large eddy simulations of the atmospheric boundary layer. In this work, we use the virtual wind farm framework—an exascale, graphics processing unit–accelerated large eddy simulation platform coupled with high-fidelity aeroservoelastic turbine models and advanced mesoscale-microscale coupling via the ExaWind software stack—to investigate turbine responses under realistic LLJ forcing. Simulations are performed over the U.S. North Atlantic offshore domain with the use of meteorological inputs from New York State Energy Research and Development Authority buoy data, focusing on a representative LLJ case impacting the International Energy Agency 15 MW reference turbine. Our results show that LLJs can cause up to 50% power deficits in downstream turbine rows and significantly amplify low-speed shaft and tower loads through nonlinear coupling between complex inflow characteristics and turbine structural dynamics. Two primary mechanisms drive these load amplifications: (1) unique LLJ inflow features—including veer and vertical/lateral shear—and (2) the downstream evolution of the flow under stable thermal stratification, which suppresses turbulence mixing and alters wake recovery. These mechanisms produce streamwise variations in turbine loading not captured by standard hub height–based metrics or existing design load case (DLC) definitions. This study highlights the critical role of rotor-scale flow gradients in driving fatigue and system-level aeroelastic responses, challenging current DLC and control strategies. We advocate the integration of full-flow field, environment-aware wind inputs into load modeling and control algorithms. By leveraging exascale computing to resolve mesoscale-microscale coupling, this work lays the groundwork for next-generation offshore wind turbine design and operation in meteorologically complex marine environments.

17 WIND ENERGY↗
DOE OSTI2019

Exascale Deep Learning to Accelerate Cancer Research

Deep learning, through the use of neural networks, has demonstrated remarkable ability to automate many routine tasks when presented with sufficient data for training. The neural network architecture (e.g. number of layers, types of layers, connections between layers, etc.) plays a critical role in determining what, if anything, the neural network is able to learn from the training data. The trend for neural network architectures, especially those trained on ImageNet, has been to grow ever deeper and more complex. The result has been ever increasing accuracy on benchmark datasets with the cost of increased computational demands. In this paper we demonstrate that neural network architectures can be automatically generated, tailored for a specific application, with dual objectives: accuracy of prediction and speed of prediction. Using MENNDL- an HPC-enabled software stack for neural architecture search-we generate a neural network with comparable accuracy to state-of-the-art networks on a cancer pathology dataset that is also 16× faster at inference. The speedup in inference is necessary because of the volume and velocity of cancer pathology data; specifically, the previous state-of-the-art networks are too slow for individual researchers without access to HPC systems to keep pace with the rate of data generation. Our new model enables researchers with modest computational resources to analyze newly generated data faster than it is collected.

Patton, Robert↗
DOE OSTI2024

Privacy-Preserving Federated Learning for Science: Challenges and Research Directions

This paper discusses the key challenges and future research directions for privacy-preserving federated learning (PPFL), with a focus on its application to large-scale scientific AI models, in particular, foundation models~(FMs). PPFL enables collaborative model training across distributed datasets while preserving privacy-- an important collaborative approach for science. We discuss the need for efficient and scalable algorithms to address the increasing complexity of FMs, particularly when dealing with heterogeneous clients. In addition, we underscore the need for developing advance privacy-preserving techniques, such as differential privacy, to balance privacy and utility in large FMs emphasizing fairness and incentive mechanisms to ensure equitable participation among heterogeneous clients. Finally, we emphasize the need for a robust software stack supporting scalable and secure PPFL deployments across multiple high-performance computing facilities. We envision that PPFL would play a crucial role to advance scientific discovery and enable large-scale, privacy-aware collaborations across science domains.

Kim, Kibaek [Argonne National Laboratory (ANL)]↗
DOE OSTI2021

IRIS: A Portable Runtime System Exploiting Multiple Heterogeneous Programming Systems

Across embedded, mobile, enterprise, and HPC systems, computer architectures are becoming more heterogeneous and complex. This complexity is causing a crisis in programming systems and performance portability. Several programming systems are working to address these challenges, but the increasing architectural diversity is forcing software stacks and applications to specialize for each architecture. As we show, all of these approaches critically depend on their runtime system for discovery, execution, scheduling, and data orchestration. To address this challenge, we believe that a more agile and proactive runtime system is essential to increase performance portability and improve user productivity. In this regard, we have designed and implemented IRIS: a portable runtime system exploiting multiple heterogeneous programming systems. IRIS can discover available resources, manage multiple diverse programming systems (e.g., CUDA, Hexagon, HIP, Level Zero, OpenCL, OpenMP) simultaneously in the same execution, respect data dependencies, orchestrate data movement proactively, and provide for user-configurable scheduling. Our evaluation on three architectures, ranging from Qualcomm Snapdragon to a Summit supercomputer node, shows that IRIS improves portability across a wide range of diverse heterogeneous architectures with negligible overhead.

Kim, Jungwon↗
DOE OSTI2021

SYMBIOSYS: A Methodology for Performance Analysis of Composable HPC Data Services

Microservices are a powerful new way of building, customizing, and deploying distributed services owing to their flexibility and maintainability. Several large-scale distributed platforms have emerged to serve the growing needs of data-centric workloads and services in commercial computing. Concurrently, high-performance computing (HPC) systems and software are rapidly evolving to meet the demands of diversified applications and heterogeneity. The interplay of hardware factors, software configuration parameters, and the flexibility offered with a microservice architecture makes it nontrivial to estimate the optimal service instantiation for a given application workload. Further, this problem is exacerbated when considering that these services operate in a dynamic and heterogeneous HPC environment. An optimally integrated service can be vastly more performant than a haphazardly integrated one. Existing performance tools for HPC either fail to understand the request-response model of communication inherent to microservices or they operate within a narrow scope, limiting the insight that can be gleaned from employing them in isolation. We propose a methodology for integrated performance analysis of HPC microservices frameworks and applications called SYMBIOSYS. We describe its design and implementation within the context of the Mochi framework. This integration is achieved by combining distributed callpath profiling and tracing with a performance data exchange strategy that collects fine-grained, low-level metrics from the RPC communication library and network layers. The result is a portable, low-overhead performance analysis setup that provides a holistic profile of the dependencies among microservices and how they interact with the Mochi RPC software stack. Using HEPnOS, a production-quality Mochi data service, we demonstrate the low-overhead operation of SYMBIOSYS at scale and use it to identify the root causes of poorly performing service configurations.

microservices↗
DOE OSTI2022

Strategies for Integrating Deep Learning Surrogate Models with HPC Simulation Applications

The emerging trend of the convergence of high performance computing (HPC), machine learning/deep learning (ML/DL), and big data analytics presents a host of challenges for large-scale computing campaigns that seek best practices to interleave traditional scientific simulation-based workloads with ML/DL models. A portfolio of systematic approaches to incorporate deep learning into modeling and simulation serves a vital need when we support AI for science at a computing facility. In this paper, we evaluate several strategies for deploying deep learning surrogate models in a representative physics application on supercomputers at the Oak Ridge Leadership Computing Facility (OLCF). We discuss a set of recommended deployment architectures and implementation approaches. We analyze and evaluate these alternatives and show their performance and scalability up to 1000 GPUs on two mainstream platforms equipped with different deep learning hardware and software stacks.

Yin, Junqi↗
DOE OSTI2026

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗
DOE OSTI2021

The Kokkos EcoSystem: Comprehensive Performance Portability For High Performance Computing

State of the art Engineering and Science codes have grown in complexity dramatically over the last two decades. As a consequence application teams have adopted more sophisticated development strategies, leveraging third party libraries, deploying comprehensive testing and using advanced debugging and profiling tools. In todays environment of diverse hardware platforms, these applications also desire performance portability - avoiding the need to duplicate work for various platforms - which makes it necessary that these tools and libraries also work across the various systems. The Kokkos EcoSystem provides that portable software stack. Based on the Kokkos Core Programming Model, the EcoSystem provides math libraries, interoperability capabilities with Python and Fortran, and Tools for analysing, debugging, and optimizing applications. In this paper we will provide an overview of the components, discuss some specific use cases, and highlight how co-designing these components enables a more developer friendly experience.

42 ENGINEERING↗
DOE OSTI2024

Co-design for Particle Applications at Exascale

Co-design across the Exascale Computing Project (ECP) has been critical for both enabling science applications and bringing disparate communities together. Developing and porting applications to the various high-performance computing (HPC) architectures on pre-exascale and exascale computers has been quite challenging due to the diversity of hardware features and software stacks. The Co-design Center for Particle Applications (CoPA) has developed and enhanced the Cabana and PROGRESS/BML libraries to facilitate the creation of new particle applications, make existing particle applications exascale capable, and allow teams to explore new capabilities. Particle methods from atomistic, mesoscale, continuum, through cosmological scales have been built with Cabana, along with new possibilities for application coupling. Similarly, the PROGRESS/BML library has enabled quantum particle applications with linear algebra solvers to use advanced hardware. Across these CoPA-developed libraries, the co-design abstraction layer combines performance portability with math library support to facilitate separation of concerns and directly support science runs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Advanced Computing is at the Forefront of a New “Moonshot” Revolutionizing the North American Power Grid

In the 50+ years since the first humans landed on the moon, computing has grown at breakneck speed. We are faced with another challenge that is just as daunting, and just as important to overcome-modernizing the North American electric power grid-and high-performance computing (HPC) systems with specialized software will be an important element in rising to this challenge. We describe at a high level how software developed in the ExaSGD project addresses this "moonshot" goal by utilizing exascale computing and a novel high performance solver software stack to support the mission of decarbonizing power grid operations in an environment of uncertain weather and climate. To reach the exascale benchmark the team has made a number of first-of-their-kind innovations, including novel method for stochastic optimization, fine grained parallel methods for modeling power systems, and GPU resident sparse numerical linear solvers.

17 WIND ENERGY↗
DOE OSTI2025

The Persistent Challenge of Data Locality in the Post-Exascale Era

The era of exascale computing, exemplified by systems like Frontier achieving exaflop-level performance, marks a milestone. However, the quest for sheer compute power leads to strong imbalance in system design. Hence, scaling advancements in memory, network bandwidth, and storage are also necessary and pose challenges, with a crucial need to address data locality issues. This article underscores the fundamental importance of data locality as a key abstraction for optimizing application performance. Despite notable software solutions, the growing complexity of parallelism and memory hierarchy demands performance-portable data locality solutions across diverse computing platforms. Additionally, the article revisits data locality aspects, covering hardware considerations, application perspectives, software stack abstractions, and tool support. It concludes with insights into data locality challenges and opportunities, emphasizing the ongoing significance of collaborative research for progress in this critical issue.

Unat, Didem [Koc University, Istanbul (Turkey)] (O↗
DOE OSTI2020

Comparison of simulated and experimentally measured detector impulse responses to pulsed x-rays

The Gamma Array Simulation Toolkit (GAST) is a multi-physics software stack comprised of Geant4, MATLAB, and experimentally measured photomultiplier tube (PMT) impulse response waveforms that is used to model and inform the end-toend performance of radiation detector systems. In order to validate the ability of GAST to predict the impulse response of a modeled detector configuration operated in current mode, experimental measurements using the Nevada National Security Site Transformational Diagnostics and Imaging 2 MeV endpoint energy electron linear accelerator (linac) with a tungsten target were performed. Seven different detector configurations consisting of unique combinations of scintillator material, size, reflectivity conditions, presence or absence of light guide, and PMT were evaluated. The resultant impulse response of each detector configuration was recorded and compared to a GAST simulation that emulated the respective linac experiment conditions. The comparison of the experimental and simulated detector impulse responses indicated that GAST can reliably predict detector impulse responses.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

WfBench: Automated Generation of Scientific Workflow Benchmarks

The prevalence of scientific workflows with high computational demands calls for their execution on various distributed computing platforms, including large-scale leadership-class high-performance computing (HPC) clusters. To handle the deployment, monitoring, and optimization of workflow executions, many workflow systems have been developed over the past decade. There is a need for workflow benchmarks that can be used to evaluate the performance of workflow systems on current and future software stacks and hardware platforms.We present a generator of realistic workflow benchmark specifications that can be translated into benchmark code to be executed with current workflow systems. Our approach generates workflow tasks with arbitrary performance characteristics (CPU, memory, and I/O usage) and with realistic task dependency structures based on those seen in production workflows. We present experimental results that show that our approach generates benchmarks that are representative of production workflows, and conduct a case study to demonstrate the use and usefulness of our generated benchmarks to evaluate the performance of workflow systems under different configuration scenarios.

Coleman, Taina↗
DOE OSTI2022

LaRIS: Targeting Portability and Productivity for LAPACK Codes on Extreme Heterogeneous Systems by Using IRIS

In keeping with the trend of heterogeneity in high-performance computing, hardware manufacturers and vendors are developing new architectures and associated software stacks (e.g., libraries) to harness the best possible performance from commonly used kernels (e.g., linear algebra kernels). However, kernels tuned for one architecture are not portable to others. Moreover, the coexistence of different architectures in a single node makes orchestration difficult. To address these challenges, we introduce LaRIS, a portable framework for LAPACK functionalities. LaRIS ensures a separation between linear algebra algorithms and vendor-library kernels by using the IRIS run time and IRIS-BLAS library. Such abstraction at the algorithm level makes the implementation completely agnostic to the vendor library and architecture. LaRIS uses the IRIS run time to dynamically select the vendor-library kernel and suitable processor architecture at run time. Through LU factorization, we demonstrate that LaRIS can fully utilize different heterogeneous systems by launching and orchestrating different vendor-library kernels without any change in the source code.

Monil, M. A. H.↗
DOE OSTI2022

A Visual Comparison of Silent Error Propagation

High-performance computing (HPC) systems play a critical role in facilitating scientific discoveries. Their scale and complexity (e.g., the number of computational units and software stack) continue to grow as new systems are expected to process increasingly more data and reduce computing time. However, with more processing elements, the probability that these systems will experience a random bit-flip error that corrupts a program's output also increases, which is often recognized as silent data corruption. Analyzing the resiliency of HPC applications in extreme-scale computing to silent data corruption is crucial but difficult. An HPC application often contains a large number of computation units that need to be tested, and error propagation caused by error corruption is complex and difficult to interpret. Here, to accommodate this challenge, we propose an interactive visualization system that helps HPC researchers understand the resiliency of HPC applications and compare their error propagation. Our system models an application's error propagation to study a program's resiliency by constructing and visualizing its fault tolerance boundary. Coordinating with multiple interactive designs, our system enables domain experts to efficiently explore the complicated spatial and temporal correlation between error propagations. At the end, the system integrated a nonmonotonic error propagation analysis with an adjustable graph propagation visualization to help domain experts examine the details of error propagation and answer such questions as why an error is mitigated or amplified by program execution.

97 MATHEMATICS AND COMPUTING↗