SEARCH · Engineering Papers

Results for “Parallel optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

All sources NASA NTRS DOE OSTI

DOE OSTI2021

CSRI Summer Proceedings 2021

The Computer Science Research Institute (CSRI) brings university faculty and students to Sandia National Laboratories for focused collaborative research on Department of Energy (DOE) computer and computational science problems. The institute provides an opportunity for university researches to learn about problems in computer and computational science at DOE laboratories, and help transfer results of their research to programs at the labs. Some specific CSRI research interest areas are: scalable solvers, optimization, algebraic preconditioners, graph-based, discrete, and combinatorial algorithms, uncertainty estimation, validation and verification methods, mesh generation, dynamic load-balancing, virus and other malicious-code defense, visualization, scalable cluster computers, beyond Moore’s Law computing, exascale computing tools and application design, reduced order and multiscale modeling, parallel input/output, and theoretical computer science. The CSRI Summer Program is organized by CSRI and includes a weekly seminar series and the publication of a summer proceedings.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

CSRI Summer Proceedings 2021

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

DGaaS: GPU as a Service on Distributed Computing System

In the rapidly evolving landscape of scientific computing, Graphics Processing Units (GPUs) have become indispensable for their unparalleled ability to handle parallel tasks in complex calculations, simulations, and data analysis. Their utility is further magnified in machine learning and AI applications, where they significantly accelerate model training and predictive analytics. Within this context, the Triton Inference Server emerges as a pivotal open-source tool, specializing in AI inferencing and optimizing GPU utilization across various platforms and frameworks. This paper presents an in-depth study on distributed High Throughput Computing (HTC), specifically focusing on the HTCondor framework and its resource provisioning tools, GlideinWMS and HEPCloud. These systems enable large-scale scientific experiments like CMS and DUNE to efficiently access and utilize vast computational resources. The paper explores the core architectural components of GlideinWMS, including jobs, user pools, and worker nodes, and discusses their integration with GPUs and the Triton server. The primary aim of this research is to develop a solution that optimizes GPU utilization by leveraging Glideins and containers. This approach allows computational jobs, particularly those involving AI models, to use GPUs only when essential, thereby facilitating efficient sharing of limited GPU resources. To validate this architecture, the study conducted three key tests involving custom scripts, container-based servers, and Triton server deployments. However, the study faces challenges, notably in locating the Triton server and ensuring secure remote access. To address these issues, future work will focus on developing a proxy mechanism and enhancing security protocols. In conclusion, this study offers a comprehensive roadmap for effective and efficient GPU utilization in distributed High Throughput Computing. It aims to contribute significantly to the scientific community by solving pressing problems and implementing robust solutions in collaboration with the GlideinWMS and HEPCloud teams. The research sets the stage for a more efficient, scalable, and cost-effective paradigm in scientific computing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Performance Results on CPU/GPU Exascale Architectures for OMEGA: The Ocean Model for E3SM Global Applications

The US Department of Energy (DOE) conducts climate simulations on some of the world’s largest supercomputers. These exascale machines use heterogeneous architectures with both CPUs and GPUs, and scientific codes must adapt to make full use of this computing power. Los Alamos National Lab is developing Omega: The Ocean Model for E3SM Global Applications, which is specifically designed for modern exascale computers. It uses external libraries that have been optimized for a variety of architectures to run on different supercomputers. Omega is an unstructured-mesh ocean model based on TRiSK numerical methods. It will be the new ocean component of the DOE’s Energy Exascale Earth System Model (E3SM). The algorithms in Omega follow those of the current ocean component, MPAS-Ocean, but it will be written in C++ rather than Fortran to take advantage of the Kokkos performance portability library. Omega spatial operators are written as Kokkos kernels to run efficiently on both CPUs and GPUs. Work on Omega began in 2023 with a new C++ framework for unstructured mesh partitioning, halo exchanges, parallel IO, and Kokkos interfaces. The current version, Omega-0, is being developed to solve the shallow water equations and at present includes all of the tendency terms but not time stepping. Here we share the results of Omega-0 verification and performance testing. Verification includes unit tests implemented with CTest as well as convergence tests in Polaris, an in-house python package with a large suite of test problems. Performance tests compare simulations conducted on CPUs versus GPUs and across different architectures: tests are run on Frontier, which has AMD “Optimized 3rd Gen EPYC” CPUs and AMD MI250X GPUs, as well as Perlmutter, which is composed of AMD EPYC 7763 CPUs and NVIDIA A100 GPUs.

58 GEOSCIENCES↗

DOE OSTI2020

Parallel high-frequency magnetic sensing with an array of flux transformers and multi-channel optically pumped magnetometer for hand MRI application

Here, we investigate an approach for parallel high-frequency magnetic sensing based on a multi-channel radio frequency (RF) optically pumped magnetometer (OPM) coupled to multiple flux transformers (FTs) with a focus on hand magnetic resonance imaging (MRI) application at ultra-low field (ULF). Multiple RF OPM sensing channels are realized by using a single large-area alkali-metal vapor cell and two laser beams for pumping and probing, shared for all the channels. This design leads to significant cost reduction when multi-channel sensing is desirable, as in the case of ULF MRI. The FT, composed of two connected coils, serves as a transmitter of a target magnetic field to the OPM, while decoupling the OPM from untargeted magnetic fields in the sensing area that can limit the OPM performance. For hand MRI application, theoretical and numerical analysis is performed to determine an optimal geometry for the FT array that could improve signal-to-noise ratio (SNR) and sufficiently reduce crosstalk between FTs. We estimate that the optimized multi-channel FT-OPM sensor can achieve a magnetic field sensitivity of the order of 1 fT / Hz 1 / 2 above 100 kHz, which would be sufficient for 1 mm resolution MRI. In general, the multi-channel capability enables simultaneous magnetic measurements, thus reducing the sensing time and improving the SNR, and we anticipate many applications of the multi-channel FT-OPM sensor beyond the targeted here hand MRI: anatomical parallel ULF MRI of the human brain and other parts of the body, airport security screening, magnetic material imaging, and many others.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2020

Efficient Treatment of Large Active Spaces through Multi-GPU Parallel Implementation of Direct Configuration Interaction

In this study, we have extended our graphical processing unit (GPU)-accelerated direct configuration interaction program to multiple devices, reducing iteration times for configuration spaces of 165 million determinants to only 3 s using NVIDIA P100 GPUs. Similar improvements in the one- and two-particle reduced density matrix formation allow for fast analytical energy gradients and electronic properties. Our parallel algorithm enables the calculation of arbitrarily large configuration spaces (limited only by available system memory), with iteration times of 13 min for an active space of 18 electrons in 18 orbitals (2.4 billion determinants) using six consumer grade NVIDIA 1080Ti GPUs. These advances enable routine molecular dynamics simulations, geometry optimizations, and absorption spectrum calculations for molecules with large configuration spaces, a task that has heretofore required massive computational effort. In this work, we demonstrate the utility of our program by generating the absorption spectrum for diphenyl acetylene at the floating occupation molecular orbital complete active space configuration interaction level of theory. Lastly, several active spaces were investigated to assess the dependence of spectral features on orbital space dimension.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Boosting RDataFrame performance with transparent bulk event processing

RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.

Guiraud, Enrico↗

DOE OSTI2023

GPU Profiling and Optimizing xRAGE (Final Report)

Our project’s objective is to increase the efficiency of GPU-enabled kernels in xRAGE. To do so, we conduct GPU profiling with NSight Systems on xRAGE tests unsplit_sod_1d and unsplit_sedov_2d to identify bottlenecks and understand the behavior of the GPU during code execution. Next, we analyze these generated GPU profiles to locate the lines of code whose optimization have the most potential for improving runtime. We replicate the structure of the code in smaller test problems that are easier to understand, edit, and run quickly. Within these test problems, we implement two different methods of improving performance: transformation of nested loops into a single MDRangePolicy and hierarchical parallelization using teams of threads. Both methods show speedups in the test code, and after transferring them to xRAGE, they both show up to 30x speedups on various computing platforms. Profiling the edited versions of xRAGE reveals that the GPU successfully executed the bottlenecks with greater efficiency

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Design, Optimization, and Control of Floating Offshore Wind Farms for Optimal Energy Production (Final report)

The uncertainty and irregularity of ocean waves and the ocean environment is a major factor in the development of commercial scale floating wind turbines as the operation of floating structures in such an environment can lead to irregular and unpredictable loading, fatigue, and ultimately a reduction in the operational life of the turbine system which affects energy production over the lifetime of the turbine. Control solutions that can limit float motions and mitigate stressful events on the structure become essential for extending lifetime and limiting the operational uncertainty of a floating wind turbine. Digital twins are computational replicas of physical systems that operate in parallel with the operation of the physical system. Given advanced knowledge of a systems input, digital twins have the ability to predict the behavior of a system in advance, which can be valuable in the control of that system. In this project, we developed and assessed potential digital twin models developed in house and openly available (OpenFast) for use in the real time control of the six degree of freedom response motions of a floating wind turbine in ocean waves. Coupling these models with near-field real time irregular sea surface (wve) measurement/sensings and prediction models, we used the digital twin to predict how the floating turbine will respond to the incoming waves. Applying this information to a motion control system of the float, one can limit and control float motions to prevent undesirable loading events/large angular motions, thus increasing system life and ultimately contributing to optimizing energy production. Due to the computational intensity of operating a digital twin in real time, we investigated the use of artificial intelligence techniques to speed-up the processes of the digital twin, as well as the wave reconstruction/prediction models. Model tank testing at the University of Rhode Island and University of Maine both validated and demonstrated the developed techniques on simple float geometries and a scale model of the NREL 15 MW reference turbine.

17 WIND ENERGY↗

DOE OSTI2023

Evaluation of pre-training large language models on leadership-class supercomputers

Large language models (LLMs) have arisen rapidly to the center stage of artificial intelligence as the foundation models applicable to many downstream learning tasks. However, how to effectively build, train, and serve such models for many high-stake and first-principle-based scientific use cases are both of great interests and of great challenges. Moreover, pre-training LLMs with billions or even trillions of parameters can be prohibitively expensive not just for academic institutions, but also for well-funded industrial and government labs. Furthermore, the energy cost and the environmental impact of developing LLMs must be kept in mind. Here, in this work, we conduct a first-of-its-kind performance analysis to understand the time and energy cost of pre-training LLMs on the Department of Energy (DOE)’s leadership-class supercomputers. Employing state-of-the-art distributed training techniques, we evaluate the computational performance of various parallelization approaches at scale for a range of model sizes, and establish a projection model for the cost of full training. Our findings provide baseline results, best practices, and heuristics for pre-training such large models that should be valuable to HPC community at large. We also offer insights and optimization strategies for using the first exascale computing system, Frontier, to train models of the size of GPT-3 and beyond.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Improved Organic Electrochemical Transistors via Directed Crystallizable Small Molecule Templating

Organic electrochemical transistors (OECTs) are of great interest as biosensors and for flexible electronics applications. Historically, processing techniques have been widely used to improve the performance of polymers in organic electronics but have been underutilized in the field of OECTs. Here, in this work, we study the effects of a crystallizable small molecule template for improving the performance of poly(3-hexylthiophene-2,5-diyl) (P3HT)-based OECTs. We utilize 1,3,5-trichlorobenzene (TCB) as a crystallizable additive to induce directional crystallization during blade coating. This procedure results in an aligned, highly ordered, and moderately porous layer of the semiconducting polymer. We find that compared to neat P3HT solutions, films cast with TCB additive show a greater than 8× improvement in the steady state figure of merit for OECTs. Additionally, the optimum TCB loading produces anisotropic OECTs where the polymer chain aligned parallel to the current outperforms the perpendicular alignment by more than a factor of 10. We investigated the mechanism of this improvement and found that polymer alignment, polymer ordering, and film porosity all play a role in improving the device performance. Overall, these results demonstrate the importance of processing on the optimization of polymers for OECTs and suggest a route to greatly improve the performance of commercially available hydrophobic polymers.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Identification of 2-Hydroxyacyl-CoA Synthases with High Acyloin Condensation Activity for Orthogonal One-Carbon Bioconversion

One-carbon (C1) compounds are emerging as cost-effective and potentially carbon-negative feedstocks for biomanufacturing, which require efficient, versatile metabolic platforms for the synthesis of value-added products. Synthetic formyl-CoA elongation (FORCE) pathways allow diverse product synthesis from C1 compounds via iterative C1 elongation, operating independently from the host metabolism with reduced engineering complexity and improved theoretical yields. However, a major bottleneck was identified as the suboptimal kinetics of the core C1–C1 condensation enzyme, 2-hydroxyacyl-CoA synthase (HACS), catalyzing the acyloin condensation reaction between formaldehyde and formyl-CoA. Furthermore, we used a combinatorial approach of bioprospecting and rational protein engineering to identify multiple HACS variants with significantly improved activities toward C1 substrates. Sequence and structure alignment of the active variants elucidated the key regions for the catalytic function, which were targeted for mutagenesis, leading to improved catalytic efficiency. In parallel, a consecutive round of bioprospecting for homologs with high similarity with active variants revealed a highly active HACS variant exhibiting up to 7-fold improvement in catalytic efficiency (k cat /K M ) and 14-fold improvement in the FORCE pathway flux in vivo compared to the previous reports. Upon further optimization of the downstream pathway, the orthogonal C1-to-product bioconversion system showed a metabolic flux of up to 700 μM glycolate OD –1 h –1 (2.1 mmol gDCW –1 h –1 ) and an industrially relevant glycolate titer, rate, and yield of 5.2 g L –1 (67.8 mM), 0.22 g L –1 h –1 , and 94% carbon yield, respectively.

2-hydroxyacyl-CoA synthase↗

DOE OSTI2021

Invited: Bambu: an Open-Source Research Framework for the High-Level Synthesis of Complex Applications

This paper presents the open-source High-Level Synthesis research framework Bambu. The framework provides an open-source starting point to experiment with new ideas across High-Level Synthesis, high-level verification and debugging, FPGA/ASIC design, design flow space exploration, and parallel hardware accelerator design. The tool accepts as input standard C/C++ specifications and compiler intermediate representations (IRs) coming from the well-known Clang/LLVM and GCC com- pilers. The broad spectrum and flexibility of input formats allow the electronic design automation (EDA) research community to explore and integrate new transformations and optimizations. The easily extendable modular framework already includes many op- timizations and HLS benchmarks. The integration with synthesis and verification backends (commercial and open-source) allows researchers to quickly test any new finding and easily obtain performance and resource usage metrics for a given application. Different FPGA devices are supported from several different vendors: AMD/XILINX, Intel/Altera, Lattice Semiconductor, and NanoXplore. Finally, integration with the OpenRoad open-source end-to-end silicon compiler perfectly fits with the recent push towards open-source EDA.

Ferrandi, Fabrizio↗

DOE OSTI2025

Characterization and Optimization of the Fitting of Quantum Correlation Functions

This case study presents a characterization and optimization of an application code for extracting parton distribution functions from high energy electron-proton scattering data. Profiling this application code reveals that the phase-space density computation accounts for 93% of the overall execution time for a single iteration on a single core. When executing multiple iterations in parallel on a multicore system, the application spends 78% of its overall execution time idling due to load imbalance. We address these issues by first transforming the application code from Python to C++ and then tackling the application load imbalance via a hybrid scheduling strategy that combines dynamic and static scheduling. These techniques result in a 62% reduction in CPU idle time and a 2.46x speedup in overall execution time per node. In addition, the typically enabled power-management mechanisms in supercomputers (e.g., AMD Turbo Core, Intel Turbo Boost, and RAPL) can significantly impact intra-node scalability when more than 50% of the CPU cores are used. This finding underscores the importance of understanding system interactions with power management, as they can adversely impact application performance, and highlights the necessity of intra-node scaling tests to identify performance degradation that inter-node scaling tests might otherwise overlook.

Chuang, Pi-Yueh [Virginia Tech,Dept. of Computer S↗

DOE OSTI2021

A High Performance Sparse Tensor Algebra Compiler in MLIR

Sparse tensor algebra is widely used in many applications, including scientific computing, machine learning, and data analytics. The performance of sparse tensor algebra kernels strongly depends on the intrinsic characteristics of the input tensors, hence many storage formats are designed for tensors to achieve optimal performance for particular applications/architectures, which makes it challenging to implement and optimize every tensor operation of interest on a given architecture. We propose a tensor algebra domain-specific language (DSL) and compiler framework to automatically generate kernels for mixed sparse-dense tensor algebra operations. The proposed DSL provides high-level programming abstractions that resemble the familiar Einstein notation to represent tensor algebra operations. The compiler introduces a new Sparse Tensor Algebra dialect built on top of LLVM's extensible MLIR compiler infrastructure for efficient code generation while covering a wide range of tensor storage formats. Our compiler also leverages input-dependent code optimization to enhance data locality for better performance. Our results show that the performance of automatically generated kernels outperforms the state-of-the-art sparse tensor algebra compiler, with up to 20.92x, 6.39x, and 13.9x performance improvement over state-of-the-art tensor algebra compilers, for parallel SpMV, SpMM, and TTM, respectively.

Tian, Ruiqin↗

DOE OSTI2024

CG-Kit: Code Generation Toolkit for performant and maintainable variants of source code applied to Flash-X hydrodynamics simulations

CG-Kit is a new Code Generation tool-Kit that we have developed as a part of the solution for portability and maintainability for multiphysics computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. To efficiently use computing resources on a heterogeneous node, an application must have a map of computation to resources and a mechanism to move the data and computation to the resources according to the map. Most existing performance portability solutions are focussed on abstracting the expression of computations so that a unified source code can be specialized to run on different resources. However, such an approach is insufficient for a code like Flash-X, which has a multitude of code components that can be assembled in various permutations and combinations to form different instances of applications. Similar challenges apply to any code that has composability, where a single specified way of apportioning work among devices may not be optimal. Additionally, use cases arise where the optimal control flow of computation may differ for different devices while the underlying numerics remain identical. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed and shared memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. All of these challenges demand that scientific software developers apply existing knowledge about domain applications, algorithms, and computing platforms to determine custom abstractions and granularity for code generation. There is a critical lack of tools to tackle those problems. CG-Kit is designed to fill this gap by providing a user with the ability to express their desired control flow and computation-to-resource map in the form a pseudocode-like recipe. It consists of standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability toolchains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. They are agnostic to the programming language of the source code targeted for code generation. In conclusion, we demonstrate the capabilities of the toolkit with two examples, first, multithreaded variants of the basic AXPY operation, and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes.

Algorithmic portability↗

DOE OSTI2020

Hardware Design and Demonstration of a 100kW, 99% Efficiency Dual Active Half Bridge Converter Based on 1700V SiC Power MOSFET

High efficiency, high density and galvanically isolated power converters are attractive for numerous medium voltage high power applications. This paper presents a 1.5kVdc, 100kW bidirectional Dual Active Half Bridge (DAHB) using newly developed 1.7kV SiC MOSFET modules. The DAHB achieved an efficiency of 98.6% at full load of 100kW and a maximum of 99% at 45kW, operating as an isolated DC/DC converter. The converter also achieved an efficiency of 97.8% operating as an isolated DC/AC inverter. An optimized PCB-based busbar design has significantly reduced the voltage overshoot across the device, making the design suitable for 1500Vdc input application such as 1500V PV inverters. Typical partial discharge inception voltage (PDIV) of the optimized PCB busbar is 1.7kVpeak with total charge <10pC. The power density of the DAHB converter is 1.8MVA/m 3 which is much higher than traditional two-stage industry products. The DAHB converter can be used as a building block for a 4.16kV/1MVA utility scale PV systems with input parallel and output series configuration.

Xu, Wei↗

DOE OSTI2026

User Manual - HydraGNN v5.0: Distributed Implementation of Multi-Tasking Graph Neural Networks

This document serves as the user manual for HydraGNN v5.0, a scalable graph neural network (GNN) architecture for simultaneous prediction of multiple target properties using multi-task learning (MTL). This version of HydraGNN has been developed primarily to support the development, training, and deployment of predictive graph-based deep learning (DL) models for atomistic materials modeling. HydraGNN is templated over 13 message-passing policies, including invariant models (GIN, PNA, PNAPlus, GAT, MFC, CGCNN, SAGE, SchNet, DimeNet) and equivariant models (EGNN, PNAEq, PAINN, MACE), and supports distributed training via distributed data parallelism (DDP), DeepSpeed, and Fully Sharded Data Parallelism (FSDP) on leadership-class supercomputers. Although HydraGNN can be applied to problems beyond atomistic materials modeling, its current use is confined to homogeneous graphs. Additional capabilities include machine-learned interatomic potentials with energy-conserving forces, General, Powerful, and Scalable Graph Transformer (GraphGPS) global attention, periodic boundary conditions, hyperparameter optimization, mixed-precision training, and uncertainty quantification.

97 MATHEMATICS AND COMPUTING↗