Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel application”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Analysis of Threading Libraries for High Performance Computing

With the appearance of multi-/many core machines, applications and runtime systems have evolved in order to exploit the new on-node concurrency brought by new software paradigms. POSIX threads (Pthreads) was widely-adopted for that purpose and it remains as the most used threading solution in current hardware. Lightweight thread (LWT) libraries emerged as an alternative offering lighter mechanisms to tackle the massive concurrency of current hardware. In this article, we analyze in detail the most representative threading libraries including Pthread- and LWT-based solutions. In addition, to examine the suitability of LWTs for different use cases, we develop a set of microbenchmarks consisting of OpenMP patterns commonly found in current parallel codes, and we compare the results using threading libraries and OpenMP implementations. Moreover, we study the semantics offered by threading libraries in order to expose the similarities among different LWT application programming interfaces and their advantages over Pthreads. This article exposes that LWT libraries outperform solutions based on operating system threads when tasks and nested parallelism are required.

GLT↗
DOE OSTI2022

ADG: automated generation and evaluation of many-body diagrams

The goal of the present paper is twofold. First, a novel expansion many-body method applicable to superfluid open-shell nuclei, the so-called Bogoliubov in-medium similarity renormalization group (BIMSRG) theory, is formulated. This generalization of standard single-reference IMSRG theory for closed-shell systems parallels the recent extensions of coupled cluster, self-consistent Green’s function or many-body perturbation theory. Within the realm of IMSRG theories, BIMSRG provides an interesting alternative to the already existing multi-reference IMSRG (MR-IMSRG) method applicable to open-shell nuclei. The algebraic equations for low-order approximations, i.e., BIMSRG(1) and BIMSRG(2), can be derived manually without much difficulty. However, such a methodology becomes already impractical and error prone for the derivation of the BIMSRG(3) equations, which are eventually needed to reach high accuracy. Based on a diagrammatic formulation of BIMSRG theory, the second objective of the present paper is thus to describe the third version (v3.0) of the code that automatically (1) generates all valid BIMSRG(n) diagrams and (2) evaluates their algebraic expressions in a matter of seconds. This is achieved in such a way that equations can easily be retrieved for both the flow equation and the Magnus expansion formulations of BIMSRG. Expanding on this work, the first future objective is to numerically implement BIMSRG(2) (eventually BIMSRG(3)) equations and perform ab initio calculations of mid-mass open-shell nuclei.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Fused Deposition Modeling Additive Manufacturing of Carbonized Structures via Waste-Enhanced Filaments

This report details the development, characterization, and use of coal-enhanced composite materials in additive manufacturing applications. High coal loading formulations—containing up to 70 wt.% coal—were successfully extruded and processed using commercially available 3D printers. Extensive experimental testing was conducted to assess the mechanical, thermal, and microstructural properties of the composites. In parallel, multi-scale computational modeling was employed to elucidate atomistic interactions and evaluate the effects of printing-induced defects on structural performance. Large-scale printability trials demonstrated the feasibility of fabricating complex components for tooling and construction applications, including wind turbine blade molds and modular wall sections. Techno-economic analyses demonstrated the cost-effectiveness and scalability of coal-enhanced composites for large-scale additive manufacturing applications such as wind turbine blade tooling.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2021

On ParELAG's Parallel Element-based Algebraic Multigrid and its MFEM Miniapps for H(curl) and H(div) Problems: a report including lowest and next to the lowest order numerical results

This paper presents the utilization of element-based algebraic multigrid (AMGe) hierarchies, implemented in the ParELAG (Parallel Element Agglomeration Algebraic Multigrid Upscaling and Solvers) library, to produce multilevel preconditioners and solvers for H(curl) and H(div) formulations. This involves the construction of hierarchies of compatible nested spaces, forming an exact de Rham sequence on each level. This allows the application of hybrid smoothers on all levels and AMS (Auxiliary-space Maxwell Solver) or ADS (Auxiliary-space Divergence Solver) on the coarsest levels, obtaining complete multigrid cycles. Numerical results are presented, showing the parallel performance of the proposed methods. As a part of the exposition, this paper demonstrates some of the capabilities of ParELAG and outlines some of the components and procedures within the library.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective

Emerging workloads such as artificial intelligence, big data analytics and complex multi-step workflows alongside future exascale applications are anticipated future HPC workloads, which will result in a more diverse I/O system workload and even less predictable I/O behavior and access patterns. Along with the ever increasing gap between the compute and storage performance capabilities, the in-depth understanding of extreme-scale I/O behavior and the I/O performance modeling and prediction are essential tools of the large-scale I/O evaluation process for addressing the needs of extreme-scale hybrid workloads. In this survey article, we focus on the state-of-the-art of the I/O behavior and performance analysis process for HPC systems in a 5-year time window and identify future research challenges.

Neuwirth, Sarah↗
DOE OSTI2022

Machine Learning Assisted HPC Workload Trace Generation for Leadership Scale Storage Systems

Monitoring and analyzing a wide range of I/O activities in an HPC cluster is important in maintaining mission-critical performance in a large-scale, multi-user, parallel storage system. Center-wide I/O traces can provide high-level information and fine-grained activities per application or per user running in the system. Studying such large-scale traces can provide helpful insights into the system. It can be used to develop predictive methods for making predictive decisions, adjusting scheduling policies, or providing decisions for the design of next-generation systems. However, sharing real-world I/O traces to expedite such research efforts leaves a few concerns; i) the cost of sharing the large traces is expensive due to this large size, and ii) privacy concern is an issue.We address such issues by building an end-to-end machine learn- ing (ML) workflow that can generate I/O traces for large-scale HPC applications. We leverage ML based feature selection and gener- ative models for I/O trace generation. The generative models are trained on I/O traces collected by the darshan I/O characterization tool over a period of one year. We present a two-step generation process consisting of two deep-learning models, called the feature generator and the trace generator. The combination of two-step generative models provides robustness by reducing the bias of the model and accounting for the stochastic nature of the I/O traces across different runs of an application. We evaluate the performance of the generative models and show that the two-step model can generate time-series I/O traces with less than 20% root mean square error.

Paul, Arnab↗
DOE OSTI2024

Integrating ytopt and libEnsemble to autotune OpenMC

Ytopt is a Python machine-learning-based autotuning software package developed within the ECP PROTEAS-TUNE project. The ytopt software adopts an asynchronous search framework that consists of sampling a small number of input parameter configurations and progressively fitting a surrogate model over the input-output space until exhausting the user-defined maximum number of evaluations or the wall-clock time. libEnsemble is a Python toolkit for coordinating workflows of asynchronous and dynamic ensembles of calculations across massively parallel resources developed within the ECP PETSc/TAO project. libEnsemble helps users take advantage of massively parallel resources to solve design, decision, and inference problems and expands the class of problems that can benefit from increased parallelism. In this paper we present our methodology and framework to integrate ytopt and libEnsemble to take advantage of massively parallel resources to accelerate the autotuning process. Specifically, we focus on using the proposed framework to autotune the ECP ExaSMR application OpenMC, an open source Monte Carlo particle transport code. OpenMC has seven tunable parameters some of which have large ranges such as the number of particles in-flight, which is in the range of 100,000 to 8 million, with its default setting of 1 million. Setting the proper combination of these parameter values to achieve the best performance is extremely time-consuming. Therefore, we apply the proposed framework to autotune the MPI/OpenMP offload version of OpenMC based on a user-defined metric such as the figure of merit (FoM) (particles/s) or energy efficiency energy-delay product (EDP) on Crusher at Oak Ridge Leadership Computing Facility. In conclusion, the experimental results show that we achieve the improvement up to 29.49% in FoM and up to 30.44% in EDP.

Autotuning↗
DOE OSTI2021

Using Computational Storage Devices: OpenMP/MPI and Charliecloud [Slides]

Originally used Spark and HadoopFS. Collected interesting results, but this method had its issues: limited application, too much overhead to gauge CSDs’ raw performance. Solution? Rewrite our benchmarks without Spark using Serial Python and Serial & Parallel C++ (Combinations of OpenMP & OpenMPI). Compared to Spark and Python, C++ implementation is a lot faster. Caveat: an expert with Spark or Python would likely be able to improve the performance of those implementations. Computational power of our CSDs seem to be much lower than the host machine. Using all 4 cores of a single CSD, the job takes ~6.8x longer than using just one core on the host machine. Host also seems to scale better with increasing file size. Resulting Question: When, if ever, would it make sense to use CSDs for compute rather than a much-faster host?

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Parallel-in-Time Methods for Method-of-Lines Discretizations of Nonlinear Hyperbolic PDEs and Systems (Final Report)

The work for the subcontract is situated in the area of parallel-in-time integration for hyperbolic partial differential equations (PDEs). Parallel-in-time integration is an active area of research due to its ability to enable faster numerical simulations for applications throughout many areas of science. The work in this subcontract builds on a variety of results that were obtained, as part of the work performed for Subcontract No. B648355, for the Multigrid Reduction-in-Time (MGRIT) method from [1] applied to hyperbolic PDEs. This subcontract extends these results further to more efficient methods and to the case of method-of-lines discretizations for nonlinear hyperbolic PDES and systems of PDEs. The following is a summary of the research performed and results achieved during milestone periods 1, 2 and 3 by the PI (Hans De Sterck) and Postdoctoral Research Associate (Oliver Krzysik), for required tasks 1-4 (as listed in the Statement of Work): Research over the previous year has been split into three main projects: (i) solution of acoustic equation system; (ii) solution of nonlinear scalar hyperbolic PDEs; (iii) solution of nonlinear hyperbolic systems of PDEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

The Urban Observatory: A Multi-Modal Imaging Platform for the Study of Dynamics in Complex Urban Systems

We describe an “Urban Observatory” facility designed for the study of complex urban systems via persistent, synoptic, and granular imaging of dynamical processes in cities. An initial deployment of the facility has been demonstrated in New York City and consists of a suite of imaging systems—both broadband and hyperspectral—sensitive to wavelengths from the visible (∼400 nm) to the infrared (∼13 micron) operating at cadences of ∼0.01–30 Hz (characteristically ∼0.1 Hz). Much like an astronomical survey, the facility generates a large imaging catalog from which we have extracted observables (e.g., time-dependent brightnesses, spectra, temperatures, chemical species, etc.), collecting them in a parallel source catalog. We have demonstrated that, in addition to the urban science of cities as systems, these data are applicable to a myriad of domain-specific scientific inquiries related to urban functioning including energy consumption and end use, environmental impacts of cities, and patterns of life and public health. We show that an Urban Observatory facility of this type has the potential to improve both a city’s operations and the quality of life of its inhabitants.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Metadata compression

A computer-executable method, system, and computer program product for managing a Parallel-Logged File System (PLFS) on a data storage system, wherein the data storage system is in communication with an application that updates the data storage system with checkpoint data, the computer-executable method, system, and computer program product comprising receiving a first checkpoint including data from the application; analyzing the data in the first checkpoint to determine whether a pattern exists in the checkpoint data; and upon a positive determination, adding the pattern into PLFS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Rapid Targeted Quantitation of Protein Overexpression with Direct Infusion Shotgun Proteome Analysis (DISPA-PRM)

While much effort has been placed on comprehensive quantitative proteome analysis, certain applications demand measurement of only a few target proteins from complex systems. Traditional approaches to targeted proteomics rely on nano-liquid chromatography (nLC) and targeted mass spectrometry (MS) methods, e.g., parallel reaction monitoring (PRM). However, the time requirement for nLC can limit throughput of targeted proteomics. To achieve rapid and high-throughput targeted methods, here we show that nLC separations can be eliminated and replaced with direct infusion shotgun proteome analysis (DISPA) using high-field asymmetric waveform ion mobility spectrometry (FAIMS) with PRM. We demonstrate application of DISPA-PRM for rapid targeted quantification of bacterial enzymes utilized in the production of biofuels by monitoring temporal expression in 72 metabolically engineered bacterial cultures in under 2.5 hours, with a measured dynamic range >1200-fold. Here, we conclude that DISPA-PRM presents a valuable innovative tool with results comparable to nLC-MS/MS, enabling fast and rapid detection of targeted proteins in complex mixtures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Applicability of the Delplot method for the determination of catalytic reaction sequences: Hydrodeoxygenation of $\gamma$-valerolactone on Ni 2 P/MCM-41

Contact-time measurements based on sites are used to investigate the reaction sequence of the hydrodeoxygenation of γ-valerolactone (GVL) on Ni 2 P/MCM-41. Here the study is carried out at 300 °C and 0.5 MPa with site times of 0.29–6.1 s. A reaction network is proposed in which pentanoic acid and 2-methyltetrahydrofuran (2-MTHF) are formed in parallel initially and pentanal, butane and pentane are formed subsequently. A first-rank Delplot analysis (selectivity) gives a good fit, but higher-rank Delplots do not. The lack of applicability is explained by the magnitude of the rate constants, in which the rate constant for one of the later steps (the consumption of pentanal) is substantially larger than those for the preceding steps. For this reason, pentanal demonstrated the characteristics of a primary product and consequently affected the discrimination of higher rank products. A simulation of a model sequence with assumed rate constants confirms the reason why the Delplot method does not work for this reaction. Since the rate constants cannot be determined a priori to be of close magnitude, the Delplot method cannot be generally used to ascertain reaction sequences.

42 ENGINEERING↗
DOE OSTI2026

Modeling supercritical CO 2 flow and mineralization in reactive host rocks with PFLOTRAN v7.0

Understanding the flow and reactivity of CO 2 injected into geological reservoirs is important for many subsurface applications including secure geologic carbon storage (GCS), critical mineral extraction, enhanced geothermal systems (EGS), and enhanced oil recovery (EOR). Traditionally, subsurface CO 2 injection for GCS applications has focused on geologic formations with favorable subsurface configurations for CO 2 migration and trapping through non-reactive mechanisms such as structural, solubility, and petrophysical trapping. Recently, CO 2 -reactive rocks such as mafic and ultramafic basalts have been investigated for their potential to react with injected CO 2 in situ to simultaneously dissolve host rock minerals and mineralize CO 2 as carbonates. Engineering rapid CO 2 mineralization in the subsurface is attractive because of the increased density of stored CO 2 , the additional safety factors associated with solidification, and the potential to extract valuable critical minerals. Here we present recent developments in the parallel flow and reactive transport simulator PFLOTRAN to model coupled CO 2 -brine flow and reactive transport for a wide range of injection and production applications involving reactive CO 2 -brine systems. These developments are based on the well established and trusted CO 2 flow capabilities in the STOMP-CO 2 simulator. New capabilities added to PFLOTRAN include new CO 2 -brine equations of state with optional thermal coupling, several new constitutive relationships like capillary pressure smoothing and scanning path hysteresis, a fully implicit well model, and native linkage with PFLOTRAN's well-established reactive transport libraries. A series of benchmarks between PFLOTRAN and STOMP-CO 2 verify the newly developed CO 2 -brine flow capabilities. Demonstrations of coupled CO 2 -brine flow modeling and reactive transport show how CO 2 mineralization can be engineered in reactive host rocks. Finally, an example use case involving copper leaching by CO 2 and critical mineral extraction is presented to showcase the strengths of this new implementation. Several limitations still remain, including limited availability of field data to parameterize models. Future work should constrain the evolution of mineral surface area during mineralization and the temperature and/or pH dependence of geochemical reactions for specific systems of interest.

Critical Minerals↗
DOE OSTI2021

Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression

Lossy compression techniques have demonstrated promising results in significantly reducing the scientific data size while guaranteeing the compression error bounds. However, one important yet often neglected side effect of lossy scientific data compression is its impact on the performance of parallel I/O. Our key observation is that the compressed data size is often highly skewed across processes in lossy scientific compression. To understand this behavior, we conduct extensive experiments where we apply three lossy compressors MGARD, ZFP, and SZ, which are specifically designed and optimized for scientific data, to three real-world scientific applications Gray-Scott simulation, WarpX, and XGC. Our analysis result demonstrates that the size of the compressed data is always skewed even if the original data is evenly decomposed among processes. Such skewness widely exists in different scientific applications using different compressors as long as the information density of the data varies across processes. We then systematically study how this side effect of lossy scientific data compression impacts the performance of parallel I/O. We observe that the skewness in the sizes of the compressed data often leads to I/O imbalance, which can significantly reduce the efficiency of I/O bandwidth utilization if not properly handled. In addition, writing data concurrently to a single shared file through MPI-IO library is more sensitive to the unbalanced I/O loads. Therefore, we believe our research community should pay more attention to the unbalanced parallel I/O caused by lossy scientific data compression.

Wang, Xinying↗
DOE OSTI2022

Unified Language Frontend for Physic-Informed AI/ML

Artificial intelligence and machine learning (AI/ML) are becoming important tools for scientific modeling and simulation as in several other fields such as image analysis and natural language processing. ML techniques can leverage the computing power available in modern systems and reduce the human effort needed to configure experiments, interpret and visualize results, draw conclusions from huge quantities of raw data, and build surrogates for physics based models. Domain scientists in fields like fluid dynamics, microelectronics and chemistry can automate many of their most difficult and repetitive tasks or improve the design times by use of the faster ML-surrogates. However, modern ML and traditional scientific highperformance computing (HPC) tend to use completely different software ecosystems. While ML frameworks like PyTorch and TensorFlow provide Python APIs, most HPC applications and libraries are written in C++. Direct interoperability between the two languages is possible but is tedious and error-prone. In this work, we show that a compiler-based approach can bridge the gap between ML frameworks and scientific software with less developer effort and better efficiency. We use the MLIR (multi-level intermediate representation) ecosystem to compile a pre-trained convolutional neural network (CNN) in PyTorch to freestanding C++ source code in the Kokkos programming model. Kokkos is a programming model widely used in HPC to write portable, shared-memory parallel code that can natively target a variety of CPU and GPU architectures. Our compiler-generated source code can be directly integrated into any Kokkosbased application with no dependencies on Python or cross-language interfaces.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Pathway Evolution Through a Bottlenecking-Debottlenecking Strategy and Machine Learning-Aided Flux Balancing

The evolution of pathway enzymes enhances the biosynthesis of high-value chemicals, crucial for pharmaceutical, and agrochemical applications. However, unpredictable evolutionary landscapes of pathway genes often hinder successful evolution. Here, the presence of complex epistasis is identifued within the representative naringenin biosynthetic pathway enzymes, hampering straightforward directed evolution. Subsequently, a biofoundry-assisted strategy is developed for pathway bottlenecking and debottlenecking, enabling the parallel evolution of all pathway enzymes along a predictable evolutionary trajectory in six weeks. This study then utilizes a machine learning model, ProEnsemble, to further balance the pathway by optimizing the transcription of individual genes. The broad applicability of this strategy is demonstrated by constructing an Escherichia coli chassis with evolved and balanced pathway genes, resulting in 3.65 g L -1 naringenin. The optimized naringenin chassis also demonstrates enhanced production of other flavonoids. This approach can be readily adapted for any given number of enzymes in the specific metabolic pathway, paving the way for automated chassis construction in contemporary biofoundries.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

On a Simplified Approach to Achieve Parallel Performance and Portability Across CPU and GPU Architectures

This paper presents software advances to easily exploit computer architectures consisting of a multi-core CPU and CPU+GPU to accelerate diverse types of high-performance computing (HPC) applications using a single code implementation. The paper describes and demonstrates the performance of the open-source C++ matrix and array (MATAR) library that uniquely offers: (1) a straightforward syntax for programming productivity, (2) usable data structures for data-oriented programming (DOP) for performance, and (3) a simple interface to the open-source C++ Kokkos library for portability and memory management across CPUs and GPUs. The portability across architectures with a single code implementation is achieved by automatically switching between diverse fine-grained parallelism backends (e.g., CUDA, HIP, OpenMP, pthreads, etc.) at compile time. The MATAR library solves many longstanding challenges associated with easily writing software that can run in parallel on any computer architecture. This work benefits projects seeking to write new C++ codes while also addressing the challenges of quickly making existing Fortran codes performant and portable over modern computer architectures with minimal syntactical changes from Fortran to C++. We demonstrate the feasibility of readily writing new C++ codes and modernizing existing codes with MATAR to be performant, parallel, and portable across diverse computer architectures.

97 MATHEMATICS AND COMPUTING↗