Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

NCCS High Performance GMRES Mixed Precision

HPG-MxP is a software package that performs a fixed number of multigrid preconditioned (using a Gauss-Seidel smoother) Generalized minimal residual (PGMRES) iterations in order to solve a possibly nonsymmetric large sparse linear system of equations. It is designed to be a benchmark to measure a computer's performance for sparse linear algebra workloads typical in scientific computing while allowing the use of mixed precision methods. The solution is required to have convergence characteristics and accuracy similar to double precision GMRES. It is based on the High Performance Conjugate Gradient Benchmark (HPCG) which restricts all implementations to use only the IEEE double precision format (FP64). The original implementation (https://github.com/hpg-mxp/hpg-mxp) was written by Ichitaro Yamazaki, Jennifer Loe, Christian Glusa, Sivasankaran Rajamanickam, Piotr Luszczek, and Jack Dongarra. Please refer to that repository for documentation on the original implementation. This version is maintained by the National Center for Computational Sciences at Oak Ridge National Laboratory. It is highly scalable and optimized for Oak Ridge Leadership Computing Facility (OLCF) systems, particularly Frontier.

Kashi, Aditya [Oak Ridge National Laboratory (ORNL↗
DOE OSTI2023

Investigation of thermal hydraulic behavior of the High Temperature Test Facility's lower plenum via large eddy simulation

A high-fidelity computational fluid dynamics (CFD) analysis was performed using the Large Eddy Simulation (LES) model for the lower plenum of the High–Temperature Test Facility (HTTF), a ¼ scale test facility of the modular high temperature gas-cooled reactor (MHTGR) managed by Oregon State University. In most next–generation nuclear reactors, thermal stress due to thermal striping is one of the risks to be curiously considered. This is also true for HTGRs, especially since the exhaust helium gas temperature is high. In order to evaluate these risks and performance, organizations in the United States led by the OECD NEA are conducting a thermal hydraulic code benchmark for HTGR, and the test facility used for this benchmark is HTTF. HTTF can perform experiments in both normal and accident situations and provide high-quality experimental data. However, it is difficult to provide sufficient data for benchmarking through experiments, and there is a problem with the reliability of CFD analysis results based on Reynolds–averaged Navier–Stokes to analyze thermal hydraulic behavior without verification. To solve this problem, high-fidelity 3-D CFD analysis was performed using the LES model for HTTF. It was also verified that the LES model can properly simulate this jet mixing phenomenon via a unit cell test that provides experimental information. As a result of CFD analysis, the lower the dependency of the sub-grid scale model, the closer to the actual analysis result. In the case of unit cell test CFD analysis and HTTF CFD analysis, the volume-averaged sub-grid scale model dependency was calculated to be 13.0% and 9.16%, respectively. As a result of HTTF analysis, quantitative data of the fluid inside the HTTF lower plenum was provided in this paper. As a result of qualitative analysis, the temperature was highest at the center of the lower plenum, while the temperature fluctuation was highest near the edge of the lower plenum wall. The power spectral density of temperature was analyzed via fast Fourier transform (FFT) for specific points on the center and side of the lower plenum. FFT results did not reveal specific frequency-dominant temperature fluctuations in the center part. It was confirmed that the temperature power spectral density (PSD) at the top increased from the center to the wake. The vortex was visualized using the well-known scalar Q-criterion, and as a result, the closer to the outlet duct, the greater the influence of the mainstream, so that the inflow jet vortex was dissipated and mixed at the top of the lower plenum. Additionally, FFT analysis was performed on the support structure near the corner of the lower plenum with large temperature fluctuations, and as a result, it was confirmed that the temperature fluctuation of the flow did not have a significant effect near the corner wall. In addition, the vortices generated from the lower plenum to the outlet duct were identified in this paper. It is considered that the quantitative and qualitative results presented in this paper will serve as reference data for the benchmark.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Using Computational Storage Devices: OpenMP/MPI and Charliecloud [Slides]

Originally used Spark and HadoopFS. Collected interesting results, but this method had its issues: limited application, too much overhead to gauge CSDs’ raw performance. Solution? Rewrite our benchmarks without Spark using Serial Python and Serial & Parallel C++ (Combinations of OpenMP & OpenMPI). Compared to Spark and Python, C++ implementation is a lot faster. Caveat: an expert with Spark or Python would likely be able to improve the performance of those implementations. Computational power of our CSDs seem to be much lower than the host machine. Using all 4 cores of a single CSD, the job takes ~6.8x longer than using just one core on the host machine. Host also seems to scale better with increasing file size. Resulting Question: When, if ever, would it make sense to use CSDs for compute rather than a much-faster host?

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Battle of the Defaults: Extracting Performance Characteristics of HDF5 under Production Load

Popular parallel I/O libraries, such as HDF5, provide tuning parameters to obtain superior performance. However, the selection of effective parameters on production systems is complex due to the interdependence of I/O software and file system layers. Hence, application developers typically use the default parameters and often experience poor I/O performance. This work conducts a benchmarking-based analysis on the HDF5 behaviors with a wide variety of I/O patterns to extract performance characteristics under the production workload. To make the analysis well controlled, we exercise I/O benchmarks on POSIX-IO, MPI-IO, and HDF5 using the same I/O patterns and in the same jobs. To address high performance variability in production environments, we repeat the benchmarks across I/O patterns, storage devices, and time intervals. Based on the results, we identified consistent HDF5 behaviors that appropriate configurations and operations on dataset layout and file-metadata placement can improve performance significantly. We apply our findings and evaluate the tuned I/O library on two supercomputers: Summit and Cori. The results show that our tuned parameters can achieve more than 10× I/O performance speedup than that with default parameters on both systems, suggesting the effectiveness, stability, and generality of our solution.

Xie, Bing↗
DOE OSTI2022

..beta..-Ketoadipic Acid Production in P. putida for Performance-Advantaged Nylons

Biomass derived chemicals can possess extended functionality, often in the form of heteroatoms like oxygen and nitrogen, that can enable performance advantaged properties in emergent materials. One illustrative example of this concept is ..beta..-ketoadipic acid (..beta..KA), a C6 dicarboxylic acid with a ketone in its backbone. ..beta..KA can be obtained via biological cultivation from aromatic compounds, sugars, and even waste plastics. When ..beta..KA is implemented into polymers, namely nylons, in place of adipic acid the overall material properties are increased. Specifically, the glass transition temperature (which is the measurement at which temperature a plastic softens) is increased by 69 degrees C and the water permeability is reduced by 20%. Molecular dynamic (MD) simulations reveal that the ..beta..-ketone in ..beta..KA leads to restricted movement across multiple backbone carbons when compared to adipic acid. Furthermore, MD simulations reveal that the ..beta..-ketone, when compared to no-ketone or an a-ketone, can lead to an increase in hydrogen bonding between neighboring chains. Both phenomena may help explain the molecular origin of the increases in polymer performance. Aside from performance advantages in thermomechanical performance, the production of ..beta..KA also is advantaged in its manufacturing. Although estimates for the minimum selling price (MSP) of ..beta..KA exceed adipic acid the manufacture of ..beta..KA would reduce supply chain energy and GHG emissions by >50% and >30% respectively. Further TEA analysis reveals that the MSP of ..beta..KA is primarily driven by the feedstock cost while the capital expenditures are driven largely by fermentation costs. Overall, the work presented within will be illustrative of how biomass derived molecules, namely ..beta..KA, can manifest performance advantages across multiple benchmarks and may enable a path to market for biomass-derived materials.

beta-ketoadipic acid↗
DOE OSTI2020

Characterizing Output Bottlenecks of a Production Supercomputer: Analysis and Implications

This article studies the I/O write behaviors of the Titan supercomputer and its Lustre parallel file stores under production load. The results can inform the design, deployment, and configuration of file systems along with the design of I/O software in the application, operating system, and adaptive I/O libraries.We propose a statistical benchmarking methodology to measure write performance across I/O configurations, hardware settings, and system conditions. Moreover, we introduce two relative measures to quantify the write-performance behaviors of hardware components under production load. In addition to designing experiments and benchmarking on Titan, we verify the experimental results on one real application and one real application I/O kernel, XGC and HACC IO, respectively. These two are representative and widely used to address the typical I/O behaviors of applications.In summary, we find that Titan’s I/O system is variable across the machine at fine time scales. This variability has two major implications. First, stragglers lessen the benefit of coupled I/O parallelism (striping). Peak median output bandwidths are obtained with parallel writes to many independent files, with no striping or write sharing of files across clients (compute nodes). I/O parallelism is most effective when the application—or its I/O libraries—distributes the I/O load so that each target stores files for multiple clients and each client writes files on multiple targets in a balanced way with minimal contention. Second, our results suggest that the potential benefit of dynamic adaptation is limited. In particular, it is not fruitful to attempt to identify “good locations” in the machine or in the file system: component performance is driven by transient load conditions and past performance is not a useful predictor of future performance. For example, we do not observe diurnal load patterns that are predictable.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Shielding Benchmark Comparison - MCNP6.2

This report documents the calculations performed for a shielding code comparison between various Department of Energy (DOE) sites. These shielding calculations were performed using an experiment drawn from the International Handbook of Evaluated Criticality Safety Benchmark Experiments, published by the Organisation for Economic Cooperation and Development/Nuclear Energy Agency (OECD/NEA). The benchmark selected for comparison is ALARM-CF-AIR-LAB-001 (“Neutron Fields in the Three-Section Concrete Labyrinth from Cf-252 Source, Benchmark ALARM CF AIR LAB-001”). The Y-12 submission for this code comparison was performed with Monte Carlo N Particle (MCNP) Transport Code System, Version 6.2 and Automated Variance Reduction Generator (ADVANTG).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

TransPolymer: a Transformer-based language model for polymer property predictions

Accurate and efficient prediction of polymer properties is of great significance in polymer design. Conventionally, expensive and time-consuming experiments or simulations are required to evaluate polymer functions. Recently, Transformer models, equipped with self-attention mechanisms, have exhibited superior performance in natural language processing. However, such methods have not been investigated in polymer sciences. Herein, we report TransPolymer, a Transformer-based language model for polymer property prediction. Our proposed polymer tokenizer with chemical awareness enables learning representations from polymer sequences. Rigorous experiments on ten polymer property prediction benchmarks demonstrate the superior performance of TransPolymer. Moreover, we show that TransPolymer benefits from pretraining on large unlabeled dataset via Masked Language Modeling. Experimental results further manifest the important role of self-attention in modeling polymer sequences. We highlight this model as a promising computational tool for promoting rational polymer design and understanding structure-property relationships from a data science view.

36 MATERIALS SCIENCE↗
DOE OSTI2025

System Engineers and Decisions: It?s All about Knowledge

In order to guarantee that a system meets adequate levels of reliability and availability, system performances are continuously monitored and analyzed thanks to the technological advancements driving the Industry 4.0 revolution. An Industry 4.0 approach is typically based on advanced statistical, big data mining, machine learning, and internet-of-things methods designed to detect anomalies in the behavior of system, detect the most likely failure modes, and provide indications to system engineers on when maintenance activities should be performed before system performance are deemed unacceptable (which can be generated by diagnostic and prognostic methods). However, these analyses, which are designed to automatize and increase the efficacy of the system maintenance program, require large amount of data which can come in various forms: numeric, textual, images, sounds etc. Such data constitutes the historic knowledge benchmark to track system performances and support system engineer decisions. Here we claim that data is not sufficient to support this kind of analyses when applied to systems characterized by complex architectures and behaviors. Robust system engineer decisions require the ability to understand the system operational context that lies behind the observed data elements. In this respect, system models are in fact necessary to “put data in context” and capture relationships between data elements. Industry 4.0 methods require in fact contextual knowledge as a basis upon which hypotheses can be generated and assumptions tested. In our view, for complex systems, model-based system engineering (MBSE) models can afford this contextual knowledge, as they are typically used to describe systems architecture and dynamic behaviors. System knowledge is here intended as the blending of collected data and system architecture which takes the form of a “knowledge graph”. A knowledge graph is a database which consists of a large set of nodes (in our case an entity can be either a data or an MBSE element) which are linked to each other. The types of nodes and links follow a pre-defined topology, sometimes also refers as an ontology, that is designed to fit the actual decisions that needs to be performed. We show here how a knowledge graph can be defined to support system engineer maintenance decisions and how the same graph can be built based on system MBSE models and pre-processed data from numeric (through anomaly detections and diagnostic methods) and textual elements (through technical language processing TLP).

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Techno-Economic Implications of Electrical Machine Scaling for Wave Energy Converters

The sizing of an electrical machine for a Wave Energy Converter (WEC) can have a substantial impact on the overall sizing, cost, and rating of the device. An electrical generator is typically part of the power take-off system, which is the mechanism by which the energy absorbed by the prime mover is transformed into usable electrical energy. For practically all WECs, the rate of change of actuation is predominantly determined by the wave resource (i.e., the wave height and frequency), and devices will see a sinusoidal varying velocity according to the wave conditions. The same can then be said for both directly and indirectly coupled power take-offs with electrical generators. This techno-economic study investigates electrical machine scaling and associated cost implications through core machine design theory, manufacturer data, supporting literature, and the Reference Model Project sponsored by the U.S. Department of Energy. The Reference Model Project was a partnered effort to develop open-source marine energy point designs as reference models to benchmark marine energy technology performance and costs, methods for design and analysis of marine energy technologies, estimations for capital costs, operational costs, and levelized cost of energy. The results from this study show torque is directly related to (1) the physical size of the machine required to increase the air-gap sheer stresses, (2)the amount of active material, (3) the support structure, (4) bearing size and rating, and (5) offshore cable rating, all of which have a significant effect on overall system costs in terms of both capital and operational expenditures. This paper aims to be a critical benchmark in helping determine an "optimal" nameplate rating for wave energy devices and their associated power take-offs. With an optimized rating and sizing process, WEC costs can be reduced and overall performance can be improved.

cables↗
DOE OSTI2025

tether

Tether is a python module for benchmarking and assessing large language model (LLMs) performance at generic scientific tasks. The code generates benchmarks, uses the benchmark to prompt LLMs through automatic programming interfaces (APIs), and then logs the number of prompts an LLM correctly answers and presents the results as a completed benchmark.

Kaiser, Bryan [Los Alamos National Laboratory]↗
DOE OSTI2021

Construction, installation and operation of ProtoDUNE-SP

The single-phase liquid argon TPC at CERN (ProtoDUNE-SP) is an engineering prototype for the first module of the DUNE far detector. This prototype, which has dimensions of a cube of about 10 m edge, provides full validation of the use of the membrane tank technology for large dimension cryostats. Furthermore, the very high performance of the protoDUNE-SP TPC with more than 500 days of continuous and stable operation, demonstrated the reliability of the LAr detection technology at a scale never tested before. In this talk we will review the main characteristics and milestones of the construction and installation of protoDUNE-SP which provide a series of benchmarks for DUNE. The performance for several different detector working points will also be discussed.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

LANL Critical Benchmark Comparison Study and Subsequent Revision for Cases Involving LEU and MIX

As part of an international collaboration within the DOE Nuclear Criticality Safety Program (NCSP), LANL is involved in a comparison study to quantify differences in k-effective results from neutron transport simulations of critical benchmark experiments. The DOE NCSP Mission and Vision details the activity in which the French Institut de Radioprotection et de Sûreté Nucléaire (IRSN) leads the study with LANL, ORNL, and LLNL to compare results of various neutron transport codes and nuclear data libraries for ICSBEP benchmarks held in common by the entities. The task statement from the DOE NCSP Five-Year Execution Plan is given as: "The proposal is for IRSN to lead a new intercomparison based on the MORET code with the latest JEFF-3.3 data and ENDF/B-VIII.0 data, when available, using their existing comprehensive selection of 2,714 benchmarks and collate their results together with those from LLNL (COG), LANL (MCNP) and ORNL (SCALE). Due to the large number of benchmarks involved, this effort is envisioned to take three years with an additional year for IRSN to complete a summary report. The benchmark development will be performed independently to minimize modeling errors through discovery and resolution of discrepant results. A summary report will be generated (led by IRSN) to document the results of this study." This report documents results obtained for revisions made to cases involving Intermediate Enriched Uranium (IEU) and a mixture of Pu and Uranium (MIX), with a focus on the changes made to LANL benchmarks modeled with MCNP6 using ENDF/B-VII.1 nuclear data that appeared to have discrepant results when compared with results of other codes. Feedback was used to pinpoint review of particular benchmark input files and to revise them when necessary. This report documents the results of review and revision of specific benchmarks highlighted as possibly discrepant in the comparison study. In addition, there is an effort tied to this work involving collaboration between LANL XCP and NCS Divisions in the development of a shared review/revision procedure and use of a new benchmark repository.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Verification benchmarks for single-phase flow in three-dimensional fractured porous media

Flow in fractured porous media occurs in the earth’s subsurface, in biological tissues, and in man-made materials. Fractures have a dominating influence on flow processes, and the last decade has seen an extensive development of models and numerical methods that explicitly account for their presence. To support these developments, four benchmark cases for single-phase flow in three-dimensional fractured porous media are presented. Furthermore, the cases are specifically designed to test the methods’ capabilities in handling various complexities common to the geometrical structures of fracture networks. Based on an open call for participation, results obtained with 17 numerical methods were collected. This paper presents the underlying mathematical model, an overview of the features of the participating numerical methods, and their performance in solving the benchmark cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

HAM: Hotspot-Aware Manager for Improving Communications with 3D-Stacked Memory

merging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this paper, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. %and an associated hotspot-aware page policy. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51\% and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81\% in average (up to 34.28\%) and power savings of 35.07\% over a standard 3D-stacked memory.

Wang, Xi↗
DOE OSTI2021

A consensus-based ensemble approach to improve transcriptome assembly

Systems-level analyses, such as differential gene expression analysis, co-expression analysis, and metabolic pathway reconstruction, depend on the accuracy of the transcriptome. Multiple tools exist to perform transcriptome assembly from RNAseq data. However, assembling high quality transcriptomes is still not a trivial problem. This is especially the case for non-model organisms where adequate reference genomes are often not available. Different methods produce different transcriptome models and there is no easy way to determine which are more accurate. Furthermore, having alternative-splicing events exacerbates such difficult assembly problems. While benchmarking transcriptome assemblies is critical, this is also not trivial due to the general lack of true reference transcriptomes. In this study, we first provide a pipeline to generate a set of the simulated benchmark transcriptome and corresponding RNAseq data. Using the simulated benchmarking datasets, we compared the performance of various transcriptome assembly approaches including both de novo and genome-guided methods. The results showed that the assembly performance deteriorates significantly when alternative transcripts (isoforms) exist or for genome-guided methods when the reference is not available from the same genome. To improve the transcriptome assembly performance, leveraging the overlapping predictions between different assemblies, we present a new consensus-based ensemble transcriptome assembly approach, ConSemble. Without using a reference genome, ConSemble using four de novo assemblers achieved an accuracy up to twice as high as any de novo assemblers we compared. When a reference genome is available, ConSemble using four genome-guided assemblies removed many incorrectly assembled contigs with minimal impact on correctly assembled contigs, achieving higher precision and accuracy than individual genome-guided methods. Furthermore, ConSemble using de novo assemblers matched or exceeded the best performing genome-guided assemblers even when the transcriptomes included isoforms. We thus demonstrated that the ConSemble consensus strategy both for de novo and genome-guided assemblers can improve transcriptome assembly. The RNAseq simulation pipeline, the benchmark transcriptome datasets, and the script to perform the ConSemble assembly are all freely available from: http://bioinfolab.unl.edu/emlab/consemble/.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Real-time streamflow forecasting: AI vs. Hydrologic insights

In this paper, we propose a set of simple benchmarks for the evaluation of data-based models for real-time streamflow forecasting, such as those developed with sophisticated Artificial Intelligence (AI) algorithms. The benchmarks are also data-based and provide context to judge incremental improvements in the performance metrics from the more complicated approaches. The benchmarks include temporal and spatial persistence, persistence corrected for baseflow and streamflow, as well as river distance weighted runoff obtained from space-time distributed rainfall. In the development of the benchmarks, we use basic hydrologic insights such as flow aggregation by the river network, scale-dependence in basin response, streamflow partitioning into quick flow and baseflow, water travel time, and rainfall averaging by the basin width function. The study uses 140 streamflow gauges in Iowa that cover a range of basin scales between 7 and 37,000 km 2 . The data cover 17 years. This work demonstrates that the proposed benchmarks can provide good performance according to several commonly used metrics. For example, streamflow forecasting at half of the test locations across years achieves a Kling-Gupta Efficiency (KGE) score of 0.6 or higher at one-day ahead lead time, and 20% of cases reach the KGE of 0.8 or higher. The proposed benchmarks are easy to implement and should prove useful for developers of data-based as well as physics-based hydrologic models and real-time data assimilation techniques.

54 ENVIRONMENTAL SCIENCES↗