Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “application benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Adding Fault Tolerance to NPB Benchmarks Using ULFM

In the world of high-performance computing, fault tolerance and application resilience are becoming some of the primary concerns because of increasing hardware failures and memory corruptions. While the research community has been investigating various options, from system-level solutions to application-level solutions, standards such as the Message Passing Interface (MPI) are also starting to include such capabilities. The current proposal for MPI fault tolerant is centered around the User-Level Failure Mitigation (ULFM) concept, which provides means for fault detection and recovery of the MPI layer. This approach does not address application-level recovery, which is currently left to application developers. In this work, we present a mod- ification of some of the benchmarks of the NAS parallel benchmark (NPB) to include support of the ULFM capabilities as well as application-level strategies and mechanisms for application-level failure recovery. As such, we present: (i) an application-level library to checkpoint and restore data, (ii) extensions of NPB benchmarks for fault tolerance based on different strategies, (iii) a fault injection tool, and (iv) some preliminary results that show the impact of such fault tolerant strategies on the application execution.

Parchman, Zachary W↗
DOE OSTI2016

Uncertainty Quantification by Monte Carlo Analysis using CFD Simulations for GEMIX Benchmark Activities

Uncertainty quantification (UQ) methods can be applied to computational fluid dynamics (CFD) simulations for determining the uncertainty bands of the simulations results. In order to determine the applicability of these different methods, simple benchmark activities have been conducted by various group for comparison. The GEMIX (Generic Mixing experiment) test facility was used to develop benchmark data for this purpose by the Paul Scherrer Institute (PSI). The facility involves the mixing of two fluids that are initially separated. One of the two fluids is water with sugar dissolved in it to create a fluid with similar fluid properties to water, except for density. The other fluid was distilled water to reduce the uncertainty in the fluid's properties. During the experiments, velocity fields and concentration measurements were acquired at several planes of interest. The velocity fields were acquired using particle image velocimetry (PIV) and concentrations of each fluid were acquired using laser induced fluorescence (LIF). The inlet velocity profiles were acquired using laser Doppler velocimetry (LDV). The activity provided the opportunity to vary turbulence models, meshing methodologies, and numerical schemes for creation of uncertainty estimates. The present study investigates the usage of a simple Monte Carlo (MC) technique for varying the inlet boundary conditions of each inlet. The velocity profiles and densities of each inlet were specifically varied based on experiment measurement uncertainty. The intent was to isolate the uncertainty due to inlet quantities and not related to meshing, numerical scheme, or turbulence models. The comparisons of experimental and simulation results were found to be reasonably accurate. Though deficiencies were identified in the modeling approach to be corrected in future works by the authors. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1994

Performance Evaluation of Three Distributed Computing Environments for Scientific Applications

We present performance results for three distributed computing environments using the three simulated CFD applications in the NAS Parallel Benchmark suite. These environments are the DCF cluster, the LACE cluster, and an Intel iPSC/860 machine. The DCF is a prototypic cluster of loosely coupled SGI R3000 machines connected by Ethernet. The LACE cluster is a tightly coupled cluster of 32 IBM RS6000/560 machines connected by Ethernet as well as by either FDDI or an IBM Allnode switch. Results of several parallel algorithms for the three simulated applications are presented and analyzed based on the interplay between the communication requirements of an algorithm and the characteristics of the communication network of a distributed system.

Fatoohi, Rod↗
DOE OSTI2018

Sensitivity/uncertainty analyses comparing LR-0 reactor experiments containing FLiBe salt with models for molten-salt-cooled and molten-salt-fueled reactors

Here, critical experiments using an insertion zone with FLiBe salt performed at the LR-0 reactor at Research Centre Rež (RC Rež) have been compared to application models for solid-fueled, fluoride-salt-cooled high-temperature reactor (FHR) and liquid-fueled molten salt reactor (MSR) concepts using sensitivity and uncertainty (S/U) analysis techniques. These experiments support FHR and MSR advanced reactor concepts by informing on neutron spectral effects and nuclear data uncertainties related to fluoride salts. The FLiBe salt in the LR-0 experiments is from the Oak Ridge National Laboratory Molten Salt Reactor Experiment and is enriched to greater than 99.99% 7 Li. This work is part of a broader collaboration between the United States and the Czech Republic on MSR and FHR technology. Results from the S/U analyses for FHR and MSR models indicated the most significant potential source of eigenvalue bias due to nuclear data within the FLiBe salt is radiative capture in 7 Li. Other smaller but potentially significant contributions come from 19 F, 6 Li, and other 7 Li reactions. Similarity comparisons of the RC Rež LR-0 experiments and FHR and MSR application models indicate that the LR-0 FLiBe experiments could be useful as candidate benchmarks for low-enriched uranium–fueled application models. However, sensitivity differences observed included both spectral effects, due to the LR-0 reactor being moderated by water instead of graphite and fluoride salt, and sensitivity magnitude effects driven by the amount of FLiBe inserted and its location. New experiments with refined designs to increase the volume and importance of salt in the system could provide improved data. Results also demonstrate that salt systems using thorium and/or 233 U fuel will require additional experiments with relevant driver fuels. Increased contributions from the graphite moderator and 19 F and covariance between reactions such as 233 U fission and 233 U radiative capture indicate higher uncertainty contributions from the salt and significant differences in contributions from the fuel species due to underlying uncertainties in their nuclear data.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Verification of the Uniformly-Ordered Binary Decision Algorithm in Correlated-Benchmark Whisper Calculations

Whisper is a nuclear criticality safety code package that aids analysts in validation exercises by computing upper subcritical limits (USL) for applications of interest. To obtain statistically meaningful, significant, and conservative USLs, the analyst must ensure that Whisper selects a sufficient number of benchmarks that are neutronically similar to the application. Many of the available benchmarks are correlated but are currently treated as independent, leading to an artificially small sample size, as their individual information contributions will be overestimated. To aid the analyst in obtaining a sufficient sample size, prior work [2] demonstrated application of the Uniformly-Ordered Binary Decision (UOBD) algorithm in adjusting benchmark weights to account for benchmark correlations. This work provides verification of the Whisper implementation and considers the impact of updated benchmark correlations compared to those available previously. We demonstrate that the UOBD algorithm performs as expected with an analytic example. With HEU-SOL-THERM-001 cases 1 through 10 as the applications, we compare the USLs computed with benchmark correlations available in the Whisper 1.1 release only to those computed with additional benchmark correlations from DICE 2023 and demonstrate substantive differences.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Chimbuko: A Workflow-Level Scalable Performance Trace Analysis Tool

ABSTRACT Due to the sheer volume of data it is typically impractical to analyze the detailed performance of an HPC application running at-scale. While conventional small-scale benchmarking and scaling studies are often sufficient for simple applications, many modern workflow-based applications couple multiple elements with competing resource demands and complex inter-communication patterns for which performance cannot easily be studied in isolation and at small scale. This work discusses Chimbuko, a performance analysis framework that provides real-time, in situ anomaly detection. By focusing specifically on performance anomalies and their origin (aka provenance), data volumes are dramatically reduced without losing necessary details. To the best of our knowledge, Chimbuko is the first online, distributed, and scalable workflow-level performance trace analysis framework. We demonstrate the tool's usefulness on Oak Ridge National Laboratory's Summit system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Availability of Shielding Benchmark Experiment Data in the ICSBEP Handbook

The International Criticality Safety Benchmark Evaluation Projects (ICSBEP) has over two decades of experience under the auspices of the Organisation for Economic Co-operation and Development (OECD) Nuclear Energy Agency (NEA) to identify, evaluate, and compile a comprehensive data set of criticality safety benchmark data [1]. The activities of the ICSBEP have led to the continued success of the International Handbook of Evaluated Criticality Safety Benchmark Experiments (ICSBEP Handbook) [2]. The contents of the ICSBEP Handbook include benchmark specifications derived from nuclear facilities around the world, which have been utilized to support criticality safety efforts and also integral testing of nuclear data [3]. The 2020 edition of the ICSBEP Handbook will include data from 582 evaluations containing benchmark specifications for 5,053 critical, subcritical, or near-critical configurations. There are seven criticality-alarm-placement/shielding evaluations containing a total of 45 benchmark configurations, and ten fundamental physics evaluations containing a total of 237 measurements relevant to criticality safety applications. The purpose of this paper is to summarize those benchmarks relative to validation needs in shielding applications currently found within the ICSBEP Handbook. The current and future plans of shielding benchmarks with regards to the ICSBEP will also be discussed.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

Chimbuko: A Workflow-Level Scalable Performance Trace Analysis Tool

Due to the sheer volume of data it is typically impractical to analyze the detailed performance of an HPC application running at-scale. While conventional small-scale benchmarking and scaling studies are often sufficient for simple applications, many modern workflow-based applications couple multiple elements with competing resource demands and complex inter-communication patterns for which performance cannot easily be studied in isolation and at small scale. This work discusses Chimbuko, a performance analysis framework that provides real-time, in situ anomaly detection. By focusing specifically on performance anomalies and their origin (aka provenance), data volumes are dramatically reduced without losing necessary details. To the best of our knowledge, Chimbuko is the first online, distributed, and scalable workflow-level performance trace analysis framework. We demonstrate the tool's usefulness on Oak Ridge National Laboratory's Summit system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Model Assumptions and Data Characteristics: Impacts on Domain Adaptation in Building Segmentation

Studies on domain adaptation (DA) for remote sensing (RS) imagery analysis lack consistency in selection and description of evaluation scenarios. Without properly characterizing datasets, model assumptions, and evaluation scenarios, it is difficult to objectively compare DA methods and reach conclusions about their suitability across different applications. With this motivation, this work seeks to empirically assess to which extent the interaction between data characteristics and model assumptions influences the effectiveness of DA methods. Using the widely explored task of building footprint segmentation as a case study, we perform a large-scale study across over 200 DA scenarios that include variations across view angles, areas observed, and sensors used for data acquisition. Rather than adopting different model architectures or optimization criteria, we contrast the performances of two DA methods based on adversarial learning that differ only in their assumptions about source and target domains. Informed by metadata and data characteristics unveiled using traditional computer vision (CV) techniques as well as pretrained deep models, we provide a detailed meta-analysis of experiments highlighting the importance of accurately considering data assumptions for DA in RS segmentation tasks. As demonstrated by a “cherry-picking” exercise, different claims regarding which model is best could be made by selecting different subsets of evaluation scenarios. While well-calibrated assumptions can be beneficial, mismatching assumptions can lead to negative biases in DA applications. Furthermore, this study intends to motivate the community toward more consistent evaluation protocols while providing recommendations and insights toward creating novel benchmark datasets, documenting data characteristics, application-specific knowledge, and model assumptions.

42 ENGINEERING↗
DOE OSTI2021

$\mathrm{PPT}$-Multicore: performance prediction of Open$\mathrm{MP}$ applications using reuse profiles and analytical modeling

In this report we present PPT-Multicore, an analytical model embedded in the Performance Prediction Toolkit (PPT) to predict parallel applications’ performance running on a multicore processor. PPT-Multicore builds upon our previous work towards a multicore cache model. We extract LLVM basic block labeled memory trace using an architecture-independent LLVM-based instrumentation tool only once in an application’s lifetime. The model uses the memory trace and other parameters from an instrumented sequentially executed binary. We use probabilistic and computationally efficient reuse profiles to predict the cache hit rates and runtimes of OpenMP programs’ parallel sections. We model Intel’s Broadwell, Haswell, and AMD’s Zen2 architectures and validate our framework using different applications from PolyBench and PARSEC benchmark suites. The results show that PPT-Multicore can predict cache hit rates with an overall average error rate of 1.23% while predicting the runtime with an error rate of 9.08%.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2010

Dynamic Load Balancing on Single- and Multi-GPU Systems

The computational power provided by many-core graphics processing units (GPUs) has been exploited in many applications. The programming techniques supported and employed on these GPUs are not sufficient to address problems exhibiting irregular, and unbalanced workload. The problem is exacerbated when trying to effectively exploit multiple GPUs, which are commonly available in many modern systems. In this paper, we propose a task-based dynamic load-balancing solution for single- and multi-GPU systems. The solution allows load balancing at a finer granularity than what is supported in existing APIs such as NVIDIA’s CUDA. We evaluate our approach using both micro-benchmarks and a molecular dynamics application that exhibits significant load imbalance. Experimental results with a single-GPU configuration show that our fine-grained task solution can utilize the hardware more efficiently than the CUDA scheduler for unbalanced workload. On multi-GPU systems, our solution achieves near-linear speedup, load balance, and significant performance improvement over techniques based on standard CUDA APIs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Uniformly Ordered Binary Decision Algorithm for Benchmark Experiment Correlations in Whisper Validation

When performing a validation exercise for determining the upper subcritical limit of a nuclear criticality safety application, an analyst should select and perform a statistical analysis on a population of benchmark experiments that are neutronically similar to the application. The size of this population should be sufficiently large such that the statistical analysis has a high degree of confidence that the bias plus bias uncertainty (calculational margin) has been accurately quantified. A complication arises because many benchmark experiments share common components, leading to correlations in their measured effective multiplication factors. Correlations between benchmark experiments within the population reduces its predictive power. This motivates the need for methods that consider benchmark experiment correlations and ensure adequate statistical significance of results. The Whisper code is a statistical analysis pack- age that incorporates nuclear data sensitivity coefficients from MCNP to assess benchmark experiment similarity and then performs an extreme-value analysis to estimate the bias plus bias uncertainty. The original methodology in Whisper does not consider the effect of benchmark experiment correlations when making this estimation, and this summary proposes the uniformly ordered binary decision algorithm to address this shortcoming. The original methodology in Whisper computes similarity coefficients ck for an application compared to all benchmark experiments in its library and develops weighting factors for a selected population proportional to the ck values. The methodology can be interpreted as statistically emulating a validation exercise for a particular application where the weighting factors may be viewed as the likelihood that an analyst would include a particular benchmark experiment within the population. The effective sample size of the population is the expected or mean number of benchmark experiments in the population. The uniformly ordered binary decision algorithm identifies clusters of correlated benchmark experiments within the population and then computes adjusted weighting factors based on the magnitude of the correlation coefficients within the cluster to compute a reduced effective sample size accounting for the lower information content because of correlations. Benchmark experiments within the cluster are ordered randomly with equal probability and probabilistic decisions are made as to whether a benchmark. experiment within the cluster should treated as redundant with a previous one; if two redundant benchmark experiments are included, then the conservative worst case bias plus bias uncertainty is used and the pair is counted as a single benchmark experiment in the population. Results are provided for HEU solutions in a research version of the Whisper software using benchmark experiment correlations provided by DICE, the Database for the International Criticality Safety Benchmark Evaluation Project (ICSBEP). These show that there can be a significant increase in the bias plus bias uncertainty because the effective sample size is reduced, and therefore the algorithm, needing to meet sample size requirements, expands the benchmark experiment population by accepting less similar benchmark experiments that would have otherwise not been included.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRSDate not supplied

Optimization Algorithms as Quantum Performance Benchmarks

Combinatorial optimization is anticipated to be one of the primary use cases for quantum computation in the coming years. The Quantum Approximate Optimization Algorithm (QAOA) and Quantum Annealing (QA) have the potential to demonstrate significant run-time performance benefits over current state-of-the-art solutions. Using existing methods for characterizing classical optimization algorithms, we analyze solution quality obtained by solving Max-Cut problems using a quantum annealing device and gate-model quantum simulators and devices. This is used to guide the development of an advanced benchmarking framework for quantum computers designed to evaluate the trade-off between run-time execution performance and the solution quality for iterative hybrid quantum-classical applications. The framework generates performance profiles through effective visualizations that show performance progression as a function of time for various problem sizes and illustrates algorithm limitations uncovered by the benchmarking approach. The framework is an enhancement to the existing open-source QED-C Application-Oriented Benchmark suite and can connect to the open-source analysis libraries. The suite can be executed on various quantum simulators and quantum hardware systems.

benchmarking↗
DOE OSTI2022

Evaluation of accuracy and convergence of numerical coupling approaches for poroelasticity benchmark problems

Accurate modeling of subsurface flow and transport processes is vital as the prevalence of subsurface activities such as carbon sequestration, geothermal recovery, and nuclear waste disposal increases. Computational modeling of these problems leverages poroelasticity theory, which describes coupled fluid flow and mechanical deformation. Although fully coupled monolithic schemes are accurate for coupled problems, they can demand significant computational resources for large problems. In this work, a fixed stress scheme is implemented into the Sandia Sierra Multiphysics toolkit. Two implementation methods, along with the fully coupled method, are verified with one-dimensional (1D) Terzaghi, 2D Mandel, and 3D Cryer sphere benchmark problems. The impact of a range of material parameters and convergence tolerances on numerical accuracy and efficiency was evaluated. Overall the fixed stress schemes achieved acceptable numerical accuracy and efficiency compared to the fully coupled scheme. However, the accuracy of the fixed stress scheme tends to decrease with low permeable cases, requiring the finer tolerance to achieve a desired numerical accuracy. For the fully coupled scheme, high numerical accuracy was observed in most of cases except a low permeability case where an order of magnitude finer tolerance was required for accurate results. Finally, a two-layer Terzaghi problem and an injection–production well system were used to demonstrate the applicability of findings from the benchmark problems for more realistic conditions over a range of permeability. Simulation results suggest that the fixed stress scheme provides accurate solutions for all cases considered with the proper adjustment of the tolerance. Furthermore, this work clearly demonstrates the robustness of the fixed stress scheme for coupled poroelastic problems, while a cautious selection of numerical tolerance may be required under certain conditions with low permeable materials.

58 GEOSCIENCES↗
DOE OSTI2021

Random circuit block-encoded matrix and a proposal of quantum LINPACK benchmark

The LINPACK benchmark reports the performance of a computer for solving a system of linear equations with dense random matrices. Although this task was not designed with a real application directly in mind, the LINPACK benchmark has been used to define the list of TOP500 supercomputers since the debut of the list in 1993. We propose that a similar benchmark, called the quantum LINPACK benchmark, could be used to measure the whole machine performance of quantum computers. The success of the quantum LINPACK benchmark should be viewed as the minimal requirement for a quantum computer to perform a useful task of solving linear algebra problems, such as linear systems of equations. We propose an input model called the Random Circuit Block-Encoded Matrix (RACBEM), which is a proper generalization of a dense random matrix in the quantum setting. The RACBEM model is efficient to be implemented on a quantum computer and can be designed to optimally adapt to any given quantum architecture, with relying on a black-box quantum compiler. Besides solving linear systems, the RACBEM model can be used to perform a variety of linear algebra tasks relevant to many physical applications, such as computing spectral measures, time series generated by a Hamiltonian simulation, and thermal averages of the energy. We implement these linear algebra operations on IBM Q quantum devices as well as quantum virtual machines, and demonstrate their performance in solving scientific computing problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2018

Performance Model of MapReduce Iterative Applications for Hybrid Cloud Bursting

Hybrid cloud bursting (i.e., leasing temporary off-premise cloud resources to boost the overall capacity during peak utilization) can be a cost-effective way to deal with the increasing complexity of big data analytics, especially for iterative applications. However, the low throughput, high latency network link between the on-premise and off-premise resources (“weak link”) makes maintaining scalability difficult. While several data locality techniques have been designed for big data bursting on hybrid clouds, their effectiveness is difficult to estimate in advance. Yet such estimations are critical, because they help users decide whether the extra pay-as-you-go cost incurred by using the off-premise resources justifies the runtime speed-up. To this end, the current paper presents a performance model and methodology to estimate the runtime of iterative MapReduce applications in a hybrid cloud-bursting scenario. The paper focuses on the overhead incurred by the weak link at fine granularity, for both the map and the reduce phases. This approach enables high estimation accuracy, as demonstrated by extensive experiments at scale using a mix of real-world iterative MapReduce applications from standard big data benchmarking suites that cover a broad spectrum of data patterns. As a result, not only are the produced estimations accurate in absolute terms compared with experimental results, but they are also up to an order of magnitude more accurate than applying state-of-art estimation approaches originally designed for single-site MapReduce deployments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2011

Application of the INSTANT-HPS PN Transport Code to the C5G7 Benchmark Problem

INSTANT is the INL's next generation neutron transport solver to support high-fidelity multi-physics reactor simulation INSTANT is in continuous development to extend its capability Code is designed to take full advantage of middle to large cluster (10-1000 processors) Code is designed to focus on method adaptation while also mesh adaptation will be possible. It utilizes the most modern computing techniques to generate a neutronics tool of full-core transport calculations for reactor analysis and design. It can perform calculations on unstructured 2D/3D triangular, hexagonal and Cartesian geometries. Calculations can be easily extended to more geometries because of the independent mesh framework coded with the model Fortran. This code has a multigroup solver with thermal rebalance and Chebyshev acceleration. It employs second-order PN and Hybrid Finite Element method (PNHFEM) discretization scheme. Three different in-group solvers - preconditioned Conjugate Gradient (CG) method, preconditioned Generalized Minimal Residual Method (GMRES) and Red-Black iteration - have been implemented and parallelized with the spatial domain decomposition in the code. The input is managed with extensible markup language (XML) format. 3D variables including the flux distributions are outputted into VTK files, which can be visualized by tools such as VisIt and ParaView. An extension of the code named INSTANTHPS provides the capability to perform 3D heterogeneous transport calculations within fuel pins. C5G7 is an OECD/NEA benchmark problem created to test the ability of modern deterministic transport methods and codes to treat reactor core problems without spatial homogenization. This benchmark problem had been widely analyzed with various code packages. In this transaction, results of the applying the INSTANT-HPS code to the C5G7 problem are summarized.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗