Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

A programming model performance study using the NAS parallel benchmarks

Harnessing the power of multicore platforms is challenging due to the additional levels of parallelism present. In this paper we use the NAS Parallel Benchmarks to study three programming models, MPI, OpenMP and PGAS to understand their performance and memory usage characteristics on current multicore architectures. To understand these characteristics we use the Integrated Performance Monitoring tool and other ways to measure communication versus computation time, as well as the fraction of the run time spent in OpenMP. The benchmarks are run on two different Cray XT5 systems and an Infiniband cluster. Our results show that in general the three programming models exhibit very similar performance characteristics. In a few cases, OpenMP is significantly faster because it explicitly avoids communication. For these particular cases, we were able to re-write the UPC versions and achieve equal performance to OpenMP. Using OpenMP was also the most advantageous in terms of memory usage. Also we compare performance differences between the two Cray systems, which have quad-core and hex-core processors.We show that at scale the performance is almost always slower on the hex-core system because of increased contention for network resources.

97 MATHEMATICS AND COMPUTING↗

Posiform planting: generating QUBO instances for benchmarking

We are interested in benchmarking both quantum annealing and classical algorithms for minimizing quadratic unconstrained binary optimization (QUBO) problems. Such problems are NP-hard in general, implying that the exact minima of randomly generated instances are hard to find and thus typically unknown. While brute forcing smaller instances is possible, such instances are typically not interesting due to being too easy for both quantum and classical algorithms. In this contribution, we propose a novel method, called posiform planting , for generating random QUBO instances of arbitrary size with known optimal solutions, and use those instances to benchmark the sampling quality of four D-Wave quantum annealers utilizing different interconnection structures (Chimera, Pegasus, and Zephyr hardware graphs) and the simulated annealing algorithm. Posiform planting differs from many existing methods in two key ways. It ensures the uniqueness of the planted optimal solution, thus avoiding groundstate degeneracy, and it enables the generation of QUBOs that are tailored to a given hardware connectivity structure, provided that the connectivity is not too sparse. Posiform planted QUBOs are a type of 2-SAT boolean satisfiability combinatorial optimization problems. Our experiments demonstrate the capability of the D-Wave quantum annealers to sample the optimal planted solution of combinatorial optimization problems with up to 5, 627 qubits.

97 MATHEMATICS AND COMPUTING↗

Validation of time-dependent shift using the pulsed sphere benchmarks

The detailed behavior of neutrons in a rapidly changing time-dependent physical system is a challenging computational physics problem, particularly when using Monte Carlo methods on heterogeneous high-performance computing architectures. A small number of algorithms and code implementations have been shown to be performant for time-independent (fixed source and k-eigenvalue) Monte Carlo, and there are existing simulation tools that successfully solve the time-dependent Monte Carlo problem on smaller computing platforms. To bridge this gap, a time-dependent version of ORNL’s Shift code has been recently developed. Shift’s history-based algorithm on CPUs, and its event-based algorithm on GPUs, have both been observed to scale well to very large numbers of processors, which motivated the extension of this code to solve time-dependent problems. The validation of this new capability requires a comparison with time-dependent neutron experiments. Lawrence Livermore National Laboratory’s (LLNL) pulsed sphere benchmark experiments were simulated in Shift to validate both the time-independent as well as new time-dependent features recently incorporated into Shift. A suite of pulsed-sphere models was simulated using Shift and compared to the available experimental data and simulations with MCNP. Overall results indicate that Shift accurately simulates the pulsed sphere benchmarks, and that the new time-dependent modifications of Shift are working as intended. Validated exascale neutron transport codes are essential for a wide variety of future multiphysics applications.

Palmer, Camille J.↗

Benchmarking a Tunable Quantum Neural Network on Trapped-Ion and Superconducting Hardware

We implement a quantum generalization of a neural network on trapped-ion and IBM superconducting quantum computers to classify MNIST images, a common benchmark in computer vision. The network feedforward involves qubit rotations whose angles depend on the results of measurements in the previous layer. The network is trained via simulation, but inference is performed experimentally on quantum hardware. The classical-to-quantum correspondence is controlled by an interpolation parameter, $a$, which is zero in the classical limit. Increasing $a$ introduces quantum uncertainty into the measurements, which is shown to improve network performance at moderate values of the interpolation parameter. We then focus on particular images that fail to be classified by a classical neural network but are detected correctly in the quantum network. For such borderline cases, we observe strong deviations from the simulated behavior. We attribute this to physical noise, which causes the output to fluctuate between nearby minima of the classification energy landscape. Such strong sensitivity to physical noise is absent for clear images. We further benchmark physical noise by inserting additional single-qubit and two-qubit gate pairs into the neural network circuits. Our work provides a springboard toward more complex quantum neural networks on current devices: while the approach is rooted in standard classical machine learning, scaling up such networks may prove classically non-simulable and could offer a route to near-term quantum advantage.

FOS: Physical sciences↗

Reviews and syntheses: Field data to benchmark the carbon cycle models for tropical forests

Abstract. For more accurate projections of both the global carbon (C) cycle and the changing climate, a critical current need is to improve the representation of tropical forests in Earth system models. Tropical forests exchange more C, energy, and water with the atmosphere than any other class of land ecosystems. Further, tropical-forest C cycling is likely responding to the rapid global warming, intensifying water stress, and increasing atmospheric CO2 levels. Projections of the future C balance of the tropics vary widely among global models. A current effort of the modeling community, the ILAMB (International Land Model Benchmarking) project, is to compile robust observations that can be used to improve the accuracy and realism of the land models for all major biomes. Our goal with this paper is to identify field observations of tropical-forest ecosystem C stocks and fluxes, and of their long-term trends and climatic and CO2 sensitivities, that can serve this effort. We propose criteria for reference-level field data from this biome and present a set of documented examples from old-growth lowland tropical forests. We offer these as a starting point towards the goal of a regularly updated consensus set of benchmark field observations of C cycling in tropical forests.

54 ENVIRONMENTAL SCIENCES↗

Panta Rhei benchmark dataset: socio-hydrological data of paired events of floods and droughts

As the adverse impacts of hydrological extremes increase in many regions of the world, a better understanding of the drivers of changes in risk and impacts is essential for effective flood and drought risk management and climate adaptation. However, there is currently a lack of comprehensive, empirical data about the processes, interactions, and feedbacks in complex human–water systems leading to flood and drought impacts. Here we present a benchmark dataset containing socio-hydrological data of paired events, i.e. two floods or two droughts that occurred in the same area. The 45 paired events occurred in 42 different study areas and cover a wide range of socio-economic and hydro-climatic conditions. The dataset is unique in covering both floods and droughts, in the number of cases assessed and in the quantity of socio-hydrological data. The benchmark dataset comprises (1) detailed review-style reports about the events and key processes between the two events of a pair; (2) the key data table containing variables that assess the indicators which characterize management shortcomings, hazard, exposure, vulnerability, and impacts of all events; and (3) a table of the indicators of change that indicate the differences between the first and second event of a pair. The advantages of the dataset are that it enables comparative analyses across all the paired events based on the indicators of change and allows for detailed context- and location-specific assessments based on the extensive data and reports of the individual study areas. The dataset can be used by the scientific community for exploratory data analyses, e.g. focused on causal links between risk management; changes in hazard, exposure and vulnerability; and flood or drought impacts. The data can also be used for the development, calibration, and validation of socio-hydrological models. The dataset is available to the public through the GFZ Data Services (Kreibich et al., 2023, https://doi.org/10.5880/GFZ.4.4.2023.001).

54 ENVIRONMENTAL SCIENCES↗

The AWAKEN wind farm benchmark, Part 2: Modeling results

Accurately modeling wind farm performance in complex atmospheric flows remains a challenge. This paper presents the modeling results of the American WAKE experimeNt (AWAKEN) wind farm benchmark, a collaborative effort involving 16 research groups from academia and industry within the International Energy Agency Wind Technology Collaboration Programme Task 57. The study evaluates a diverse suite of simulation tools, ranging from fast-running engineering wake models to high-fidelity large-eddy simulations, against a diurnal case study observed during the AWAKEN campaign. The benchmark utilized a three-phase structure to progressively assess model performance as observational data availability increased. Initial blind predictions showed that higher-fidelity models did not uniformly outperform simpler simulation tools. A distinct spatial bias was observed where models struggled to resolve the interplay between a low-level jet, wakes, and terrain-induced flow acceleration. In subsequent phases, leveraging additional measurements for model improvement led to a reduction in mean absolute error across the model ensemble; however, this effect was most pronounced in engineering wake models, where targeted calibration reduced error by up to 40~\%. Overall, the study demonstrates that inflow characterization remains a primary prerequisite for accuracy, particularly for models relying on coarse forcing datasets. While the limited ability to resolve local terrain-flow interactions under single-day conditions represent a recognized constraint, the overall findings on wake modeling and real-world validation still provide valuable guidance for model application and for mitigating this limitation.

Bodini, Nicola↗

Q1-2024 Solar Cost Benchmarks

Each year, the U.S. Department of Energy’s (DOE) Solar Energy Technologies Office (SETO) and its national laboratory partners develop cost benchmarks for U.S. solar photovoltaic (PV) systems. These benchmarks track progress toward reducing solar costs and guide R&D priorities. Unlike typical studies that report only $/W, SETO uses intrinsic units (e.g., $/m² for mounting structures) to better capture how technology improvements such as module efficiency would impact system costs. This allows flexible modeling where inputs can vary significantly to assess cost sensitivity. Costs are reported in two ways: Minimum Sustainable Price (MSP): Long term, financially viable price under stable market conditions. Modeled Market Price (MMP): Actual market price, influenced by short term distortions such as tariffs or subsidies. Three national labs collect cost data from industry stakeholders, ensuring no duplication in outreach to stakeholders. Data reflects real transactions (primarily from Q1) and is weighted based on the number of sources per cost element. The PV System Cost Model (PVSCM) divides total installed system cost into eight categories: 1. Module (PV) 2. Inverter 3. Energy Storage System (ESS) 4. Structural BOS (SBOS) 5. Electrical BOS (EBOS) 6. Fieldwork 7. Office work 8. Other (developer/EPC costs) The first five are hardware costs, while the last three are soft costs. Each category includes fixed and variable cost components, where “size” depends on context (e.g., manufacturing capacity for modules vs. system capacity for installation costs). Variable costs are expressed using appropriate intrinsic units. The model reflects the owner’s upfront overnight capital cost, excluding tax credits. Tariffs and subsidies are treated as temporary market distortions affecting MMP but not MSP. PVSCM is implemented in Excel, where cost elements are aggregated into total system cost. Additional sheets handle unit conversions and operation & maintenance (O&M), with O&M costs levelized over the system’s lifetime.

14 SOLAR ENERGY↗

SILENE Benchmark Critical Experiments for Criticality Accident Alarm Systems

In October 2010 a series of benchmark experiments was conducted at the Commissariat a Energie Atomique et aux Energies Alternatives (CEA) Valduc SILENE [1] facility. These experiments were a joint effort between the US Department of Energy (DOE) and the French CEA. The purpose of these experiments was to create three benchmarks for the verification and validation of radiation transport codes and evaluated nuclear data used in the analysis of criticality accident alarm systems (CAASs). This presentation will discuss the geometric configuration of these experiments and the quantities that were measured and will present some preliminary comparisons between the measured data and calculations. This series consisted of three single-pulsed experiments with the SILENE reactor. During the first experiment the reactor was bare (unshielded), but during the second and third experiments it was shielded by lead and polyethylene, respectively. During each experiment several neutron activation foils and thermoluminescent dosimeters (TLDs) were placed around the reactor, and some of these detectors were themselves shielded from the reactor by high-density magnetite and barite concrete, standard concrete, and/or BoroBond. All the concrete was provided by CEA Saclay, and the BoroBond was provided by Y-12 National Security Complex. Figure 1 is a picture of the SILENE reactor cell configured for pulse 1. Also included in these experiments were measurements of the neutron and photon spectra with two BICRON BC-501A liquid scintillators. These two detectors were provided and operated by CEA Valduc. They were set up just outside the SILENE reactor cell with additional lead shielding to prevent the detectors from being saturated. The final detectors involved in the experiments were two different types of CAAS detectors. The Babcock International Group provided three CIDAS CAAS detectors, which measured photon dose and dose rate with a Geiger-Mueller tube. CIDAS detectors are currently in use at Y-12 in the newly constructed Highly Enriched Uranium Materials Facility. The second CAAS detector used a 6 LiF TLD to absorb neutrons and a silicon detector to count the charge particles released by these absorption events. Lawrence Livermore National Laboratory provided four of these detectors, which had formerly been used at the Rocky Flats facility in the United States.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Graphite and Beryllium Reflector Critical Assemblies of UO 2 (Benchmark Experiments 2 and 3)

A series of experiments was carried out in 1962-65 at the Oak Ridge National Laboratory Critical Experiments Facility (ORCEF) for use in space reactor research programs. A core containing 93.2 wt% enriched UO 2 fuel rods was used in these experiments. The first part of the experimental series consisted of 252 tightly packed fuel rods (1.27-cm triangular pitch) with graphite reflectors, the second part used 252 graphite-reflected fuel rods organized in a 1.506-cm triangular-pitch array, and the final part of the experimental series consisted of 253 beryllium-reflected fuel rods in a 1.506-cm-triangular-pitch configuration and in a 7-tube-cluster configuration. Fission rate distribution and cadmium ratio measurements were taken for all three parts of the experimental series. Reactivity coefficient measurements were taken for various materials placed in the beryllium reflected core. All three experiments in the series have been evaluated for inclusion in the International Reactor Physics Experiment Evaluation Project (IRPhEP) and the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbooks. The evaluation of the first experiment in the series was discussed at the 2011 ANS Winter meeting. The evaluations of the second and third experiments are discussed below. These experiments are of interest as benchmarks because they support the validation of compact reactor designs with similar characteristics to the design parameters for a space nuclear fission surface power system.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Thermal Performance Benchmarking (Presentation)

This project will benchmark the thermal characteristics of automotive power electronics and electric motor thermal management systems. Recent vehicle systems will be benchmarked to establish baseline metrics, evaluate advantages and disadvantages of different thermal management systems, and identify areas of improvement to advance the state-of-the-art.

30 DIRECT ENERGY CONVERSION↗

Thermal Performance Benchmarking; NREL (National Renewable Energy Laboratory)

This project proposes to seek out the SOA power electronics and motor technologies to thermally benchmark their performance. The benchmarking will focus on the thermal aspects of the system. System metrics including the junction-to-coolant thermal resistance and the parasitic power consumption (i.e., coolant flow rates and pressure drop performance) of the heat exchanger will be measured. The type of heat exchanger (i.e., channel flow, brazed, folded-fin) and any enhancement features (i.e., enhanced surfaces) will be identified and evaluated to understand their effect on performance. Additionally, the thermal resistance/conductivity of the power module’s passive stack and motor’s laminations and copper winding bundles will also be measured. The research conducted will allow insight into the various cooling strategies to understand which heat exchangers are most effective in terms of thermal performance and efficiency. Modeling analysis and fluid-flow visualization may also be carried out to better understand the heat transfer and fluid dynamics of the systems.

30 DIRECT ENERGY CONVERSION↗

Benchmark models and experimental data for a U(20) polyethylene-moderated critical system

This work involves the analysis of recent experiments performed on the Aerojet General Nucleonics (AGN)-201M (AGN) polyethylene-moderated research reactor at the University of New Mexico (UNM). The experiments include 36 delayed critical (DC) configurations and 11 positive-period and rod-drop measurements (transient sequences). The Even Parity Neutron Transport (EVENT) radiation transport code was chosen to analyze these steady state and time-dependent experimental configurations. The UNM AGN specifications provided in a benchmark calculation report (2007) were used to initiate AGN EVENT model development and to test the EVENT AGN calculation methodology. The results of the EVENT DC experimental analyses compared well with the experimental data; the average AGN EVENT calculation bias in the k eff is –0.0048% for the Legrendre Flux Expansion Order of 11 (P 11 ) cases and +0.0119% for the P 13 cases. The EVENT transient analysis also compared well with the AGN experimental data with respect to predicting the reactor period and control rod worth values. This paper discusses the benchmark models used, the recent experimental configurations, and the EVENT experimental analysis.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

BENCHMARK ACCELERATED AGING OF HARVESTED HYPALON/EPR AND CSPE/XLPE POWER AND I&C CABLE IN NUCLEAR POWER PLANTS

As part of the Light Water Reactor and Sustainability (LWRS) program in the U.S. Department of Energy (DOE) Office of Nuclear Energy, material aging and degradation research is currently geared to support the long-term operation of existing nuclear power plants (NPPs) as they move beyond their initial 40 year licenses. The goal of this research is to provide information so that NPPs can develop aging management programs (AMPs) to address replacement and monitoring needs as they look to operate for 20 years, and in some cases 40 years, beyond their initial operating lifetimes. For cable insulation and jacket materials that support instrument, control, and safety systems, accelerated aging data are needed to determine priorities in cable aging management programs. Before accelerated thermal and radiation aging of harvested, representative cable insulation and jacket materials, the benchmark performance of a new test capability at Oak Ridge National Laboratory (ORNL) was evaluated for temperatures between 70 and 135 C, dose rates between 100 and 500 Gy/h, and accumulated doses up to 20 kGy, Samples that were characterized and are representative of current materials in use were harvested from the Callaway NPP near Fulton, Missouri, and the San Onofre NPP north of San Diego, California. From the Callaway NPP, a multiconductor control rod cable manufactured by Boston Insulated Wire (BIW), with a Hypalon/ chorolosulfonated polyethylene (CSPE) jacket and ethylene-propylene rubber (EPR) insulation, was harvested from the auxiliary space during a planned outage in 2013. This cable was placed into service when the plant was started in 1984. From the San Onofre NPP, a Rockbestos Firewall III (FRIII) cable with a Hypalon/ CSPE jacket with cross-linked polyethylene (XLPE) insulation was harvested from an on-site, climate-controlled storage area. This conductor, which was never placed into service, was procured around 2007 in anticipation of future operation that did not occur. Benchmark aging for both jacket and insulation material was carried out in air at a temperature of 125 C or in uniform 140 Gy/h gamma irradiation over a period of 60 days. Their mechanical properties over the course of their exposures were compared with reference data from comparable cable jacket/insulation compositions and aging conditions. For both accelerated thermal and radiation aging, it was observed that the mechanical properties for the Callaway BIW control rod cable were consistent with those previously measured. However, for the San Onofre Rockbestos FRIII, there was an observable functional difference for accelerated thermal aging at 125 C. Details on possible sources for this difference and plans for resolving each source are given in this paper.

Duckworth, Robert C↗

Semi-Analytical Benchmarks for MCNP6 [Slides]

This presentation begins with an Introduction and some background on semi-analytical benchmarks for MCNP6. Discussion topics include: verification & validation, MCNP history of V&V, semi-analytic benchmarks, numerical Results, comparison of semi-analytic to MCNP, and tips for proper comparison. This presentation ends with conclusions & future Work.

97 MATHEMATICS AND COMPUTING↗

Demonstration of MAMMOTH Strongly-Coupled Multiphysics Simulation with the Godiva Benchmark Problem

This paper provides a detailed description of the Godiva benchmark problem suitable for multiphysics code verification. Multigroup transport weak forms for general curvilinear coordinates are derived using SAAF (self-adjoint angular flux) formulation. A new scheme based on spherical harmonics expansion for evaluating the angular derivative terms using the SN method are presented and tested. A one-dimensional spherical coordinate system is used by MAMMOTH for modeling the Godiva benchmark. Strongly-coupled multiphysics simulation mode of MAMMOTH is exercised and numerical results are given.

97 MATHEMATICS AND COMPUTING↗

Algebraic Multigrid Benchmark

AMG is a parallel algebraic multigrid solver for linear systems arising from problems on unstructured grids. It has been derived directly from the BoomerAMG solver in the hypre library, a large linear solvers library that is being developed in the Center for Applied Scientific Computing (CASC) at LLNL and is very similar to the AMG2013 benchmark with additional optimizations. The driver provided in the benchmark can build various test problems. The default problem is a Laplace type problem with a 27-point stencil, which can be scaled up and is designed to solve a very large problem. A second problem simulates a time dependent problem, in which successively various smnllcr systems are solved.

Yang, Ulrike↗

U.S. Solar Photovoltaic System Cost Benchmark: Q1 2017

NREL has been modeling U.S. photovoltaic (PV) system costs since 2009. This year, our report benchmarks costs of U.S. solar PV for residential, commercial, and utility-scale systems built in the first quarter of 2017 (Q1 2017). Costs are represented from the perspective of the developer/installer, thus all hardware costs represent the price at which components are purchased by the developer/installer, not accounting for preexisting supply agreements or other contracts. Importantly, the benchmark this year (2017) also represents the sales price paid to the installer; therefore, it includes profit in the cost of the hardware, along with the profit the installer/developer receives, as a separate cost category. However, it does not include any additional net profit, such as a developer fee or price gross-up, which are common in the marketplace. We adopt this approach owing to the wide variation in developer profits in all three sectors, where project pricing is highly dependent on region and project specifics such as local retail electricity rate structures, local rebate and incentive structures, competitive environment, and overall project or deal structures.

14 SOLAR ENERGY↗