Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Evaluation of HEU-Beryllium Benchmark Experiments to Improve Computational Analysis of Space Reactors

An assessment was previously performed to evaluate modeling capabilities and quantify preliminary biases and uncertainties associated with the modeling methods and data utilized in designing a nuclear reactor such as a beryllium-reflected, highly-enriched-uranium (HEU)-O 2 fission surface power (FSP) system for space nuclear power. The conclusion of the previous study was that current capabilities could preclude the necessity of a cold critical test of the FSP; however, additional testing would reduce uncertainties in the beryllium and uranium cross-section data and the overall uncertainty in the computational models. A series of critical experiments using HEU metal were performed in the 1960s and 1970s in support of criticality safety operations at the Y-12 Plant. Of the hundreds of experiments, three were identified as fast-fission configurations reflected by beryllium metal. These experiments have been evaluated as benchmarks for inclusion in the International Handbook of Evaluated Criticality Safety Benchmark Experiments (IHECSBE). Further evaluation of the benchmark experiments was performed using the sensitivity and uncertainty analysis capabilities of SCALE 6. The data adjustment methods of SCALE 6 have been employed in the validation of an example FSP design model to reduce the uncertainty due to the beryllium cross section data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

2010 Criticality Accident Alarm System Benchmark Experiment at the CEA Valduc SILENE Facility

Several experiments were performed at the CEA Valduc SILENE reactor facility, which are intended to be published as evaluated benchmark experiments in the ICSBEP Handbook. These evaluated benchmarks will be useful for the verification and validation of radiation transport codes and evaluated nuclear data, particularly those that are used in the analysis of CAASs. During these experiments SILENE was operated in pulsed mode in order to be representative of a criticality accident, which is rare among shielding benchmarks. Measurements of the neutron flux were made with neutron activation foils and measurements of photon doses were made with TLDs. Also unique to these experiments was the presence of several detectors used in actual CAASs, which allowed for the observation of their behavior during an actual critical pulse. This paper presents the preliminary measurement data currently available from these experiments. Also presented are comparisons of preliminary computational results with Scale and TRIPOLI-4 to the preliminary measurement data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Criticality Benchmark Analysis of the HTTR Annular Startup Core Configurations

One of the high priority benchmarking activities for corroborating the Next Generation Nuclear Plant (NGNP) Project and Very High Temperature Reactor (VHTR) Program is evaluation of Japan's existing High Temperature Engineering Test Reactor (HTTR). The HTTR is a 30 MWt engineering test reactor utilizing graphite moderation, helium coolant, and prismatic TRISO fuel. A large amount of critical reactor physics data is available for validation efforts of High Temperature Gas-cooled Reactors (HTGRs). Previous international reactor physics benchmarking activities provided a collation of mixed results that inaccurately predicted actual experimental performance. Reevaluations were performed by the Japanese to reduce the discrepancy between actual and computationally-determined critical configurations. Current efforts at the Idaho National Laboratory (INL) involve development of reactor physics benchmark models in conjunction with the International Reactor Physics Experiment Evaluation Project (IRPhEP) for use with verification and validation methods in the VHTR Program. Annular cores demonstrate inherent safety characteristics that are of interest in developing future HTGRs.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Current Reactor Physics Benchmark Activities at the Idaho National Laboratory

The International Reactor Physics Experiment Evaluation Project (IRPhEP) and the International Criticality Safety Benchmark Evaluation Project (ICSBEP) were established to preserve integral reactor physics and criticality experiment data for present and future research. These valuable assets provide the basis for recording, developing, and validating our integral nuclear data, and experimental and computational methods. These projects are managed through the Idaho National Laboratory (INL) and the Organisation for Economic Co-operation and Development Nuclear Energy Agency (OECD-NEA). Staff and students at the Department of Energy - Idaho (DOE-ID) and INL are engaged in the development of benchmarks to support ongoing research activities. These benchmarks include reactors or assemblies that support Next Generation Nuclear Plant (NGNP) research, space nuclear Fission Surface Power System (FSPS) design validation, and currently operational facilities in Southeastern Idaho.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Testing (Validating?) Cross Sections with ICSBEP Benchmarks [Slides]

We discuss how to use critical benchmarks from the International Handbook of Evaluated Criticality Safety Benchmark Experiments to determine the applicability of specific cross sections to the end-user's problem of interest. Particular attention is paid to making sure the selected suite of benchmarks includes the user's range of applicability (ROA).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

BENCHMARK EVALUATION OF THE INITIAL ISOTHERMAL PHYSICS MEASUREMENTS AT THE FAST FLUX TEST FACILITY

The benchmark evaluation of the initial isothermal physics tests performed at the Fast Flux Test Facility, in support of Fuel Cycle Research and Development and Generation-IV activities at the Idaho National Laboratory, has been completed. The evaluation was performed using MCNP5 with ENDF/B-VII.0 nuclear data libraries and according to guidelines provided for inclusion in the International Reactor Physics Experiment Evaluation Project Handbook. Results provided include evaluation of the initial fully-loaded core critical, two neutron spectra measurements near the axial core center, 32 reactivity effects measurements (21 control rod worths, two control rod bank worths, six differential control rod worths, two shutdown margins, and one excess reactivity), isothermal temperature coefficient, and low-energy electron and gamma spectra measurements at the core center. All measurements were performed at 400 ºF. There was good agreement between the calculated and benchmark values for the fully-loaded core critical eigenvalue, reactivity effects measurements, and isothermal temperature coefficient. General agreement between benchmark experiment measurements and calculated spectra for neutrons and low-energy gammas at the core midplane exists, but calculations of the neutron spectra below the core and the low-energy gamma spectra at core midplane did not agree well. Homogenization of core components may have had a significant impact upon computational assessment of these effects. Future work includes development of a fully-heterogeneous model for comprehensive evaluation. The reactor physics measurement data can be used in nuclear data adjustment and validation of computational methods for advanced fuel cycle and nuclear reactor systems using Liquid Metal Fast Reactor technology.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Benchmark Development in Support of Generation-IV Reactor Validation (IRPhEP 2010 Handbook)

The March 2010 edition of the International Reactor Physics Experiment Evaluation Project (IRPhEP) Handbook includes additional benchmark data that can be implemented in the validation of data and methods for Generation IV (GEN-IV) reactor designs. Evaluations supporting sodium-cooled fast reactor (SFR) efforts include the initial isothermal tests of the Fast Flux Test Facility (FFTF) at the Hanford Site, the Zero Power Physics Reactor (ZPPR) 10B and 10C experiments at the Idaho National Laboratory (INL), and the burn-up reactivity coefficient of Japan’s JOYO reactor. An assessment of Russia’s BFS-61 assemblies at the Institute of Physics and Power Engineering (IPPE) provides additional information for lead-cooled fast reactor (LFR) systems. Benchmarks in support of the very high temperature reactor (VHTR) project include evaluations of the HTR-PROTEUS experiments performed at the Paul Scherrer Institut (PSI) in Switzerland and the start-up core physics tests of Japan’s High Temperature Engineering Test Reactor. The critical configuration of the Power Burst Facility (PBF) at the INL which used ternary ceramic fuel, U(18)O2-CaO-ZrO2, is of interest for fuel cycle research and development (FCR&D) and has some similarities to “inert-matrix” fuels that are of interest in GEN-IV advanced reactor design. Two additional evaluations were revised to include additional evaluated experimental data, in support of light water reactor (LWR) and heavy water reactor (HWR) research; these include reactor physics experiments at Brazil’s IPEN/MB-01 Research Reactor Facility and the French High Flux Reactor (RHF), respectively. The IRPhEP Handbook now includes data from 45 experimental series (representing 24 reactor facilities) and represents contributions from 15 countries. These experimental measurements represent large investments of infrastructure, experience, and cost that have been evaluated and preserved as benchmarks for the validation of methods and collection of data in support of current and future reactor design and development.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

h5bench: A unified benchmark suite for evaluating HDF5 I/O performance on pre‐exascale platforms

Summary Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputers. With massive amounts of data produced or consumed by compute nodes, high‐performant parallel I/O is essential. I/O benchmarks play an important role in this process; however, there is a scarcity of I/O benchmarks representative of current workloads on HPC systems. Toward creating representative I/O kernels from real‐world applications, we have created h5bench , a set of I/O kernels that exercise hierarchical data format version 5 (HDF5) I/O on parallel file systems in numerous dimensions. Our focus on HDF5 is due to the parallel I/O library's heavy usage in various scientific applications running on supercomputing systems. The various tests benchmarked in the h5bench suite include I/O operations (read and write), data locality (arrays of basic data types and arrays of structures), array dimensionality (one‐dimensional arrays, two‐dimensional meshes, three‐dimensional cubes), I/O modes (synchronous and asynchronous). In this paper, we present the observed performance of h5bench executed along several of these dimensions on existing supercomputers (Cori and Summit) and pre‐exascale platforms (Perlmutter, Theta, and Polaris). h5bench measurements can be used to identify performance bottlenecks and their root causes and evaluate I/O optimizations. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful to the broader supercomputing and I/O community.

97 MATHEMATICS AND COMPUTING↗

Benchmarking Quantum Chemistry Computations with Variational, Imaginary Time Evolution, and Krylov Space Solver Algorithms

Quantum chemistry is a key application area for noisy-intermediate scale quantum (NISQ) devices, and therefore serves as an important benchmark for current and future quantum computer performance. Previous benchmarks in this field have focused on variational methods for computing ground and excited states of various molecules, including a benchmarking suite focused on the performance of computing ground states for alkali-hydrides under an array of error mitigation methods. State-of-the-art methods to reach chemical accuracy in hybrid quantum-classical electronic structure calculations of alkali hydride molecules on NISQ devices from IBM are outlined here. Here it is demonstrated how to extend the reach of variational eigensolvers with symmetry preserving Ansätze. Next, it is outlined how to use quantum imaginary time evolution and Lanczos as a complementary method to variational techniques, highlighting the advantages of each approach. Finally, a new error mitigation method is demonstrated which uses systematic error cancellation via hidden inverse gate constructions, improving the performance of typical variational algorithms. These results show that electronic structure calculations have advanced rapidly, to routine chemical accuracy for simple molecules, from their inception on quantum computers a few short years ago, and they point to further rapid progress to larger molecules as the power of NISQ devices grows.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Simulation of mineral dissolution at the pore scale with evolving fluid-solid interfaces: review of approaches and benchmark problem set

This manuscript presents a benchmark problem for the simulation of single-phase flow, reactive transport, and solid geometry evolution at the pore scale. The problem is organized in three parts that focus on specific aspects: flow and reactive transport (part I), dissolution-driven geometry evolution in two dimensions (part II), and an experimental validation of three-dimensional dissolution-driven geometry evolution (part III). Five codes are used to obtain the solution to this benchmark problem, including Chombo-Crunch, OpenFOAM-DBS, a lattice Boltzman code, Vortex, and dissolFoam. These codes cover a good portion of the wide range of approaches typically employed for solving pore-scale problems in the literature, including discretization methods, characterization of the fluid-solid interfaces, and methods to move these interfaces as a result of fluid-solid reactions. A short review of these approaches is given in relation to selected published studies. Results from the simulations performed by the five codes show remarkable agreement both quantitatively--based on upscaled parameters such as surface area, solid volume, and effective reaction rate--and qualitatively--based on comparisons of shape evolution. This outcome is especially notable given the disparity of approaches used by the codes. Therefore, these results establish a strong benchmark for the validation and testing of pore-scale codes developed for the simulation of flow and reactive transport with evolving geometries. They also underscore the significant advances seen in the last decade in tools and approaches for simulating this type of problem.

58 GEOSCIENCES↗

PASTA: A Parallel Sparse Tensor Algorithm Benchmark Suite

Tensor methods have gained increasingly attention from various applications, including machine learning, quantum chemistry, healthcare analytics, social network analysis, data mining, and signal processing, to name a few. Sparse tensors and their algorithms become critical to further improve the performance of these methods and enhance the interpretability of their output. This work presents a sparse tensor algorithm benchmark suite (PASTA) for single- and multi-core CPUs. To the best of our knowledge, this is the first benchmark suite for sparse tensor world. PASTA targets on: 1) helping application users to evaluate different computer systems using its representative computational workloads; 2) providing insights to better utilize existed computer architecture and systems and inspiration for the future design. This benchmark suite will be publicly released.

HPC, sparse tensor, benchmark↗

Towards robust surrogate models: Benchmarking machine learning approaches to expediting phase field simulations of brittle fracture

Data-driven approaches have the potential to make modeling complex, nonlinear physical phenomena significantly more computationally tractable. For example, computational modeling of fracture is a core challenge where machine learning techniques have the potential to provide a much needed speedup that would enable progress in areas such as multi-scale modeling and uncertainty quantification. Currently, phase field modeling (PFM) of fracture is one such approach that offers a convenient variational formulation to model crack nucleation, branching and propagation. To date, machine learning techniques have shown promise in approximating PFM simulations. While standard fracture benchmarks represent realistic scenarios frequently observed in practice, they typically do not provide sufficiently challenging tests for data-driven methods. Here, to address this gap, we introduce a challenging dataset based on PFM simulations designed to benchmark and advance ML methods for fracture modeling. This dataset includes three energy decomposition methods, two boundary conditions, and 1000 random initial crack configurations for a total of 6000 simulations. Each sample contains 100 time steps capturing the temporal evolution of the crack field. Alongside this dataset, we also implement and evaluate Physics Informed Neural Networks (PINN), Fourier Neural Operators (FNO), and UNet models as baselines, and explore the impact of ensembling strategies on prediction accuracy. With this combination of our dataset and baseline models drawn from the literature we aim to provide a standardized and challenging benchmark for evaluating machine learning approaches to solid mechanics. Our results highlight both the promise and limitations of popular current models, and demonstrate the utility of this dataset as a testbed for advancing machine learning in fracture mechanics research.

Benchmark dataset↗

Benchmark study of a new simplified DFN model for shearing of intersecting fractures and faults

It is challenging to quantitatively predict shearing of intersecting fractures/faults because of dynamic frictional contacts accompanied by possible nonlinear rock deformation. To address such challenges, a new conceptual model—the simplified DFN model—was proposed and validated by Hu et al. 46 to use major paths (MPs) to represent complicated DFNs for calculation of shearing. In this work, we conducted a benchmark study for three examples that involve different levels of complexity of intersecting fractures, and correspondingly different numbers of MPs. The codes and software that were used in the benchmark cover a range of continuum, discontinuum and hybrid numerical methods: NMM (LBNL), FLAC3D (LBNL), GBDEM (KIGAM), FRACOD (DynaFrax), and CASRock (CAS). The general consistency between DFN and MP cases as predicted by all the codes/software demonstrates that major paths can be used to simplify the geometry of DFNs in a wide range of software. Disagreement in results made by some software and potential future improvements are discussed. We show that (1) shearing of one or multiple major fractures can be reduced if there are multiple smaller intersecting fractures in that area, which is a useful basis for understanding and controlling induced seismicity and merits further analysis, and (2) the agreement achieved in the benchmark examples provide confidence that the simplified DFN model is a promising conceptual model that can be used for different types of numerical approaches and software for simplifying the analysis of the shearing of intersecting fractures and faults.

58 GEOSCIENCES↗

Benchmarking Dosimetric Quality Assessment of Prostate Intensity-Modulated Radiotherapy

Purpose: To benchmark the dosimetric quality assessment of prostate intensity-modulated radiotherapy and determine whether the quality is influenced by disease or treatment factors. Patients and Methods: We retrospectively analyzed the data from 155 consecutive men treated radically for prostate cancer using intensity-modulated radiotherapy to 78 Gy between January 2007 and March 2009 across six radiotherapy treatment centers. The plan quality was determined by the measures of coverage, homogeneity, and conformity. Tumor coverage was measured using the planning target volume (PTV) receiving 95% and 100% of the prescribed dose (V{sub 95%} and V{sub 100%}, respectively) and the clinical target volume (CTV) receiving 95% and 100% of the prescribed dose. Homogeneity was measured using the sigma index of the PTV and CTV. Conformity was measured using the lesion coverage factor, healthy tissue conformity index, and the conformity number. Multivariate regression models were created to determine the relationship between these and T stage, risk status, androgen deprivation therapy use, treatment center, planning system, and treatment date. Results: The largest discriminatory measurements of coverage, homogeneity, and conformity were the PTV V{sub 95%}, PTV sigma index, and conformity number. The mean PTV V{sub 95%} was 92.5% (95% confidence interval, 91.3-93.7%). The mean PTV sigma index was 2.10 Gy (95% confidence interval, 1.90-2.20). The mean conformity number was 0.78 (95% confidence interval, 0.76-0.79). The treatment center independently influenced the coverage, homogeneity, and conformity (all p < .0001). The planning system independently influenced homogeneity (p = .038) and conformity (p = .021). The treatment date independently influenced the PTV V{sub 95%} only, with it being better at the start (p = .013). Risk status, T stage, and the use of androgen deprivation therapy did not influence any aspect of plan quality. Conclusion: Our study has benchmarked measures of coverage, homogeneity, and conformity for the treatment of prostate cancer using IMRT. The differences seen between centers and planning systems and the coverage deterioration over time highlight the need for every center to determine their own benchmarks and apply clinical vigilance with respect to maintaining these through quality assurance.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Deep-Learning-Derived Evaluation Metrics Enable Effective Benchmarking of Computational Tools for Phosphopeptide Identification

Tandem mass spectrometry (MS/MS)-based phosphoproteomics is a powerful technology for global phosphorylation analysis. However, applying four computational pipelines to a typical mass spectrometry (MS)-based phosphoproteomic dataset from a human cancer study, we observed a large discrepancy among the reported phosphopeptide identification and phosphosite localization results, underscoring a critical need for benchmarking. While efforts have been made to compare performance of computational pipelines using data from synthetic phosphopeptides, evaluations involving real application data have been largely limited to comparing the numbers of phosphopeptide identifications due to the lack of appropriate evaluation metrics. We investigated three deep learning-derived features as potential evaluation metrics: phosphosite probability, Delta RT and spectral similarity. Predicted phosphosite probability is computed by MusiteDeep, which provides high accuracy as previously reported; Delta RT is defined as the absolute retention time (RT) difference between RTs observed and predicted by AutoRT; and spectral similarity is defined as the Pearson’s correlation coefficient between spectra observed and predicted by pDeep2. Using a synthetic peptide dataset, we found that both Delta RT and spectral similarity provided excellent discrimination between correct and incorrect peptide-spectrum matches (PSMs) both when incorrect PSMs involved wrong peptide sequences and even when incorrect PSMs were caused by only incorrect phosphosite localization. Based on these results, we used all the three deep learning-derived features as evaluation metrics to compare different computational pipelines on diverse set of phosphoproteomic datasets and showed their utility in benchmarking performance of the pipelines. The benchmark metrics demonstrated in this study will enable users to select computational pipelines and parameters for routine analysis of phosphoproteomics data and will offer guidance for developers to improve computational methods.

59 BASIC BIOLOGICAL SCIENCES↗

‘Wasteaware’ benchmark indicators for integrated sustainable waste management in cities

Highlights: • Solid waste management (SWM) is a key utility service, but data is often lacking. • Measuring their SWM performance helps a city establish priorities for action. • The Wasteaware benchmark indicators: measure both technical and governance aspects. • Have been developed over 5 years and tested in more than 50 cities on 6 continents. • Enable consistent comparison between cities and countries and monitoring progress. - Abstract: This paper addresses a major problem in international solid waste management, which is twofold: a lack of data, and a lack of consistent data to allow comparison between cities. The paper presents an indicator set for integrated sustainable waste management (ISWM) in cities both North and South, to allow benchmarking of a city’s performance, comparing cities and monitoring developments over time. It builds on pioneering work for UN-Habitat’s solid waste management in the World’s cities. The comprehensive analytical framework of a city’s solid waste management system is divided into two overlapping ‘triangles’ – one comprising the three physical components, i.e. collection, recycling, and disposal, and the other comprising three governance aspects, i.e. inclusivity; financial sustainability; and sound institutions and proactive policies. The indicator set includes essential quantitative indicators as well as qualitative composite indicators. This updated and revised ‘Wasteaware’ set of ISWM benchmark indicators is the cumulative result of testing various prototypes in more than 50 cities around the world. This experience confirms the utility of indicators in allowing comprehensive performance measurement and comparison of both ‘hard’ physical components and ‘soft’ governance aspects; and in prioritising ‘next steps’ in developing a city’s solid waste management system, by identifying both local strengths that can be built on and weak points to be addressed. The Wasteaware ISWM indicators are applicable to a broad range of cities with very different levels of income and solid waste management practices. Their wide application as a standard methodology will help to fill the historical data gap.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Benchmark Energetic Data in a Model System for Grubbs II Metathesis Catalysis and Their Use for the Development, Assessment, and Validation of Electronic Structure Methods

We present benchmark relative energetics in the catalytic cycle of a model system for Grubbs second-generation olefin metathesis catalysts. The benchmark data were determined by a composite approach based on CCSD(T) calculations, and they were used as a training set to develop a new spin-component-scaled MP2 method optimized for catalysis, which is called SCSC-MP2. The SCSC-MP2 method has improved performance for modeling Grubbs II olefin metathesis catalysts as compared to canonical MP2 or SCS-MP2. We also employed the benchmark data to test 17 WFT methods and 39 density functionals. Among the tested density functionals, M06 is the best performing functional. M06/TZQS gives an MUE of only 1.06 kcal/mol, and it is a much more affordable method than the SCSC-MP2 method or any other correlated WFT methods. The best performing meta-GGA is M06-L, and M06-L/DZQ gives an MUE of 1.77 kcal/mol. PBEh is the best performing hybrid GGA, with an MUE of 3.01 kcal/mol; however, it does not perform well for the larger, real Grubbs II catalyst. B3LYP and many other functionals containing the LYP correlation functional perform poorly, and B3LYP underestimates the stability of stationary points for the cis-pathway of the model system by a large margin. From the assessments, we recommend the M06, M06-L, and MPW1B95 functionals for modeling Grubbs II olefin metathesis catalysts. The local M06-L method is especially efficient for calculations on large systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

RADIOGRAPHIC BENCHMARK PROBLEM 2009 - SCATTER CALCULATIONS IN MODELLING

Code Validation is a permanent concern in computer simulation, and has been addressed repeatedly in eddy current and ultrasonic modelling. A good benchmark problem is sufficiently simple to be taken into account by various codes without strong requirements on geometry representation capabilities, focuses on few or even a single aspect of the problem at hand to facilitate interpretation and to avoid that compound errors compensate themselves, yields a quantitative result and is experimentally accessible. In this paper we attempt to address code validation for one aspect of radio-graphic modelling, the scattered radiation prediction. An update of the results of the 2008 benchmark is presented. Additionally we discuss the extension of this benchmark on the lower energy part for 60 and 80 keV as well as for higher energies up to 10 MeV to study the contribution of pair production. Of special interest will be the primary radiation (attenuation law as reference), the total scattered radiation, the relative contribution of scattered radiation separated by order of scatter events (1st, 2nd, ..., 20th), and the spectrum of scattered radiation. We present the results of three Monte Carlo codes (MC-Ray, Sindbad and Moderato) as well as an analytical first order scattering code (VXI) and compare to MCNP as reference.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗