Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

Cosmogenic radionuclide production modeling with Geant4: Experimental benchmarking and application to nuclear spectroscopy of asteroid (16) Psyche

Measurements of gamma-ray emissions from the decay of cosmogenic radionuclides provide an opportunity to characterize the elemental composition of a terrestrial planet or asteroid surface. We report on the development of a Geant4 application which models cosmogenic radionuclide production on metal-rich surfaces. The model was benchmarked using measurements of radionuclides produced during 1 GeV proton irradiation of a target made from the Campo del Cielo iron meteorite. The gamma-ray emitting radionuclides 58 Co, 57 Co, 56 Co, 54 Mn, 52 Mn, 51 Cr, 48 V, 46 Sc and 22 Na were observed following the irradiation. Our model reproduced the measured radionuclide production rates to within a factor of 2.5 or better. All but two of the elements ( 54 Mn, 46 Sc) have a perfect model-to-data match within their measurement uncertainties. The benchmarked model was used to predict cosmogenic radionuclide production on a large (~100-km radius) metal-rich asteroid. The results are relevant for planned gamma-ray measurements of the metallic asteroid (16) Psyche, which will be visited by the Psyche spacecraft in 2026. We found that galactic-proton-induced radionuclide decay is unlikely to be observed by the Psyche Gamma-Ray Spectrometer (GRS), however an intense solar proton event (>2×10 6 protons cm -2 over <5 days) will produce measurable quantities of radioisotopes. Measurements of these radioisotopes could provide an independent method of determining the Ni-to-Fe ratio of materials at Psyche’s surface. Such an analysis will require the use of radionuclide production cross sections to convert GRS-measured cosmogenic radionuclide decay rates to elemental composition information with the ~10% precision required for planetary geochemical studies.

79 ASTRONOMY AND ASTROPHYSICS↗

Material Interactions in Severe Accidents – Benchmarking the MELCOR V2.2 Eutectics Model for a BWR-3 MARK-I Station Blackout: Part I – Single Case Analysis

Here in this analysis, the two material interaction models available in the MELCOR code are benchmarked for a severe accident at a BWR under representative Fukushima Daiichi boundary conditions. This part of the benchmark investigates the impact of each material interaction model on accident progression through a detailed single case analysis. It is found that the eutectics model simulation exhibits more rapid accident progression for the duration of the accident. The slower accident progression exhibited by the interactive materials model simulation, however, allows for a greater degree of core material oxidation and hydrogen generation to occur, as well as elevated core temperatures during the ex-vessel accident phase. The eutectics model simulation exhibits more significant degradation of core components during the late in-vessel accident phase – more debris forms and relocates to the lower plenum before lower head failure. The larger debris bed observed in the eutectics model simulation also reaches higher temperatures, presenting a more significant thermal challenge to the lower head until its failure. At the end of the simulated accident scenario, however, core damage is comparable between both simulations due to significant core degradation that occurs during the ex-vessel phase in the interactive materials model simulation. A key difference between the two models’ performance is the maximum temperatures that can be reached in the core and therefore the maximum ΔT between any two components. When implementing the interactive materials model, users have the option to modify the liquefaction temperature of the ZrO 2 -interactive and UO 2 -interactive materials as a way to mimic early fuel rod failure due to material interactions. Through modification of the liquefaction of high melting point materials with significant mass, users may inadvertently limit maximum core temperatures for fuel, cladding, and debris components.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Flow reversal benchmark of a one-sided heated narrow rectangular channel with CATHARE and RELAP5

Flow reversal in narrow coolant channels can be a crucial phenomenon for the safety of research reactors with a downward nominal flow direction. During a loss of forced flow accident, the downward flow stagnates briefly before transitioning into an upward natural circulation flow. The fuel may be damaged if dryout occurs and threshold fuel and/or cladding temperatures are exceeded. A comprehensive study is provided for flow reversal in narrow rectangular channels by examining experimental data and conducting software model analyses. The literature on flow reversal was reviewed, and selected experimental datasets were used to benchmark against CATHARE and RELAP5 models and also compare the code calculations with each other. The experimental data comes from flow reversal tests conducted with a narrow rectangular channel with one-sided heating. The results were compared with experimental data for successful flow reversal tests and predicted dryout power for dryout conditions. Also, the study examined the effects of the pump coastdown period, inlet liquid temperature, system pressure, and localized pressure drops. The experimental results showed that shorter coastdown periods, reduced pressure drops, and lower coolant inlet temperatures increased the dryout power. However, the system pressure did not noticeably affect the results. The simulation results showed that both CATHARE and RELAP5 agreed with experimental data, capturing the trends of the experimental results. Slight differences between each code calculation, as well as the predicted and measured dryout powers, were attributed to experimental uncertainties and the modeling of physical phenomena such as wall nucleation, interfacial heat transfer, drag coefficients, and critical heat flux. Overall, this study provides an understanding of flow reversal and the prediction capabilities of thermal-hydraulics software models. In conclusion, a future study of the flow reversal benchmark of a narrow rectangular channel with two-sided heating may provide additional valuable insights.

CATHARE↗

Benchmark of interfacial area concentration approaches for the two-fluid model in gas-dispersed condensing flow

Accurate predictions of interfacial area concentration (IAC) are required for closure of the condensation phase change term. The interfacial area transport equation (IATE) can dynamically predict interfacial area concentration and is increasingly being used in the benchmarking of phase change flows over static based correlations which require a flow regime map. For higher flow regimes, the two-group two-fluid IATE model can separately model the transport of spherical bubbles and cap/slug bubbles. Recently, a modified formulation of the one-group and two-group two-fluid model with interfacial area transport equation was proposed for gas-dispersed condensing flows. Three datasets have been used for validation consisting of condensing two-phase flow data in the unheated section of a vertical annulus with conditions spanning a range of pressures, inlet subcoolings and mass fluxes for benchmarking the various IAC approaches. For bubbly flows, the coupled void transport - IATE simulation results in good agreement in the predictions of the void fraction and the interfacial area concentration under nearly all sixty conditions. Considering the proposed two-group two-fluid IATE model formulation, the group-2 Nusselt number correlation is investigated to account for the appropriate heat transfer length scale for group-2 bubbles. Validation of the proposed two-group two-fluid model against the new dataset shows good agreement with experimental data with significant improvement over the one-group IATE model for conditions with substantial group-2 void fraction throughout the simulation domain.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Observations of modal coupling due to bolted joints in an experimental benchmark structure

The goal of this paper is to present a set of measurements from a benchmark structure containing two bolted joints to support future efforts to predict the damping due to the joints and to model nonlinear coupling between the first two elastic modes. Bolted joints introduce nonlinearities in structures, typically causing a softening in the natural frequency and an increase in damping because of frictional slip between the contact interfaces within the joint. These nonlinearities pose significant challenges when characterizing the response of the structure under a large range of load amplitudes, especially when the modal responses become coupled, causing the effective damping and natural frequency to not only depend on the excitation amplitude of the targeted mode, but also the relative amplitudes of other modes. In this work, two nominally identical benchmark structures, known in some prior works as the S4 beam, are tested to characterize their nonlinear properties for the first two elastic modes. Detailed surface measurements are presented and validated through finite element analysis and reveal distinct contact interactions between the two sets of beams. The free-free test structures are excited with an impact hammer and the transient response is analyzed to extract the damping and frequency backbone curves. A range of impact amplitudes and drive points are used to isolate a single mode or to excite both modes simultaneously. Differences in the nonlinear response correlate with the relative strength of the modes that are excited, allowing one to characterize mode coupling. Each of the beams shows different nonlinear properties for each mode, which is attributed to the different contact pressure distributions between the parts, although the mode coupling relationship is found to be consistent between the two. Lastly, the test data key finding are presented in this paper and the supporting data is available on a public repository for interested researchers.

42 ENGINEERING↗

Benchmarking Coordination Number Prediction Algorithms on Inorganic Crystal Structures

Coordination numbers and geometries form a theoretical framework for understanding and predicting materials properties. Algorithms to determine coordination numbers automatically are increasingly used for machine learning and automatic structural analysis. In this work, we introduce MaterialsCoord, a benchmark suite containing 56 experimentally-derived crystal structures (spanning elements, binaries, and ternary compounds) and their corresponding coordination environments as described in the research literature. We also describe CrystalNN, a novel algorithm for determining near neighbors. We compare CrystalNN against 7 existing near-neighbor algorithms on the MaterialsCoord benchmark, finding CrystalNN to perform similarly to several well-established algorithms. For each algorithm, we also assess computational demand and sensitivity towards small perturbations that mimic thermal motion. Finally, we investigate the similarity between bonding algorithms when applied to the Materials Project database. Finally, we expect that this work will aid the development of coordination prediction algorithms as well as improve structural descriptors for machine learning and other applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Benchmarking the GW Approximation and Bethe–Salpeter Equation for Groups IB and IIB Atoms and Monoxides

Energies from the GW approximation and the Bethe–Salpeter equation (BSE) are benchmarked against the excitation energies of transition-metal (Cu, Zn, Ag, and Cd) single atoms and monoxide anions. We demonstrate that best estimates of GW quasiparticle energies at the complete basis set limit should be obtained via extrapolation or closure relations, while numerically converged GW -BSE eigenvalues can be obtained on a finite basis set. Calculations using real-space wave functions and pseudopotentials are shown to give best-estimate GW energies that agree (up to the extrapolation error) with calculations using all-electron Gaussian basis sets. We benchmark the effects of a vertex approximation (ΓLDA) and the mean-field starting point in GW and the BSE, performing computations using a real-space, transition-space basis and scalar-relativistic pseudopotentials. Here, while no variant of GW improves on perturbative G 0 W 0 at predicting ionization energies, G 0 W 0 Γ LDA -BSE computations give excellent agreement with experimental absorption spectra as long as off-diagonal self-energy terms are included. We also present G 0 W 0 quasiparticle energies for the CuO – , ZnO – , AgO – , and CdO – anions, in comparison to available anion photoelectron spectra.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Benchmark Electronic Structure Calculations for H 3 O + (H 2 O) n , n = 0–5, Clusters and Tests of an Existing 1,2,3-Body Potential Energy Surface with a New 4-Body Correction

We report a revision of a previously developed many-body (up to 3-body inter- actions) CCSD(T)-based potential energy surface (PES) for the hydrated proton and its extension that incorporates a simple 4-body term for the hydronium-water-water-water interactions. The new PES is parametrized from and tested against new extensive benchmark CCSD(T) Complete Basis Set (CBS) estimates of the binding energies of H3O+(H2O)n clusters, n=0-5, which include all known to date low-lying energy isomers. We report the energetics, structures and harmonic frequencies of the hydronium-water clusters using the PES and compare them against the new benchmarks as well as other best available ab initio results. This critical comparison demonstrates the high accuracy of the new many-body hydronium-water PES.

Heindel, Joseph P.↗

A Theoretical Benchmark of the Geometric and Optical Properties for 3d Transition Metal Nanoclusters via Density Functional Theory

Understanding structure–property relationships in atomically precise metal nanoclusters is vital in finding selective and tunable catalysts. Here, in this study, density functional theory (DFT) was used to benchmark seven exchange correlation functionals at different basis sets for 17 atomically precise nanoclusters against experimentally determined geometries, band gaps, and optical gaps. The set contains both monometallic and bimetallic clusters that possess at least two types of 3d transition metals (specifically, Cu, Ni, Fe, or Co). The benchmark highlights that PBE0 is a good functional to use regardless of the basis set, and Minnesota functionals do well with respect to specific metals. Further, while long-range corrected functionals overestimate band and optical gaps, they model absorption features better than the other considered functionals. The study additionally looks at the photoinduced hydrogen evolution reaction (HER) and the CO 2 reduction mechanism on nanoclusters reported from the literature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Concordant Mode Approach (CMA): Vibrational Analysis of New and Upgraded Intermolecular Benchmarks for Noncovalent Bonding

The Concordant Mode Approach (CMA) is a novel method that offers tremendous potential for increasing the system size and the level of theory attainable in quantum chemical computations of molecular vibrational frequencies. To investigate the extension of CMA to intermolecular vibrations, computations with coupled cluster singles and doubles with perturbative triples theory [CCSD(T)] using two augmented correlation-consistent polarized-valence triple-ζ basis sets (aug-cc-pVTZ or h-aug-ccpVTZ) were performed on 17 prototypical loosely bound complexes of hydrogen-bonded, dispersion, and mixed character. These Level A results provide new and upgraded benchmarks for noncovalent bonding and a severe test for CMA vibrational analyses. The Level A target frequencies were recovered remarkably well using second-order Møller−Plesset perturbation theory (MP2) with h-aug-cc-pVTZ for generating the underlying (Level B) normal modes of the CMA scheme. Employing this Level B within the lowest-rung CMA-0A method reproduces the 435 benchmark frequencies with a mean absolute error (MAE) of 0.23 cm −1 and a corresponding standard deviation (σ) of 0.84 cm −1 ; strikingly, the corresponding subset of 106 interfragment frequencies exhibits MAE = 0.34 cm −1 and σ = 0.90 cm −1 . Subsequent application of the higher-rung CMA-2A scheme eliminates all outliers and reduces the overall MAE to a minuscule 0.08 cm −1 with the inclusion of only 3.0% of the off-diagonal couplings not accounted for by CMA-0A. Accordingly, the highly efficient CMA methodology proves to be robust even for vibrations on flat potential energy surfaces.

Aromatic compounds↗

Diffusion Quantum Monte Carlo Benchmarking of Magnetic Moments in MnBi 2 Te 4

The intrinsically antiferromagnetic topological insulator, MnBi 2 Te 4 (MBT), has garnered significant attention recently due to its potential to host numerous exotic topological quantum states. Unfortunately, their consistent realization has been hindered by intrinsic antisite defects among the Mn and Bi sublattices. In this work, we establish Mn magnetization of pristine MBT through high level diffusion Monte Carlo calculations, which can serve as a precise starting point for various models to estimate antisite defect concentrations in actual MBT samples. The benchmark quality of DMC calculations is further identified from out model estimating antisite defect concentrations, which combines the benchmarked Mn magnetization with data from magnetic susceptibility and intermediate field magnetization measurements. This reproduces well Bi Mn and Mn Bi concentrations measured in the experiments. Here, we anticipate these theoretically based magnetic purity measures may be used as minimization targets in cycles of refinement to synthesize MBT with low antisite defect concentrations and more reproducible topological properties.

Defects↗

Guest-Host Interactions in Clathrate Hydrates: Benchmark MP2 and CCSD(T)/CBS Binding Energies of CH4, CO2 and H2S in (H2O)20 Cages

We present benchmark binding energies of naturally occurring gas molecules CH4, CO2, and H2S in the small cage, namely the pentagonal dodecahedron (512) (H2O)20, which is one of the constituent cages of the 3 major lattices (structures I, II and H) of clathrate hydrates. These weak interactions require higher levels of electron correlation and converge slowly with increasing basis set to the Complete Basis Set (CBS) limit, necessitating the use of large basis sets up to the augcc- pV5Z and subsequent correction for Basis Set Superposition Error (BSSE). For the host hollow (H2O)20 cages we have identified a most stable isomer with binding energy of -200.8 ± 2.1 kcal/mol at the CCSD(T)/CBS limit (-199.2 ± 0.5 kcal/mol at the MP2/CBS limit). Additionally, we report converged second order Moller-Plesset (MP2) CBS binding energies for the encapsulation of guests in the (H2O)20 cage of -4.3 ± 0.1 for CH4@(H2O)20, -6.6 ± 0.1 for CO2@(H2O)20 and -8.5 ± 0.1 kcal/mol for H2S@(H2O)20, respectively. For CH4@(H2O)20, exhibiting the weakest encapsulation affinity among the three, we report CCSD(T)/aug-cc-pVTZ binding energies and, based on them, a CCSD(T)/CBS estimate of -4.75 ± 0.1 kcal/mol. To the best of our knowledge, the CCSD(T)/aug-cc-pVTZ calculation for CH4@(H2O)20 is the largest one reported to date (168 valence electrons, 1978 basis functions and the correlation of 84 doubly occupied and 1873 virtual orbitals) and required a scalable implementation of the (T) module on 6144 nodes (350208 cores) of the “Cori” supercomputer at the National Energy Research Supercomputing Center (NERSC) for a total execution time of 195 minutes (for the (T) part). These efficient scalable implementations of highly correlated methods offer the capability to obtain long-lasting benchmarks of intermolecular interactions in complex systems. They also provide a path towards parametrizing classical potentials needed to study the dynamical and transport properties in these complex systems as well as assess the accuracy of lower scaling electronic structure methods such as Density Functional Theory (DFT) and MP2 including its spin-biased variants.

Heindel, Joseph↗

Barrier Heights for Diels–Alder Transition States Leading to Pentacyclic Adducts: A Benchmark Study of Crowded, Strained Transition States of Large Molecules

Theoretical characterization of reactions of complex molecules depends on providing consistent accuracy for the relative energies of intermediates and transition states. Here we employ the DLPNO-CCSD(T) method with core–valence correlation, large basis sets, and extrapolation to the CBS limit to provide benchmark values for Diels–Alder transition states leading to competitive strained pentacyclic adducts. We then used those benchmarks to test a diverse set of wave function and density functional methods for the absolute and relative barrier heights of these transition states. Our results show that only a few of the tested density functionals can predict the absolute barrier heights satisfactorily, although relative barrier heights are more accurate. The most accurate functionals tested are ωB97M-V, M11plus, ωB97X-V, PBE-D3(0), M11, and MN15 with MUDs from best estimates less than 3.0 kcal. Furthermore, these findings can guide selection of density functionals for future studies of crowded, strained transition states of large molecules.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Representing Nitrogen, Phosphorus, and Carbon Interactions in the E3SM Land Model: Development and Global Benchmarking

Over the past several decades, the land modeling community has recognized the importance of nutrient regulation on the global terrestrial carbon cycle. Implementations of nutrient limitation in land models are diverse, varying from applying simple empirical down–regulation of potential gross primary productivity under nutrient deficit conditions to more mechanistic treatments. In this study, we introduce a new approach to model multinutrient (nitrogen [N] and phosphorus [P]) limitations in the Energy Exascale Earth System Model (E3SM) Land Model version 1 (ELMv1–ECA). The development is grounded on (1) advances in representing multiple–consumer, multiple–nutrient competition; (2) a generic dynamic allocation scheme based on water, N, P, and light availability; (3) flexible plant CNP stoichiometry; (4) prognostic treatment of N and P constraints on several carbon cycle processes; and (5) global data sets of plant physiological traits. Through benchmarking the model against best knowledge of global plant and soil carbon pools and fluxes, we show that our implementation of nutrient constraints on the present–day carbon cycle is robust at the global scale. Compared with predecessor versions, ELMv1–ECA better predicts global–scale gross primary productivity, ecosystem respiration, leaf area index, vegetation biomass, soil carbon stocks, evapotranspiration, N2O emissions, and NO3– leaching. Factorial experiments imply that representing the phosphorus cycle improves modeled carbon fluxes, while considering dynamic allocation improves modeled carbon stock density. We also illustrate the value of using the International Land Model Benchmarking (ILAMB) package to evaluate and document performance during model development.

58 GEOSCIENCES↗

Critical Assessment of Metagenome Interpretation—a benchmark of metagenomics software

Methods for assembly, taxonomic profiling and binning are key to interpreting metagenome data, but a lack of consensus about benchmarking complicates performance assessment. The Critical Assessment of Metagenome Interpretation (CAMI) challenge has engaged the global developer community to benchmark their programs on highly complex and realistic data sets, generated from ~700 newly sequenced microorganisms and ~600 novel viruses and plasmids and representing common experimental setups. Assembly and genome binning programs performed well for species represented by individual genomes but were substantially affected by the presence of related strains. Taxonomic profiling and binning programs were proficient at high taxonomic ranks, with a notable performance decrease below family level. Parameter settings markedly affected performance, underscoring their importance for program reproducibility. The CAMI results highlight current challenges but also provide a roadmap for software selection to answer specific research questions.

59 BASIC BIOLOGICAL SCIENCES↗

Benchmarking quantum logic operations relative to thresholds for fault tolerance

Contemporary methods for benchmarking noisy quantum processors typically measure average error rates or process infidelities. However, thresholds for fault-tolerant quantum error correction are given in terms of worst-case error rates—defined via the diamond norm—which can differ from average error rates by orders of magnitude. One method for resolving this discrepancy is to randomize the physical implementation of quantum gates, using techniques like randomized compiling (RC). In this work, we use gate set tomography to perform precision characterization of a set of two-qubit logic gates to study RC on a superconducting quantum processor. We find that, under RC, gate errors are accurately described by a stochastic Pauli noise model without coherent errors, and that spatially correlated coherent errors and non-Markovian errors are strongly suppressed. We further show that the average and worst-case error rates are equal for randomly compiled gates, and measure a maximum worst-case error of 0.0197(3) for our gate set. Our results show that randomized benchmarks are a viable route to both verifying that a quantum processor’s error rates are below a fault-tolerance threshold, and to bounding the failure rates of near-term algorithms, if—and only if—gates are implemented via randomization methods which tailor noise.

97 MATHEMATICS AND COMPUTING↗

Quantum annealing for combinatorial optimization: a benchmarking study

Quantum annealing (QA) has the potential to significantly improve solution quality and reduce time complexity in solving combinatorial optimization problems compared to classical optimization methods. However, due to the limited number of qubits and their connectivity, the QA hardware did not show such an advantage over classical methods in past benchmarking studies. Recent advancements in QA with more than 5000 qubits, enhanced qubit connectivity, and the hybrid architecture promise to realize the quantum advantage. Here, we use a quantum annealer with state-of-the-art techniques and benchmark its performance against classical solvers. To compare their performance, we solve over 50 optimization problem instances represented by large and dense Hamiltonian matrices using quantum and classical solvers. The results demonstrate that a state-of-the-art quantum solver has higher accuracy (~0.013%) and a significantly faster problem-solving time (~6561×) than the best classical solver. Our results highlight the advantages of leveraging QA over classical counterparts, particularly in hybrid configurations, for achieving high accuracy and substantially reduced problem solving time in large-scale real-world optimization problems.

97 MATHEMATICS AND COMPUTING↗

Benchmarking highly entangled states on a 60-atom analogue quantum simulator

Abstract Quantum systems have entered a competitive regime in which classical computers must make approximations to represent highly entangled quantum states 1,2 . However, in this beyond-classically-exact regime, fidelity comparisons between quantum and classical systems have so far been limited to digital quantum devices 2–5 , and it remains unsolved how to estimate the actual entanglement content of experiments 6 . Here, we perform fidelity benchmarking and mixed-state entanglement estimation with a 60-atom analogue Rydberg quantum simulator, reaching a high-entanglement entropy regime in which exact classical simulation becomes impractical. Our benchmarking protocol involves extrapolation from comparisons against an approximate classical algorithm, introduced here, with varying entanglement limits. We then develop and demonstrate an estimator of the experimental mixed-state entanglement 6 , finding our experiment is competitive with state-of-the-art digital quantum devices performing random circuit evolution 2–5 . Finally, we compare the experimental fidelity against that achieved by various approximate classical algorithms, and find that only the algorithm we introduce is able to keep pace with the experiment on the classical hardware we use. Our results enable a new model for evaluating the ability of both analogue and digital quantum devices to generate entanglement in the beyond-classically-exact regime, and highlight the evolving divide between quantum and classical systems.

Science & Technology - Other Topics↗