Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

Role of electron correlation on the adenine dimer interaction for non-equilibrium geometries: a benchmark Quantum Monte Carlo study

The accurate description of non-covalent interactions is critical for understanding the structure, dynamics, and eventual function of biomolecules. The adenine dimer serves as a benchmark system for computational methods due to its role in nucleic acid structures and its rich conformational landscape. In this study, we employ benchmark diffusion quantum Monte Carlo (DMC) methods to investigate the relative energies and role of electron correlation on a set of adenine dimer conformations generated via a search of the potential energy landscape using the global optimizer algorithm. Relative DMC energies are compared against a wide range of density functional theory (DFT) approximation results. We find that although most of the DFT functionals perform well for low-energy structures, their accuracy varies significantly for higher-energy conformations, including stacked and T-shaped structures. A large fraction of the variation is due to the treatment of the van der Waals interaction. BLYP, B3LYP, and PBE0 significantly improve with added D4 dispersion, while the recent r2SCAN-D4 and ωB97M-V functionals show the least scatter and closest agreement with the DMC. These findings highlight the delicate nature of these interactions in biomolecular systems and provide guidance for simulations of their structure and dynamics and for the development of machine learned interatomic potentials.

Washburn, Laurel [ORNL] (ORCID:0000000324179335)↗

The Role of Nuclear Data Sensitivities in Prompt α-Eigenvalue Predictions of Delayed Critical Benchmarks

Alpha (α) eigenvalues, which describe the logarithmic time derivative of the neutron population in a multiplying system, are integral to time-dependent behavior and diagnostic applications. However, uncertainties in the evaluated nuclear data can significantly impact the accuracy of transport simulations for such quantities. This work explores the use of machine learning models to predict two key outputs, α-eigenvalues and keff bias, using input features derived from α-eigenvalue sensitivities to nuclear data. The criticality safety benchmark models used in this study come from the International Handbook of Evaluated Criticality Safety Benchmark Experiments. Three models, random forest, XGBoost, and NGBoost, are trained on both energy-resolved and energy-summed α sensitivities. For the α-eigenvalue bias prediction, NGBoost achieved the highest R 2 (0.9476) using energy-resolved features, while XGBoost performed best using summed sensitivities. In contrast, when predicting the keff bias, all the models showed moderate predictive capability (best R 2 ≈ 0.72), as the mapping from the static α-sensitivities to the static keff bias was less direct. SHAP (SHapley Additive exPlanations) analysis was used to interpret the model predictions. Across both prediction tasks, the features associated with neutron capture [H-1 (n, γ)], uranium scattering reactions (such as 235 U elastic/inelastic), and actinide capture/fission reactions (such as 239 Pu and 234 U) were consistently identified as the most impactful. This highlights the key role of specific nuclear reactions and energy ranges in shaping both time-dependent and steady-state criticality behavior. These results demonstrated that α-sensitivities, despite being computed for time-dependent metrics, can provide valuable insights for predicting both α-eigenvalues and the keff bias. Moreover, machine learning models offer a promising pathway for uncovering important nuclear data dependencies and guiding future data evaluation efforts.

Nuclear data↗

Long-Term Fish Monitoring in Large Rivers: Utility of “Benchmarking” across Basins

Abstract In business, benchmarking is a widely used practice of comparing your own business processes to those of other comparable companies and incorporating identified best practices to improve performance. Biologists and resource managers designing and conducting monitoring programs for fish in large river systems tend to focus on single river basins or segments of large rivers, missing opportunities to learn from those conducting fish monitoring in other rivers. We briefly examine five long-term fish monitoring programs in large rivers in the United States (Colorado, Columbia, Mississippi, Illinois, and Tallapoosa rivers) and identify opportunities for learning across programs by detailing best monitoring practices and why these practices were chosen. Although monitoring objectives, methods, and program maturity differ between each river system, examples from these five case studies illustrate the important role that long-term monitoring programs play in interpreting temporal and spatial shifts in fish populations for both established objectives and newly emerging questions. We suggest that deliberate efforts to develop a broader collaborative network through benchmarking will facilitate sharing of ideas and development of more effective monitoring programs.

Ward, David L.↗

APSIDAL MOTION OF THE MASSIVE, BENCHMARK ECLIPSING BINARY V578 Mon

V578 Mon is a system of two early B-type stars in the Rosette Nebula star-forming region (NGC 2244) and is one of only nine eclipsing binaries with component masses greater than 10 M{sub sun} whose physical parameters have been determined with an accuracy of better than 3%. It is therefore a benchmark system for evolutionary and stellar structure models of newly formed massive stars. Combining our multi-band light curves spanning 40 yr with previous light curve data from the literature, we fit a model light curve that for the first time includes the effects of apsidal motion of the system. We measure an apsidal period of 33.48{sup +0.10}{sub -0.06} yr. As a consequence of incorporating the apsidal motion into the modeling of the system's orbital parameters, we determine an updated eccentricity of e = 0.07755{sup +0.00022}{sub -0.00027}, which differs significantly from the value previously reported in the literature. Evidently, the inclusion of apsidal motion in the light curve modeling significantly affects the eccentricity determination. Incorporating these key parameters into a comprehensive model of the system's physical parameters-including internal structure constraints-will bring V578 Mon to the next level of benchmark precision and utility.

79 ASTRONOMY AND ASTROPHYSICS↗

UNIFORM ATMOSPHERIC RETRIEVAL ANALYSIS OF ULTRACOOL DWARFS. I. CHARACTERIZING BENCHMARKS, Gl 570D AND HD 3651B

Interpreting the spectra of brown dwarfs is key to determining the fundamental physical and chemical processes occurring in their atmospheres. Powerful Bayesian atmospheric retrieval tools have recently been applied to both exoplanet and brown dwarf spectra to tease out the thermal structures and molecular abundances to understand those processes. In this manuscript we develop a significantly upgraded retrieval method and apply it to the SpeX spectral library data of two benchmark late T dwarfs, Gl 570D and HD 3651B, to establish the validity of our upgraded forward model parameterization and Bayesian estimator. Our retrieved metallicities, gravities, and effective temperatures are consistent with the metallicity and presumed ages of the systems. We add the carbon-to-oxygen ratio as a new dimension to benchmark systems and find good agreement between carbon-to-oxygen ratios derived in the brown dwarfs and the host stars. Furthermore, we have for the first time unambiguously determined the presence of ammonia in the low-resolution spectra of these two late T dwarfs. We also show that the retrieved results are not significantly impacted by the possible presence of clouds, though some quantities are significantly impacted by uncertainties in photometry. This investigation represents a watershed study in establishing the utility of atmospheric retrieval approaches on brown dwarf spectra.

79 ASTRONOMY AND ASTROPHYSICS↗

Benchmarking of the mono-energetic transport coefficients-results from the International Collaboration on Neoclassical Transport in Stellarators (ICNTS)

Numerical results for the three mono-energetic transport coefficients required for a complete neoclassical description of stellarator plasmas have been benchmarked within an international collaboration. These transport coefficients are flux-surface-averaged moments of solutions to the linearized drift kinetic equation which have been determined using field-line-integration techniques, Monte Carlo simulations, a variational method employing Fourier-Legendre test functions and a finite-difference scheme. The benchmarking has been successfully carried out for past, present and future devices which represent different optimization strategies within the extensive configuration space available to stellarators. A qualitative comparison of the results with theoretical expectations for simple model fields is provided. The behaviour of the results for the mono-energetic radial and parallel transport coefficients can be largely understood from such theoretical considerations but the mono-energetic bootstrap current coefficient exhibits characteristics which have not been predicted.

Beidler, C. D.↗

Benchmark of correlation matrix renormalization method in molecule calculations

In this paper, we report benchmark calculations of the correlation matrix renormalization (CMR) approach for 23 molecules in the well-established G2 molecule set. This subset represents molecules with spin-singlet ground state in a variety of chemical bonding and coordination environments. The QUAsi-atomic minimal basis-set orbitals (QUAMBOs) are used as local orbitals in both CMR and full configuration interaction (FCI) calculations for comparison. The results obtained from the calculations are also compared with available experimental data. It is shown that the CMR method produces binding and dissociation energy curves in good agreement with the QUAMBO-FCI calculations as well as experimental results. Finally, the CMR benchmark calculations yield a standard deviation of 0.09 Å for the equilibrium bond length and 0.018 Hartree/atom for the formation energy, with a gain of great computational efficiency which scales like Hartree–Fock method.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Benchmarking structural evolution methods for training of machine learned interatomic potentials

When creating training data for machine-learned interatomic potentials (MLIPs), it is common to create initial structures and evolve them using molecular dynamics (MD) to sample a larger configuration space. Here, we benchmark two other modalities of evolving structures, contour exploration (CE) and dimer-method (DM) searches against MD for their ability to produce diverse and robust density functional theory training data sets for MLIPs. We also discuss the generation of initial structures which are either from known structures or from random structures in detail to further formalize the structure-sourcing processes in the future. The polymorph-rich zirconium-oxygen composition space is used as a rigorous benchmark system for comparing the performance of MLIPs trained on structures generated from these structural evolution methods. Using Behler–Parrinello neural networks as our MLIP models, we find that CE and the DM searches are generally superior to MD in terms of spatial descriptor diversity and statistical accuracy.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Benchmarking machine learning strategies for phase-field problems

Abstract We present a comprehensive benchmarking framework for evaluating machine-learning approaches applied to phase-field problems. This framework focuses on four key analysis areas crucial for assessing the performance of such approaches in a systematic and structured way. Firstly, interpolation tasks are examined to identify trends in prediction accuracy and accumulation of error over simulation time. Secondly, extrapolation tasks are also evaluated according to the same metrics. Thirdly, the relationship between model performance and data requirements is investigated to understand the impact on predictions and robustness of these approaches. Finally, systematic errors are analyzed to identify specific events or inadvertent rare events triggering high errors. Quantitative metrics evaluating the local and global description of the microstructure evolution, along with other scalar metrics representative of phase-field problems, are used across these four analysis areas. This benchmarking framework provides a path to evaluate the effectiveness and limitations of machine-learning strategies applied to phase-field problems, ultimately facilitating their practical application.

36 MATERIALS SCIENCE↗

3-regular three-XORSAT planted solutions benchmark of classical and quantum heuristic optimizers

With current semiconductor technology reaching its physical limits, special-purpose hardware has emerged as an option to tackle specific computing-intensive challenges. Optimization in the form of solving quadratic unconstrained binary optimization problems, or equivalently Ising spin glasses, has been the focus of several new dedicated hardware platforms. These platforms come in many different flavors, from highly-efficient hardware implementations on digital-logic of established algorithms to proposals of analog hardware implementing new algorithms. In this work, we use a mapping of a specific class of linear equations whose solutions can be found efficiently, to a hard constraint satisfaction problem (three-regular three-XORSAT, or an Ising spin glass) with a 'golf-course' shaped energy landscape, to benchmark several of these different approaches. We perform a scaling and prefactor analysis of the performance of Fujitsu's digital annealer unit (DAU), the D-Wave advantage quantum annealer, a virtual MemComputing machine, Toshiba's simulated bifurcation machine (SBM), the SATonGPU algorithm from Bernaschi et al, and our implementation of parallel tempering. We identify the SATonGPU and DAU as currently having the smallest scaling exponent for this benchmark, with SATonGPU having a small scaling advantage and in addition having by far the smallest prefactor thanks to its use of massive parallelism. Furthermore, our work provides an objective assessment and a snapshot of the promise and limitations of dedicated optimization hardware relative to a particular class of optimization problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Fundamental effective temperature measurements for eclipsing binary stars – IV. Selection of new benchmark stars and first results for HD 22064

I describe the selection and initial characterization of 20 eclipsing binary stars that are suitable for calibration and testing of stellar models and data analysis algorithms used by the PLATO mission and spectroscopic surveys. The binary stars selected are F-/G-type dwarf stars with M-type dwarf companions that contribute less than 2 per cent of the flux at optical wavelengths. The light curves typically show well-defined total eclipses with very little variability between the eclipses. I have used near-infrared spectra obtained by the APOGEE survey to measure the spectroscopic orbit for both stars in HD 22064. Combined with an analysis of the TESS light curve, I derive the following masses and radii: M 1 = 1.35 ± 0.03M ⊙ , M 2 = 0.58 ± 0.01M ⊙ , R 1 = 1.554 ± 0.014R ⊙ , R 2 = 0.595 ± 0.008R ⊙ . Using R 1 and the parallax from Gaia EDR3, I find that the primary star’s angular diameter is θ = 0.1035 ± 0.0009 mas. The apparent bolometric flux of the primary star is ${\mathcal {F}}_{{\oplus },0} = (7.51\pm 0.09)\times 10^{-9}$ erg cm -2 s -1 . Hence, this F2V star has an effective temperature $T_{\rm eff,1} = 6763{\rm \, K} \pm 39{\rm \, K}$. HD 22064 is an ideal benchmark star that can be used for ‘end-to-end’ tests of the stellar parameters measured by large-scale spectroscopic surveys or stellar parameters derived from asteroseismology with PLATO. The techniques described here for HD 22064 can be applied to the other eclipsing binaries in the sample in order to create an all-sky network of such benchmark stars.

79 ASTRONOMY AND ASTROPHYSICS↗

Real singlet scalar benchmarks in the multi-TeV resonance regime

Scalar extensions of the Standard Model (SM) are of much interest at the Large Hadron Collider (LHC) and future colliders. In particular, these models can give rise to resonant di-Higgs production and alter the Higgs trilinear coupling. In this paper, we study di-Higgs production in the Standard Model extended by a real scalar singlet with no additional symmetries. We determine how large the resonant di-Higgs rate and variation in the Higgs trilinear coupling can be in four scenarios: current LHC results and projected results at the high luminosity LHC (HL-LHC), the HL-LHC combined with a circular 𝑒 − ⁢𝑒 + collider such as the Circular Electron Positron Collider or Future Circular Collider with electron-positron collisions, and the HL-LHC combined with a linear 𝑒 − ⁢𝑒 + collider such as the International Linear Collider. While these are updated results from a previous study by [I. M. Lewis and M. Sullivan, Benchmarks for double Higgs production in the singlet extended standard model at the LHC, Phys. Rev. D 96, 035037 (2017).] using current LHC data, we go further and find benchmark points in the multi-TeV resonance regime for future colliders beyond the HL-LHC. Considering current LHC results, the resonant di-Higgs rate can still be an order of magnitude larger than the SM predicted di-Higgs rate. In the HL-LHC scenario, the Higgs trilinear coupling can still be a factor of three larger than the SM prediction for resonance masses in the 1.5–3.5 TeV range, where resonant searches may have less reach. This enhancement is just at the projected 2⁢𝜎 sensitivity of the HL-LHC. We find there are resonance masses for which the change in the Higgs trilinear is maximized while the resonant rate is negligible. We provide an analytical understanding of these effects with a discussion on the interplay of various constraints on the parameter space and the Higgs trilinear coupling.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Benchmarking characterization methods for noisy quantum circuits

Effective methods for characterizing the noise in quantum computing devices are essential for programming and debugging circuit performance. Existing approaches vary in the information obtained as well as the amount of quantum and classical resources required, with more information generally requiring more resources. Here we benchmark the characterization methods of gate set tomography, Pauli channel noise reconstruction, and empirical direct characterization for developing models that describe noisy quantum circuit performance on a 27-qubit superconducting transmon device. We evaluate these models by comparing the accuracy of noisy circuit simulations with the corresponding experimental observations. Here, we find that the agreement of noise model to experiment does not correlate with the information gained by characterization and that the underlying circuit strongly influences the best choice of characterization approach. Empirical direct characterization scales best of the methods we tested and produced the most accurate characterizations across our benchmarks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Scalar quantum field theories as a benchmark for near-term quantum computers

Quantum field theory (QFT) simulations are a potentially important application for noisy intermediate scale quantum (NISQ) computers. The ability of a quantum computer to emulate a QFT therefore constitutes a natural application-centric benchmark. Foundational quantum algorithms to simulate QFT processes rely on fault-tolerant computational resources, but to be useful on NISQ machines, error-resilient algorithms are required. Here we outline and implement a hybrid algorithm to calculate the lowest energy levels of the paradigmatic 1+1–dimensional $\phi$ 4 interacting scalar QFT. We calculate energy splittings and compare results with experimental values obtained on currently available quantum hardware. We show that the accuracy of mass-renormalization calculations represents a useful metric with which near-term hardware may be benchmarked. Finally, we also discuss the prospects of scaling the algorithm to full simulation of interacting QFTs on future hardware.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Ab initio benchmarks of neutrinoless double- β decay in light nuclei with a chiral Hamiltonian

Here, we report ab initio benchmark calculations of nuclear matrix elements (NMEs) for neutrinoless double-β (0νββ) decays in light nuclei with mass numbers ranging from A = 6 to A = 22. We use the transition operator derived from light-Majorana neutrino exchange and evaluate the NME with three different methods: two variants of in-medium similarity renormalization group (IMSRG) and importance-truncated no-core shell model (IT-NCSM). The same two plus three nucleon interaction from chiral effective field theory is employed, and both isospin-conserving (ΔT = 0) and isospin-changing (ΔT = 2) transitions are studied. We compare our resulting ground-state energies and NMEs to those of recent ab initio no-core shell model and coupled-cluster calculations, also with the same inputs. We show that the NMEs of ΔT = 0 transitions are in good agreement among all calculations, at the level of 10%. For ΔT = 2, relative deviations are more significant in some nuclei. The comparison with the exact IT-NCSM result allows us to analyze these cases in detail, and indicates the next steps toward improving the IMSRG-based approaches. The present study clearly demonstrates the power of consistent cross checks that are made possible by ab initio methodology. This capability is crucial for providing meaningful many-body uncertainties in the NMEs for the 0νββ decays in heavier candidate nuclei, where quasiexact benchmarks are not available.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Benchmarking a nonequilibrium approach to photon emission in relativistic heavy-ion collisions

In this study, the production of photons through binary scattering processes is investigated for equilibrated hadronic systems. More precisely, a nonequilibrium hadronic transport approach to describe relativistic heavy-ion collisions is benchmarked with respect to photon emission. Cross sections for photon production in π + ρ → π + γ and π + π → ρ + γ scattering processes are derived from an effective chiral field theory and implemented into the hadronic transport approach, SMASH (Simulating Many Accelerated Strongly-interacting Hadrons). The implementation is verified by systematically comparing the thermal photon rate to theoretical expectations. Further, the impact of form factors is discussed, scattering processes mediated by ω mesons are found to contribute significantly to the total photon production. Several comparisons of the yielded photon rates are performed: to parametrizations of the very same rates as used in hydrodynamic simulations, to previous works relying on different cross sections for the production of direct photons from the hadronic stage, and to partonic rates. Finally, the impact of considering the finite width of the ρ meson is investigated, where a significant enhancement of photon production in the low-energy region is observed. This benchmark is the first step toward a consistent treatment of photon emission in hybrid hydrodynamics + transport approaches and toward a genuine dynamical description.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Simulating and benchmarking neutron total scattering instrumentation from inception of events to reduced and fitted data

In the design and realization of modern neutron scattering instrumentation, particularly when designing beamline concepts from the ground up, it is desirable to fully benchmark against realistically simulated data. This is especially true for total scattering beamlines, where the future deliverable data is to be analysed in both reciprocal- and real-space representations, and needs must be carefully balanced to ensure sufficient range, resolution and flux of the instrument. An approach to optimize the design of neutron scattering instrumentation via a workflow including ray-tracing simulations, event-based data reduction, heuristic analysis and fitting against realistically simulated spectra is demonstrated here. The case of the DISCOVER beamline concept at the Spallation Neutron Source is used as an example. The results of the calculations are benchmarked through simulation of existing instrumentation and subsequent direct comparison with measured data. Finally, on the basis of the validated models, the ability to explore design characteristics for future beamline concepts or future instrument improvements is demonstrated through the examples of detector tube size and detector layout.

47 OTHER INSTRUMENTATION↗

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗