Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,621 records · Page 90

Verification of the Time-dependent Transport Code TDKENO with the Monte Carlo Code KENO-VI

In this paper we focus on the computer code TDKENO that employs a hybrid methodology for solving the time dependent, three-dimensional Boltzmann transport equation. Originally, TDKENO used the Monte Carlo Transport code KENO V.a from SCALE to calculate flux shapes and fixed sources. KENO V.a is quick to compute these quantities but is limited to simple geometries. For this work TDKENO has been modified to support the Monte Carlo Transport code KENO-VI in addition to KENO V.a. KENO-VI allows generalized geometry input not possible in KENO V.a inputs. In order to verify the changes we ran TDKENO using KENO V.a and KENO-VI inputs then compared the results. Three different inputs were analyzed, the 16a1 benchmark,a simple TREAT model, and a transient model (2855) from the M8 calibration tests performed at INL.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Temporal Graph Generation Based on a Distribution of Temporal Motifs

Generating a synthetic graph that is similar to a given real-world graph is a critical requirement for privacy preservation and benchmarking purposes. Various generative models attempt to generate static graphs similar to real-world graphs. However, generation of temporal graphs is still an open research area. We present a temporal-motif based approach to generate synthetic temporal graph datasets and results from three real-world use cases.

Purohit, Sumit↗

Understanding the Impact of Dynamic Power Capping on Application Progress

Electrical power has become an important design constraint in high-performance computing (HPC) systems. On future HPC machines, power is likely to be a budgeted resource and thus managed dynamically. Power management software needs to reliably measure application performance at runtime in order to respond effectively to changes in application behavior. Execution time tells us little about how the science in the application is progressing toward an application-defined end goal. To the best of our knowledge, no study has defined or categorized online application progress in the context of power management. Based on semi-structured interviews with HPC application-specialists, we define an online notion of progress—an application-specific metric that can be monitored at runtime to provide a sense of the rate at which application science is being performed. Using instrumentation, we characterize and categorize the progress of various production scientific applications and benchmarks. We propose a model of the impact of dynamic power capping on application progress. By experimental evaluation, we show that our model accurately captures the general behavior of the progress of different classes of applications under a power cap. We believe that such a model is an important first step toward the design of more dynamic power management policies for HPC systems.

application progress↗

Why Is Attention Sparse In Particle Transformer?

Transformer-based models have achieved state-of-the-art performance in jet tagging at the CERN Large Hadron Collider (LHC), with the Particle Transformer (ParT) representing a leading example of such models. A striking feature of ParT is its sparse, nearly binary, attention structure, raising questions about the origin of this behavior and whether it encodes physically meaningful correlations. In this work, we investigate the source of ParT's sparse attention by comparing models trained on multiple benchmark datasets and examine the relative contributions of the attention term and the physics-inspired interaction matrix before softmax. We find that binary sparsity arises primarily from the attention mechanism itself, with the interaction matrix playing a secondary role. Moreove, we show that ParT is able to identify key jet substructure elements, such as leptons in semileptonic top decays, even without explicit particle identification inputs. These results provide new insight into the interpretability of transformer-based jet taggers and clarify the conditions under which sparse attention patterns emerge in ParT.

Legge, Timothy [UC, San Diego]↗

Beyond Point Estimates: Benchmarking Uncertainty Quantification Methods on the AION-1 Astronomical Foundation Model

Foundation models for astronomical surveys offer powerful learned representations that can be transferred to downstream regression tasks such as galaxy property estimation. However, point predictions alone are insufficient for scientific inference; reliable uncertainty quantification (UQ) is essential. We compare seven UQ methods on galaxy property regression using frozen AION-1 foundation-model embeddings, predicting redshift, stellar mass, stellar-population age, gas-phase metallicity, and specific star-formation rate, from Legacy Survey photometry/imaging and DESI spectra, with PROVABGS-derived labels. Distribution-free conformal methods achieve marginal coverage within $\sim$1 pp of the nominal 90% across all properties, while non-conformal baselines (Deep Ensembles, MC~Dropout) fail to calibrate reliably. Among conformal approaches, Conformalized Quantile Regression (CQR) delivers the best coverage in the bin with the poorest model predictions. More importantly, only the Locally Valid and Discriminative (LVD) framework -- particularly when operating on AION-1 embeddings -- also provides finite-sample \emph{local validity}, producing intervals that adapt to each galaxy's local prediction difficulty rather than relying on marginal guarantees alone. These results establish conformal prediction, and LVD in particular, as the preferred UQ framework for uncertainty-aware inference on foundation-model embeddings in astrophysics.

Tame-Narvaez, Karla [Fermilab] (ORCID:000000022249↗

FY 2008 Annual Progress Report for Advanced Vehicle Technology Analysis and Evaluation Activities and Heavy Vehicle Systems Optimization Program

AVTAE provides an overarching vehicle systems perspective in support of the technology R&D activities of DOE’s VT and Hydrogen, Fuel Cells & Infrastructure Technologies (HFCIT) Programs. AVTAE uses analytical and empirical tools to model and simulate potential vehicle systems, validate component performance in a systems context, verify and benchmark emerging technology, and validate computer models. Hardware-in-theloop testing allows components to be controlled in an emulated vehicle environment. Laboratory testing then provides measurement of progress toward VT technical goals and eventual validation of DOE-sponsored technologies at the Advanced Powertrain Research Facility for light- and medium-duty vehicles and at the ReFUEL Facility for heavy-duty vehicles. For this sub-program to be successful, extensive collaboration with the technology development activities within the VT and HFCIT Programs is required for both analysis and testing.

33 ADVANCED PROPULSION SYSTEMS↗

Benchmarking the variational cluster approach by means of the one-dimensional Bose-Hubbard model

Convergence properties of the variational cluster approach with respect to the variational parameter space, cluster size, and boundary conditions of the reference system are investigated and discussed for bosonic many-body systems. Specifically, the variational cluster approach is applied to the one-dimensional Bose-Hubbard model, which exhibits a quantum phase transition from Mott to superfluid phase. In order to benchmark the variational cluster approach, results for the phase boundary delimiting the first Mott lobe are compared with essentially exact density matrix renormalization group data. Furthermore, static quantities, such as the ground state energy and the one-particle density matrix are compared with high-order strong coupling perturbation theory results. For reference systems with open boundary conditions the variational parameter space is extended by an additional variational parameter which allows for a more uniform particle density on the reference system and thus drastically improves the results. It turns out that the variational cluster approach yields accurate results with relatively low-computational effort for both the phase boundary as well as the static properties of the one-dimensional Bose-Hubbard model, even at the tip of the first Mott lobe where correlation effects are most pronounced.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A performance model for GPUs with caches

To exploit the abundant computational power of the world's fastest supercomputers, an even workload distribution to the typically heterogeneous compute devices is necessary. While relatively accurate performance models exist for conventional CPUs, accurate performance estimation models for modern GPUs do not exist. This paper presents two accurate models for modern GPUs: a sampling-based linear model, and a model based on machine-learning (ML) techniques which improves the accuracy of the linear model and is applicable to modern GPUs with and without caches. We first construct the sampling-based linear model to predict the runtime of an arbitrary OpenCL kernel. Based on an analysis of NVIDIA GPUs' scheduling policies we determine the earliest sampling points that allow an accurate estimation. The linear model cannot capture well the significant effects that memory coalescing or caching as implemented in modern GPUs have on performance. We therefore propose a model based on ML techniques that takes several compiler-generated statistics about the kernel as well as the GPU's hardware performance counters as additional inputs to obtain a more accurate runtime performance estimation for modern GPUs. We demonstrate the effectiveness and broad applicability of the model by applying it to three different NVIDIA GPU architectures and one AMD GPU architecture. On an extensive set of OpenCL benchmarks, on average, the proposed model estimates the runtime performance with less than 7 percent error for a second-generation GTX 280 with no on-chip caches and less than 5 percent for the Fermi-based GTX 580 with hardware caches. On the Kepler-based GTX 680, the linear model has an error of less than 10 percent. On an AMD GPU architecture, Radeon HD 6970, the model estimates with 8 percent of error rates. As a result, the proposed technique outperforms existing models by a factor of 5 to 6 in terms of accuracy.

97 MATHEMATICS AND COMPUTING↗

Transition prediction in hypersonic regime on complex geometries with RANS-based models

In the near future, RANS computations will continue to play a significant role in the design of hypersonic vehicles with complex geometries. Therefore, it is imperative to continue testing, benchmarking, and refining the RANS models. In the present study, we evaluate RANS-like, transport equations-based models for predicting laminar-turbulent transition over a full-scale scale model of the BOLT flight configuration that was tested in the CUBRC LENS-II wind tunnel test facility. Based on the availability of the experimental results, comparisons are made between computations from several pre-existing transition models and Computational Fluid Dynamics (CFD) codes, with an emphasis on using the same computational meshes and flow conditions for all computations. The analysis covers the sensitivity of the transition predictions to the input parameters for five different transition models (four mainstream models designed for low-speed flows and a uniquely high-speed model), grid resolution, and the details of model implementation across three different flow solvers. The results show that the phenomenological models can describe significant aspects of the measured transition front. However, a number of additional improvements are required before these models can offer more reliable estimates of transition in high-speed flows.

CFD↗

Transition Prediction in Hypersonic Regime on Complex Geometries With Rans-Based Models

In the near future, RANS computations will continue to play a significant role in the design of hypersonic vehicles with complex geometries. Therefore, it is imperative to continue testing, benchmarking, and refining the RANS models. In the present study, we evaluate RANS-like, transport equations-based models for predicting laminar-turbulent transition over a full-scale scale model of the BOLT flight configuration that was tested in the CUBRC LENS-II wind tunnel test facility. Based on the availability of the experimental results, comparisons are made between computations from several pre-existing transition models and Computational Fluid Dynamics (CFD) codes, with an emphasis on using the same computational meshes and flow conditions for all computations. The analysis covers the sensitivity of the transition predictions to the input parameters for five different transition models (four mainstream models designed for low-speed flows and a uniquely high-speed model), grid resolution, and the details of model implementation across three different flow solvers. The results show that the phenomenological models can describe significant aspects of the measured transition front. However, a number of additional improvements are required before these models can offer more reliable estimates of transition in high-speed flows.

CFD↗

Neutron Number Probability Distributions in a Subcritical System Using the Forward Master Equation

The subject of this paper is the determination of the equilibrium probability distribution function (PDF) of the neutron population within a subcritical multiplying system in the presence of an intrinsic random source as a result of spontaneous fission (SF). Such systems are strongly stochastic and low order moments such as the mean and variance of the neutron number are not sufficient to characterize the true state of the neutron population. We base our work on the forward Master equation in a lumped model formulation for P{sub n} (t), n = 0, 1, 2,..., the probability of there existing η neutrons at time t and obtain solutions for the steady state distributions. The key parameters in this model, neutron lifetime and fission probability, are rigorously obtained by numerically solving the k-eigenvalue problem in a one-dimensional spherical system and using standard definitions of lumped parameters in terms of production and loss rates in finite systems. Previous work has examined single-chain PDFs (a single fission and the resulting progeny) for neutrons, uncoupled neutron-photon single-chain PDFs, both using a lumped backward Master equation formulation, as well as neutron PDFs in supercritical systems. In this paper, we analyze strictly subcritical systems, for which we compare analytically derived solutions with numerical computations for different multiplicity distribution models. For subcritical systems where the neutron population remains finite at steady state, we show that the forward Master equation can be very efficiently solved numerically to obtain the equilibrium distributions. Finally, this work is motivated by potential application to passive interrogation for detection of diverted nuclear materials as well to criticality safety. The equilibrium neutron number probability distribution in subcritical systems has been numerically investigated for several models of the induced and spontaneous fission multiplicity distribution. A well known approximate but closed form solution, namely the quadratic model, was shown to be unacceptably accurate for subcritical media. The matrix inversion and the recursion methods agree well and were benchmarked with the binary fission model, but the recursion formulation proves far more computationally efficient. A qualitative change in the shape of the neutron PDF is observed for increasing source strength, as expected, and a transition from a stochastic to a deterministic system is expected for a sufficiently strong source. We are currently extending this approach to obtain the equilibrium joint distribution of neutrons within the system and those that have leaked, to explicitly represent the neutron PDF as observed by an external detector. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Order and correlation contributions to the entropy of hydrophobic solvation

The entropy of hydrophobic solvation has been explained as the result of ordered solvation structures, of hydrogen bonds, of the small size of the water molecule, of dispersion forces, and of solvent density fluctuations. We report a new approach to the calculation of the entropy of hydrophobic solvation, along with tests of and comparisons to several other methods. The methods are assessed in the light of the available thermodynamic and spectroscopic information on the effects of temperature on hydrophobic solvation. Five model hydrophobes in SPC/E water give benchmark solvation entropies via Widom’s test-particle insertion method, and other methods and models are tested against these particle-insertion results. Entropies associated with distributions of tetrahedral order, of electric field, and of solvent dipole orientations are examined. We find these contributions are small compared to the benchmark particle-insertion entropy. Competitive with or better than other theories in accuracy, but with no free parameters, is the new estimate of the entropy contributed by correlations between dipole moments. Dipole correlations account for most of the hydrophobic solvation entropy for all models studied and capture the distinctive temperature dependence seen in thermodynamic and spectroscopic experiments. Entropies based on pair and many-body correlations in number density approach the correct magnitudes but fail to describe temperature and size dependences, respectively. Hydrogen-bond definitions and free energies that best reproduce entropies from simulations are reported, but it is difficult to choose one hydrogen bond model that fits a variety of experiments. The use of information theory, scaled-particle theory, and related methods is discussed briefly. Our results provide a test of the Frank-Evans hypothesis that the negative solvation entropy is due to structured water near the solute, complement the spectroscopic detection of that solvation structure by identifying the structural feature responsible for the entropy change, and point to a possible explanation for the observed dependence on length scale. Our key results are that the hydrophobic effect, i.e. the signature, temperature-dependent, solvation entropy of nonpolar molecules in water, is largely due to a dispersion force arising from correlations between rotating permanent dipole moments, that the strength of this force depends on the Kirkwood g-factor, and that the strength of this force may be obtained exactly without simulation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Nonequilibrium Steady-State Transport in Quantum Impurity Models: A Thermofield and Quantum Quench Approach Using Matrix Product States

Here, the numerical renormalization group (NRG) is tailored to describe interacting impurity models in equilibrium, but it faces limitations for steady-state nonequilibrium, arising, e.g., due to an applied bias voltage. We show that these limitations can be overcome by describing the thermal leads using a thermofield approach, integrating out high energy modes using NRG, and then treating the nonequilibrium dynamics at low energies using a quench protocol, implemented using the time-dependent density matrix renormalization group. This yields quantitatively reliable results for the current (with errors ≲3%) down to the exponentially small energy scales characteristic of impurity models. We present results of benchmark quality for the temperature and magnetic field dependence of the zero-bias conductance peak for the single-impurity Anderson model.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Coherent dynamic structure factors of strongly coupled plasmas: A generalized hydrodynamic approach

A generalized hydrodynamic fluctuation model is proposed to simplify the calculation of the dynamic structure factor S(ω, k) of non-ideal plasmas using the fluctuation-dissipation theorem. In this model, the kinetic and correlation effects are both included in hydrodynamic coefficients, which are considered as functions of the coupling strength (Γ) and collision parameter (kλ{sub ei}), where λ{sub ei} is the electron-ion mean free path. A particle-particle particle-mesh molecular dynamics simulation code is also developed to simulate the dynamic structure factors, which are used to benchmark the calculation of our model. A good agreement between the two different approaches confirms the reliability of our model.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DECOVALEX-2023: Task F1 Final Report

DECOVALEX-2023 Task F is a comparison of models and methods for post-closure performance assessment (PA) of a deep geologic repository for radioactive waste. The general aims of Task F are to build confidence in the models, methods, and software used for PA and to stimulate additional research and development in PA methodologies. The task objectives are to motivate development of PA modelling skills and capabilities, to examine the influence of model choices on calculated repository performance, and to compare the uncertainties introduced by model choices to other sources of uncertainty. Task F involves no actual experiment or site. It is a PA modelling exercise that requires the conceptual development of hypothetical repository designs and geologic settings. Because three of the teams were interested in salt and the rest of the teams were interested in crystalline rock, Task F was split into two branches: Task F1 for crystalline rock and Task F2 for salt. This report is for Task F1, crystalline rock. Teams from seven countries (Canada, Czech Republic, Germany, Korea, Sweden, Taiwan, and United States) participated in Task F1. The teams worked together to define the features, events, and processes of the reference case repository and established a set of performance measures. In addition, they defined a set of benchmark problems designed to test and compare modelling capabilities for fracture flow and transport at different scales. The repository design and benchmark problems are documented in a Task Specification that evolved over time as the group honed the specifications. The benchmark problems verified that each team can aptly model flow and transport in fractured media in 1-, 2-, and 3-dimensions. Two general approaches were used for the 3-dimensional benchmarks: discrete fracture network (DFN) and equivalent continuous porous medium (ECPM). DFN modelling involves explicit meshing of each fracture while ECPM modelling aims to capture the effective porosity and directional permeability of each cell in a space-filling mesh as affected by intersecting fractures. In some models, a combination of the two is used, i.e., DFN for large known fractures and ECPM for the rest of the domain. Transport is solved by using either the advection-dispersion equation or particle tracking. Although some variation is observed among model breakthrough curves in the benchmark problems, there is strong agreement in breakthrough behaviour up to at least the 75 th percentile for all benchmarks. At the 90 th percentile, breakthrough results show larger differences, suggesting several models retain substantially higher fractions of tracer in regions of slower moving water. In addition to the flow and transport benchmarks, several teams completed the source term benchmark, verifying capabilities for modelling radionuclide decay and ingrowth, waste package breach, instant release fractions, fuel matrix degradation rates, and radionuclide solubility limitations. The reference case is conceptualized as a generic spent fuel repository at a depth of 450 m in fractured crystalline rock. The repository has 50 parallel backfilled drifts, each with 50 deposition holes 6 m apart. Each deposition hole contains a 4-PWR waste package and bentonite buffer. The rock domain is 5 km in length, 2 km in width, and 1 km in depth. It has 6 deterministic fractured deformation zones and a multitude of stochastic fractures. Teams generally used the ECPM approach for the entire rock or a hybrid approach in which the deterministic fracture zones are modelled with a DFN and the rest of the rock is modelled by ECPM. Of the reference case problems specified, only the results of the initial reference case problem are compared in this report. The initial problem focuses on transport from the deposition holes to the surface, i.e., it neglects waste package performance. Tracers are released at all waste package locations at time zero and tracked for their releases to the near field and ground surface. The water fluxes calculated at the ground surface entry and exit regions of the domain are similar for all models except for two that have considerably lower fluxes. For tracer transport, large differences are observed among models in the magnitude of tracer transported. Much of the difference appears to be due to how the repository is implemented and hence the different degrees of repository simplification. Models that exclude the drifts, buffer, and backfill from the domain tend to show greater release of tracers and radionuclides from the repository. The initial study presented here indicates that major differences in modelling important processes within the repository (e.g., diffusion through buffer and backfill) can produce broadly different release and transport results, especially when those processes are excluded. Even for the models that included all specified features, events, and processes, the results show significant differences and demonstrate the importance of examining multiple modelling approaches in performance assessment. The differences in results observed in this study are expected to motivate teams to either increase complexity in future versions of the reference case models or to improve methods to account for the effects of simplified features and processes. Either way, future improvements in these models are expected to produce results that more closely agree.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Supersymmetric discovery potential and benchmarks for early runs at {radical}(s)=7 TeV at the LHC

We carry out an analysis of the potential of the Large Hadron Collider (LHC) to discover supersymmetry in runs at {radical}(s)=7 TeV with an accumulated luminosity of (0.1-2) fb{sup -1} of data. The analysis is done with both minimal supergravity and supergravity models with nonuniversal soft breaking. Benchmarks for early discovery with (0.1-2) fb{sup -1} of data are given. We provide an update of b-tagging efficiencies in PGS 4 appropriate for LHC analyses. A large number of signature channels are analyzed, and it is shown that each of the models exhibited are discoverable at the 5{sigma} level or more above the standard model background in several signature channels which would provide cross checks for a discovery of supersymmetry. It is shown that some of the benchmarks are discoverable with 0.1 fb{sup -1} of data again with detectable signals in several channels.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗