Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

Quantum-Hardware Focused Application Performance Benchmarks (Final Technical Report)

Quantum computers promise to transform how we do scientific calculations. In this project, we benchmark different current quantum computers for solving chemistry problems and find ways to build noise-resilient implementations. We also use the characterization techniques developed to test ion trap quantum computers.

74 ATOMIC AND MOLECULAR PHYSICS↗

Drive Cycles, Battery Pack Scaling, and Usage Considerations for Long-Haul and Regional-Haul Electric Trucks

Electrifying Class-8 heavy-duty trucks presents a promising opportunity to enhance energy efficiency and reduce freight transport costs. Battery electric trucks (BETs), once considered niche, are gaining traction due to advancements in battery technology and cost reductions. However, accurately predicting battery lifespan under realistic usage conditions remains a key challenge. Understanding battery failure mechanisms and their links to design, operation, and management is essential for developers and fleet operators. This study introduces a method to develop simplified, lab-testable dynamic stress test (DST) cycles for regional and long-haul Class-8 BETs, derived from real-world diesel truck usage. These DSTs enable benchmarking of battery technologies, identification of aging stressors, and optimization of battery design, life, and cost. The approach supports evaluation of key metrics such as levelized cost of driving and total cost of ownership, aiding fair comparisons and adoption decisions. We also propose feasible battery pack sizes that meet current driving demands with strategic charging, and a method to scale pack-level DSTs to cell-level cycles for lab-based testing. These tools facilitate tradeoff analysis across battery chemistries, pack sizing, and charging strategies, while offering means to get insights into battery aging under realistic conditions-ultimately supporting informed BET deployment decisions.

25 ENERGY STORAGE↗

Benchmark estimate of the effect of anthropogenic emissions on the ocean surface

Abstract Investigations into the role of anthropogenic emissions in the occurrence of extreme weather often use a method that compares simulations of atmospheric climate models run under a factual scenario of historical boundary conditions observed during the period of the event against simulations run under a counterfactual scenario of what those boundary conditions might naturally have been over that same period in the absence of anthropogenic emissions. A particular requirement for this experiment design is an accurate estimation of ocean surface boundary conditions for use by the counterfactual natural simulations. Here we use output from the CMIP5 multi‐climate‐model archive to develop a robust estimate of sea surface temperatures and sea ice conditions for use in counterfactual natural simulations, intended as a benchmark estimate to facilitate comparison across climate models and across studies. This development includes tests to ensure that the final estimate is stable from year‐to‐year and stable against other perturbations to the methodology, as well as consideration of the strengths and weaknesses in comparison to other available attributable warming estimates. While this estimate is tailored specifically for the International CLIVAR C20C+ Detection and Attribution Project, it can be used by related projects as well.

Stone, Dáithí A.↗

Reactive Transport Modeling with Physics-Informed Machine Learning for Critical Minerals Applications

This study presents a physics-informed neural network (PINN) framework for reactive transport modeling for simulating fast bimolecular reactions in porous media. Accurate characterization of cAhemical interactions and product formation in surface and subsurface environments is essential for advancing critical mineral extraction and related geoscience applications. The proposed methodology sequentially addresses the flow and diffusion–reaction subproblems. The flow field is computed using a mixed formulation, while the diffusion–reaction system is modeled via two uncoupled tensorial diffusion equations reformulated in terms of chemical invariants. PINNs are employed to solve the governing equations, enabling data-efficient, mesh-free prediction of chemical concentration fields. The framework is validated through a series of benchmark problems involving flow in heterogeneous porous media. Initial verification is conducted using patch tests for the flow field, followed by validation of the transport problem with emphasis on preserving non-negativity of concentrations. The complete fast bimolecular reaction scenario is then solved, yielding spatial distributions of reactants and product species. Results demonstrate that the PINNs-based approach effectively captures sharp, mixing-limited reaction fronts and dispersive mixing behavior, offering reliable predictions of reactive plume evolution. These capabilities are crucial for evaluating long-term subsurface behavior in applications such as fluid storage, energy extraction, and efficient extraction of critical minerals.

42 ENGINEERING↗

Unimolecular Dynamics of Partially Deuterated Hydroperoxyalkyl Intermediates (•QOOD) in Cyclohexane and Cyclopentane Oxidation

The hydroperoxyalkyl radicals (•QOOH) formed in cyclohexane and cyclopentane oxidation with carbon radical center at the β site relative to the hydroperoxy group have recently been observed through their infrared fingerprint and time- and energy-resolved unimolecular dissociation dynamics to hydroxyl (OH) radical and cyclic ether products. Partial deuteration shifts the IR transitions associated with OH/OD motion to lower frequencies, providing a new window on the IR spectroscopy and unimolecular dynamics of these transient intermediates. Specifically, the overtone OD stretch (2ν OD ) has been identified at 5215.0 cm –1 and 5216.0 cm –1 for the β-QOOD intermediates in cyclohexane and cyclopentane oxidation, respectively. Unimolecular decay rates of the •QOOD intermediates at these energies have been obtained from the time-resolved appearance of OD products. The experimental rates are compared with statistical microcanonical rates evaluated using RRKM theory, including heavy-atom tunneling associated with simultaneous O–O bond elongation and C–O–O angle contraction along the reaction pathway. The experimental rates provide a further test of the previously determined transition state (TS) barriers, which were computed utilizing a benchmarking approach that builds on higher-level reference calculations for the smaller ethane oxidation system. Here, the experimental rate measurements for both •QOOD intermediates agree well with the computed rates after accounting for small changes in zero-point energy and a minor empirical adjustment of the TS barriers in both systems.

Ethers↗

Efficient Training of Deep Neural Operator Networks via Randomized Sampling

Neural operators (NOs) employ deep neural networks to learn the mappings between infinitedimensional function spaces. Deep operator network (DeepONet), a popular NO architecture, has demonstrated success in the real-time prediction of complex dynamics across various scientific and engineering applications. In this work, we introduce a random sampling technique to be adopted during the training of DeepONet, aimed at improving the generalization ability of the model, while significantly reducing the computational time. The proposed approach targets the trunk network of the DeepONet model that outputs the basis functions corresponding to the spatiotemporal locations of the bounded domain on which the physical system is defined. While constructing the loss function, DeepONet training traditionally considers a uniform grid of spatiotemporal points at which all the output functions are evaluated for each iteration. This approach leads to a larger batch size, resulting in poor generalization and increased memory demands, due to the limitations of the stochastic gradient descent (SGD) optimizer. The proposed random sampling over the inputs of the trunk net mitigates these challenges, improving generalization and reducing the memory requirements during training, resulting in significant computational gains. We validate our hypothesis through three benchmark examples, demonstrating substantial reductions in training time while achieving comparable or lower overall test errors relative to the traditional training approach. Our results indicate that incorporating randomization in the trunk network inputs during training enhances the efficiency and robustness of DeepONet, offering a promising avenue for improving the framework’s performance in modeling complex physical systems.

Karumuri, Sharmila [Department of Civil & Systems ↗

Cumulative $\chi^2$ Metric for VALID for ENDF/B-VII.1 and ENDF/B-VIII.0 in SCALE 6.3b9

Nuclear data testing is generally performed in a manner similar to validation: a series of critical benchmark experiments is modeled, and the differences between calculated and expected results are examined to understand the performance of a code and cross section set. This process is repeated for multiple sets of experiments with different transport codes and sometimes with different datasets. Occasionally, data processing differences are also important in understanding the differences in calculated results. Relevant experts then meet periodically to compare results and set priorities for improvement in the next round of data updates. This process has been largely effective, despite the confounding factors of different benchmark sets, application areas, sponsors, nationalities, and the resulting different priorities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

High-Temperature Gas-Cooled Reactors Multiphysics Simulation Demonstration and Code Validation

This study presents a comprehensive benchmarking and verification effort of several thermal-hydraulic and multiphysics capabilities for high-temperature gas-cooled reactor applications. The first part of this effort focuses on the running-in verification of Griffin’s multiphysics capabilities, specifically for simulating the evolution of pebble-bed reactor cores from startup to equilibrium. Since Fiscal Year 2024, improvements and enhancements have been implemented in Griffin, including simplifying the process to specify streamlines and developing the online cross-section generation capability. In the absence of validation data, code-to-code comparisons are conducted with kugelpy, showing good agreement for integral quantities like k-eff predictions and predictions for maximum power density. However, accuracy issues are noted for more detailed quantities like the spatial distribution of fission rate densities which will require further work to address. The second part of this report presents an improved System Analysis Module (SAM) core channel model where the effects of cross flow are considered during the pressurized loss of forced cooling transient, resulting in an improved agreement of the predicted pebble temperature with respect to the predictions from the SAM 2D porous media model. Additionally, the wall channeling effect due to variable porosity at the near wall region of the core is also investigated. Furthermore, to demonstrate Griffin’s online cross-section generation capability, a Multiphysics simulation is performed by coupling Griffin to the SAM core channel model. In the third part of the report, as a part of the Organisation for Economic Co-operation and Development/Nuclear Energy Agency (OECD/NEA) thermal-hydraulic code validation benchmark activity for a high-temperature gas-cooled reactor, the High Temperature Test Facility (HTTF) is investigated first using the NekRS computational fluid dynamics (CFD) code to study the flow mixing phenomenon in the lower plenum of the facility. Then, code-to-code and code-to-data comparisons are performed for Test PG27, which is a pressurized conduction cooldown (PCC) test, using five different codes by six organizations from five countries. The different simulations show good agreements in terms of the general trend but there are differences in some results such as the peak temperatures of different regions and heat removal rate.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Improvement and Verification of Online Cross Section Generation Capability of Griffin for TRISO-fueled Reactors

Griffin, a MOOSE-based reactor multiphysics code jointly developed by Idaho National Laboratory and Argonne National Laboratory under the DOE Office of Nuclear Energy’s NEAMS program, has pursued the development of an online multigroup cross section generation capability for a few years to enable high-fidelity, problem-dependent neutronics analyses of advanced thermal reactors. Recent advancements in Griffin’s online multigroup cross section generation capability have significantly improved the accuracy, robustness, and efficiency of self-shielding calculations for both prismatic and pebble-bed TRISO-fueled reactor applications. Key developments include a unified fuel self-shielding method applicable to both TRISO and annular compact/spherical shell fuel zone geometries; an advanced Dancoff Category-based Equivalence Theory using a bell function for non-fuel resonance treatment, achieving more than an order-of-magnitude speedup compared to the Tone method; an on-the-fly multigroup equivalence approach to mitigate group condensation errors; and a streaming correction method for pebble-bed homogenization. A proof-of-concept demonstration of on-the-fly group condensation with consistent P0 transport correction was also achieved. The method reproduced direct fine-group solutions with excellent accuracy (eigenvalue errors within 10 pcm and pin-power differences within 0.5%), but due to performance limitations of the current fixed-source solver, improvements to solver efficiency will be addressed in future work. Verification tests were performed on graphite-moderated TRISO-fueled two-dimensional core benchmark problems representing gas-cooled microreactors, heat pipe-cooled microreactors, gas-cooled pebble-bed reactors, and fluoride salt-cooled high-temperature reactors. Across all cases, Griffin showed excellent agreement with Serpent2 continuous energy Monte Carlo solutions: eigenvalue errors within 200 pcm, pin-power root-mean-square errors within 2%, and control rod and drum worth errors less than 2%. It should be noted that, for the benchmark problem, cross section generation contributed less than 3% of the total simulation times. These results demonstrate that Griffin’s online cross section generation capability delivers accurate and efficient reactor physics solutions across a wide spectrum of TRISO-fueled advanced reactor designs. With further improvements to the fine-group fixed-source solver and planned extensions to depletion, transients, and coupled neutron–gamma transport, Griffin will be well-positioned to become a powerful and comprehensive tool for advanced reactor analysis.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Benchmarking Solid-State Batteries Containing Sulfide Separators: Effects of Electrode Composition and Stack Pressure

Integrating sulfide separators into solid-state batteries (SSBs) containing high energy cathodes typically requires one or more materials and engineering solutions including: (i) applying interfacial coatings to mitigate electrolyte decomposition, (ii) applying high stack pressures to form robust solid-solid contacts, and (iii) using alloying anodes to avoid Li dendrite formation. Despite the promise of these approaches, a lack of standardized testing protocols makes it difficult to directly compare results among different studies. To address this problem, the present work benchmarks the performance of SSBs containing β-Li 3 PS 4 (LPS) separators and composite cathodes. By systematically varying the anode/cathode composition and stack pressure, this work demonstrates that cathode design is a major bottleneck for solid-state cells cycled at low rates (<100 µA cm -2 ). Operando stack pressure measurements show that, while mechanical confinement generally promotes higher active material utilization and cycling stability, this strategy alone does not address interfacial reactivity between LPS and high voltage cathodes. Here, these results also demonstrate that stress evolution during cycling is dominated by volume changes at the Li metal anode. Finally, we show that FeS 2 cathodes with moderate operating voltages (<3 V vs Li/Li + ) exhibit superior cycling performance compared to high voltage cathodes by facilitating formation of stable cathode/electrolyte interfaces.

25 ENERGY STORAGE↗

Synthetic, Realistic Transmission and Distribution Co-Simulation for Voltage Control Benchmarking: Preprint

As distributed energy resources play an increasing role in generation and grid services, control and market paradigms will need to be tested with models that incorporate transmission and distribution systems. The test systems will need to be realistic and accessible for researchers to form conclusions about realistic behaviors and maintain comparability and repeatability of results. These test systems also must have high resolution models of sufficient size for realistic distribution and transmission interaction. This paper demonstrates the tightly coupled co-simulation of realistic synthetic distribution and transmission models for the Austin metropolitan area and how this framework can be used to analyze resultant behaviors from distributed resources and their controls. The comparison of a distribution system run with fixed feed-in voltage is compared to the closely tied distribution-transmission co-simulation. It is determined that even without advanced voltage controls, the feed-in voltage response of the transmission system shows a lower level of voltage excursions. Therefore, control methods for voltage stability should be benchmarked on realistic transmission and distribution co-simulations for more accurate assessment of voltage profiles.

27 ARPA - Advanced Research Projects Agency-Energy↗

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real- world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5- based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

azimuth↗

Linearised Fokker–Planck collision model for gyrokinetic simulations

We introduce a gyrokinetic, linearised Fokker–Planck collision model that satisfies conservation laws and is accurate at arbitrary collisionalities. The differential test-particle component of the operator is exact; the integral field-particle component is approximated using a spherical harmonic and a modified Laguerre polynomial expansion developed by Hirshman and Sigmar (1976 Phys. Fluids 19 1532). The numerical methods of the implementation in the δf-gyrokinetic code stella (Barnes et al 2019 J. Comput. Phys. 391 365–80) are discussed, and conservation properties of the operator are demonstrated. The collision model is then benchmarked against the collision model of the gyrokinetic solver GS2 in the limiting cases of a reduced test-particle collision operator and energy- and momentum-conserving operator. The accuracy of the full collision model is investigated by solving the parallel Spitzer-Härm problem for the transport coefficients. It is shown that retaining collisional energy flux and higher-order terms in the field-particle operator reduces errors in the transport coefficients from 10%–25% for a simple momentum- and energy-conserving model to under 1%.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real-world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5-based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

algorithm validation↗

Benchmarking machine learning interatomic potentials via phonon anharmonicity

Abstract Machine learning approaches have recently emerged as powerful tools to probe structure-property relationships in crystals and molecules. Specifically, machine learning interatomic potentials (MLIPs) can accurately reproduce first-principles data at a cost similar to that of conventional interatomic potential approaches. While MLIPs have been extensively tested across various classes of materials and molecules, a clear characterization of the anharmonic terms encoded in the MLIPs is lacking. Here, we benchmark popular MLIPs using the anharmonic vibrational Hamiltonian of ThO 2 in the fluorite crystal structure, which was constructed from density functional theory (DFT) using our highly accurate and efficient irreducible derivative methods. The anharmonic Hamiltonian was used to generate molecular dynamics (MD) trajectories, which were used to train three classes of MLIPs: Gaussian approximation potentials, artificial neural networks (ANN), and graph neural networks (GNN). The results were assessed by directly comparing phonons and their interactions, as well as phonon linewidths, phonon lineshifts, and thermal conductivity. The models were also trained on a DFT MD dataset, demonstrating good agreement up to fifth-order for the ANN and GNN. Our analysis demonstrates that MLIPs have great potential for accurately characterizing anharmonicity in materials systems at a fraction of the cost of conventional first principles-based approaches.

interatomic potentials↗

Concentration and Recovery of Rare Earth Elements from Eastern US Coal Refuse

Recent studies funded by the US Department of energy have shown that coal and coal byproducts contain elevated contents of Rare Earth Elements (REEs), making them a potential resource for these critical materials. The approach employed in this research focused on the concentration and extraction of REEs from fine coal refuse derived from various preparation plants in the Appalachian coal basin of the United States. Initial efforts in this research focused on the identification and characterization of REEs in various fine coal refuse streams from nine distinct industrial preparation plants in Appalachia. The average REE content in these materials was determined to be approximately 200 ppm, but the REE content showed a strong correlation to the aluminum content, suggesting that the REEs are closely associated with the clay minerals present in the refuse. Given the relatively low REE concentrations, initial efforts sought to concentrate the REEs through decarbonization and dispersive liberation steps. In these tests, high-shear agitation in the presence of a polyelectrolyte, followed by sedimentation was able to isolate the REE-enriched fine clay particles from siliceous gangue minerals. Following the dispersive liberation step, all samples were found to have an REE content greater than 300 ppm, a benchmark used for many initial exploratory studies. In one case, the REE content was increased by more than 125%. Subsequent extraction tests initially utilized a direct ion-exchange leaching approach with ammonium sulfate as lixiviant. In all cases, the simple ion-exchange leaching process failed to recover significant quantities of rare earth elements, ultimately suggesting that the REEs in fine coal waste may be passivated or bound in a colloidal phase. To access this colloidal phase, several alternative approaches were evaluated, including leaching with alternative ion-exchange lixiviants, reductive leaching, gas-purged leaching, and others. The approach that showed the most promise was strong alkaline pretreatment, followed by ion-exchange leaching with ammonium sulfate at pH 4. A combination of strong alkali and high-temperatures treatment successfully liberated the REEs, converting them to a form amenable to ion-exchange leaching. The highest REE recovery achieved with this method was determined to be 39%. Lastly, bench-scale solvent extraction tests were used to further concentrate REEs in the leach solution and demonstrate that mixed rare earth concentrates can be successfully produced from fine coal refuse.

01 COAL, LIGNITE, AND PEAT↗

Comparative verification of hydro-mechanical fracture behavior: Task G of international research project DECOVALEX–2023

Numerical simulations become a necessity when experimental approaches cannot cover the required physical and time scale of interest. One of such area is a simulation of long-term host rock behaviors for nuclear waste disposal and simulation tools involved in the assessment must go through rigorous validation tests. The DECOVALEX project (Development of COupled models and their VAlidation against EXperiments) is dedicated to this purpose by international participants. This work is part of the ongoing phase DECOVALEX–2023 (D–2023, Task G) particularly aiming to simulate fracture behaviors under various conditions. Here, in this study, we cross-verified a variety of numerical methods including continuous and discontinuous approaches against four benchmark exercises with emphasis on numerical accuracy and parameterization of the various numerical approaches. The systematic inter-comparisons of test cases highlight advantages and disadvantages of the different numerical models. Numerical details on discretization effects (e.g. mesh density and orientation) and domain size were investigated in detail for practical applications. It became evident that meticulous attention to mesh resolution and domain size is imperative for achieving accurate numerical simulations, even for static cracks. Moreover, when comparing numerical methods to closed-form solutions for static cracks, all models successfully reproduced the maximum crack opening but encountered challenges near the crack tips. Finally, the paper discusses how to convert between and therefore compare parameters of various numerical approaches. Our benchmark studies reveal that each model necessitates a distinct number of parameters, even in simple scenarios like static crack aperture benchmarks. It is generally more practical to employ fewer parameters to mitigate model over-parameterization and enhance experimental feasibility.

58 GEOSCIENCES↗

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗