Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

An Autonomous MCP Bridge to Rucio: Enhancing Data Management Accessibility for High Energy Physics

The Rucio Data Management System [1] is an important tool used by High Energy Physics experiments, including those at Fermi National Accelerator Laboratory, to store and manage exabyte-scale scientific datasets. Despite its central role in coordinating data across globally distributed storage sites, Rucio's command line interface (CLI) presents a steep learning curve, and makes it difficult for scientists to navigate through. To solve this issue, a containerized Model Context Protocol (MCP) [2] server was built that connects Large Language Models directly to Rucio, allowing AI agents to handle data tasks by using simple, natural language rather than memorized terminal commands. The core engineering focus of this project was moving the server away from slow terminal commands that require text parsing and replacing them with a native Python Client API toolset and a planned REST API framework. Moving to the Python API handles data operations directly in memory, which helps clear up formatting errors, provides the AI with clean, structured JSON data and speeds up tool execution. To prove that the system actually works, a benchmarking pipeline was also built with various questions to test the AI across four different model configurations. The questions included finding data scopes, tracking down specific datasets, and checking replication rules. Through benchmarking, early runs showed that with raw terminal text, the model would get confused and stuck, whereas switching to the Python API to feed the AI clean, structured data yielded massive improvement. By creating an intelligent and autonomous bridge to a storage network, this project shows how AI can be implemented in scientific data management, which ultimately helps scientists at Fermilab spend less time sorting through data and more time focusing on their experiments and analysis.

Akella, Kashyap [William Rainey Harper Coll.]↗
DOE OSTI2025

Evaluation of the response to electrons and pions in the scintillating fiber and lead calorimeter for the future electron-ion collider

The performance of the Baby Barrel Electromagnetic Calorimeter (Baby BCAL) — a small-scale lead-scintillating-fiber (Pb/ScFi) prototype of the GlueX Barrel Electromagnetic Calorimeter (BCAL) — was tested in a dedicated beam campaign at the Fermilab Test Beam Facility (FTBF). This study provides a benchmark for the Pb/ScFi component of the future Barrel Imaging Calorimeter (BIC) in the ePIC detector at the Electron-Ion Collider (EIC). The detector response to electrons and pions was studied at beam energies between 4 and 10 GeV, extending previous GlueX tests to a higher energy regime. The calibrated detector exhibits good linearity within uncertainties, and its electron energy resolution meets EIC requirements. The data further constrain the constant term in the energy resolution to below 1.9%, improving upon previous constraints at lower energies. Simulations reproduce key features of the electron and pion data within the limitations of the collected dataset and the FTBF test environment. Electron-pion separation in the test beam setup was analyzed using multiple methods, incorporating varying degrees of beam-related effects. The inclusion of longitudinal shower profile information enhanced the separation performance, underscoring its relevance for the full-scale BIC in ePIC. These results provide essential benchmarks for the Pb/ScFi section of the future BIC, validating detector simulations and guiding optimization strategies for electron-pion discrimination.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

ChatPORT: Fine-Tuned LLM for Easy Code {PORT}ing

Fine-tuning existing LLMs for specialized tasks has become a very attractive alternative due to its low cost and quick development cycle. With many pre-trained LLMs available, it is an increasingly complex task to choose the correct model as the starting point or base model. In this work we discuss ChatPORT - a specialized fine-tuned LLM geared towards providing correctly translated codes from one programming model to another. We evaluate a number of base models and compare and contrast their features and characteristics that make them a viable starting point. In this paper, we focus on the OpenMP offload porting capabilities of ChatPORT. We build our training data using kernels from the Heterogeneous Computing Benchmarks (HeCBench) [12] and the OpenMP Validation and Verification suite [5] to fine-tune the base models. We then test the model using unseen kernels extracted from the HeCBench benchmark suite. Our results show that: (1) not all open LLMs geared towards HPC are aware of programming models like OpenMP, (2) although all base models benefit from fine-tuning they learn differently and produce different correctness rates, (3) depending on the memory size and compute resource available, different base models can be used for fine-tuning without significantly affecting the quality of transpiled code they generate, (4) fine-tuning improved the correctness rate of the LLM by an average of 43.2%, and (5) feedback-based training data further increased the correctness rate by an average of 6% over the LLMs tested.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗
DOE OSTI2020

The ESIF-HPC-2 Benchmark Suite

We describe the development of the ESIF-HPC-2 benchmark suite, a collection of kernel and application benchmark codes for measuring computational and I/O performance from single nodes to full HPC systems that was used for acceptance testing in our recent HPC procurement. The configurations of the benchmarks used for our system is presented. We also describe a set of "dimensions" that can be used to classify benchmarks and assess coverage of a suite systematically. The collection is offered cost-free as a GitHub repository for general usage and further development.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

The ESIF-HPC-2 Benchmark Suite: Preprint

We describe the development of the ESIF-HPC-2 benchmark suite, a collection of kernel and application benchmark codes for measuring computational and I/O performance from single nodes to full HPC systems that was used for acceptance testing in our recent HPC procurement. The configurations of the benchmarks used for our system is presented. We also describe a set of 'dimensions' that can be used to classify benchmarks and assess coverage of a suite systematically. The collection is offered cost-free as a GitHub repository for general usage and further development.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

MCNP6 Developments: A 2022-23 Year in Review [Slides]

A separate MCNP6.3 V&V document reports on all the default calculations for all test suites is touched on. This report does not include the subcritical multiplication benchmark suite. After some additional clean-up and finalizing the post-processing and documentation steps, the subcritical multiplication benchmark suite will be released in the next version of our vnvstats repository. We tested the new HDF5 PTRAC feature in MCNP6.3 and found encouraging outcomes. Identical results coming out of the simulation with respect to the legacy PTRAC results. The overall runtime for all simulations is reduced by ~20% with the new HDF5 PTRAC capability. We consider giving the new HDF5 PTRAC features a try and using it for all subcritical multiplication and any other relevant (PTRAC) calculations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Development of Ionic Liquid-Additized, GF-5/6 Compatible Low-Viscosity Oils for Automotive Engine and Rear Axle Lubrication for 4% Improved Fuel Economy

This joint project among ORNL, GM, and DRO successfully developed ionic liquid-additized low viscosity lubricants, SAE 0W-12 and 70W-80, for both IC engine and rear axle lubrication, respectively, and demonstrated combined vehicle fuel economy improvement (FEI). Phosphonium-alkylphosphate ILs have been found to be the most effective among the candidates in protecting surface damage from both sliding wear and rolling contact fatigue, and optimal IL concentrations have been determined. The IL’s impact on three-way catalyst seems to be significantly less adverse than the conventional ZDDP. Oil formulations were optimized for both energy efficiency and wear protection. Tribological bench testing of the IL-additized low-viscosity oils showed at least 40% lower boundary friction, more than 60% lower elastohydrodynamic friction, and greater than 50% less wear compared with commercial baselines. IL-additized lubricants have demonstrated FEI by 9.9% in engine dynamometer tests under extreme conditions and 6.9% increased power output in gear rig tests. Prototype IL-additized engine oil and rear axle fluid exhibited combined FEI of 1.39% and 3.24% benchmarked against the state-of-the-art baselines in vehicle testing of fresh oils under the Federal Test Procedure (FTP) city cycle and Highway Fuel Economy Driving Schedule (HWFET), respectively. Used oil analysis suggested that the IL-additized low-viscosity oils produced less wear than the commercial baselines. The gained fundamental understanding and demonstrated FEI in this work pave the way for future development and implementation of the ionic liquid technology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

A subdivision-stabilized B-spline mixed material point method

Subjected to external loadings, polymeric materials, e.g., biological tissues, hydrogels, and elastomers, may undergo extreme, nearly incompressible, (self-)contact deformations. For numerical modeling employing mesh-based techniques such as the finite element method (FEM), these deformations pose significant challenges due to large distortions in the deformed geometry, accuracy issues stemming from volumetric locking effects, and increased computational cost from complex contact searches. As an alternative to mesh-based methods, the material point method (MPM), a continuum-based particle technique, is gaining attention for its ability to handle extreme distortions and capture no-slip contact without added cost. For nearly incompressible material behaviors, while mixed formulations can address locking effects by treating displacements and pressure as independent fields, they can suffer from numerical instabilities close to the incompressibility limit due to the violation of the inf-sup condition, leading to inaccurate nodal pressure solutions. Here we propose an efficient and stable mixed B-spline material point method with highest achievable regularity for quasi-compressible polymeric materials. Using the two-scale relation of B-splines, we introduce a subdivision-stabilization for the two-field mixed MPM and obtain numerically stable, oscillation-free nodal solutions with equal-order interpolations with optimal regularity. Building on the Eulerian-Lagrangian nature of MPM, a previously-converged solution framework is adopted to mitigate issues related to cell-crossing and numerical fracture artifact present in standard MPM. We assess the stability and accuracy of the developed mixed MPM at large deformations for soft materials through the benchmark Cook’s membrane problem. Additionally, we test the robustness of the proposed MPM by modeling several examples, including the compression and indentation of a circular block into a quasi-compressible substrate and the twisting deformation of a rectangular block. The findings demonstrate the MPM’s capabilities for modeling practical soft material applications.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Accelerating high-order mesh optimization using finite element partial assembly on GPUs

In this paper we present a new GPU-oriented mesh optimization method based on high order finite elements. Our approach relies on node movement with fixed topology, through the Target-Matrix Optimization Paradigm (TMOP) and uses a global nonlinear solve over the whole computational mesh, i.e., all mesh nodes are moved together. A key property of the method is that the mesh optimization process is recast in terms of finite element operations, which allows us to utilize recent advances in the field of GPU-accelerated high order finite element algorithms. For example, we reduce data motion by using tensor factorization and matrix-free methods, which have superior performance characteristics compared to traditional full finite element matrix assembly and offer advantages for GPU based HPC hardware. Furthermore, we describe the major mathematical components of the method along with their efficient GPU-oriented implementation. In addition, we propose an easily reproducible mesh optimization test that can serve as a performance benchmark for the mesh optimization community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A long path-length optical property measurement device for highly transparent detector media

Measurements of the optical properties of highly transparent liquids are necessary for large-scale detectors that seek to detect optical signatures in kiloton-scale or larger volumes. Detailed understanding of the attenuation and scattering properties of fill media is critical to the ability to accurately simulate detector performance, and yet is often unavailable until in-situ measurements can be performed. Moreover, in-situ measurements may rely on effective attenuation lengths or other approximations to characterize performance, rather than absolute measurements of attenuation and scattering. To better understand the optical properties of potential fill media and purification schemes, a system has been developed which provides simultaneous attenuation and differential scattering cross-section measurements for highly transparent liquids. This horizontal, adjustable path-length “long-arm” system provides excellent isolation from the atmosphere, a high degree of vibration insensitivity, and a simple and reliable method of calibration. Further, scattering measurements can be carried out simultaneously at ports along the beamline equipped with optics allowing the selection of scatter angle and polarization. This enables both reliable quantitative scatter measurements and phase-function separation. This system has been extensively tested using deionized (DI) water as a benchmark, and has demonstrated repeatable measurements of attenuation lengths exceeding 100 m and scattering lengths approaching 1 km.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Enhancing ChatPORT with CUDA-to-SYCL Kernel Translation Capability

Large Language Models (LLMs) have shown strong capabilities in general code translation. However, code translation involving parallel programming models remains largely unexplored. This work enhances the capabilities of code LLMs in CUDA-to-SYCL kernel translation with parameter-efficient fine-tuning. The resultant fine-tuned LLM, called ChatPORT, is an effort to provide high-fidelity translations from one programming model to another. We describe the preparation of datasets from heterogeneous computing benchmarks for model fine-tuning and testing, the parameter-efficient fine-tuning of 19 open-source code models ranging in size from 0.5 to 34 billion parameters and evaluate the correctness rates of the SYCL kernels by the fine-tuned models. The experimental results show that most code models fail to translate CUDA codes to SYCL correctly. However, fine-tuning these models using a small set of CUDA and SYCL kernels can enhance the capabilities of these models in kernel translation. Depending on the sizes of the models, the correctness rate ranges from 19.9% to 81.7% for a test dataset of 62 CUDA kernels.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2023

TCF Project - Printable low-cost catalyst for Industrial decarbonization (LLNL/MilliporeSigma)

This proposal centers in catalysts for the generation of hydrogen as a renewable energy source and other value- added carbon utilization. The goal of this project is to commercialize an LLNL-developed, low-cost 3D printable graphene oxide/MoS 2 ink tailored for DIW fabrication of a 3D catalyst electrode for hydrogen evolution. LLNL and MS will scale up and field test the catalyst inks for hydrogen evolution, benchmark them against non-3D printed electrodes, and adjust the compositions based on feedback to improve the technology's market potential and customer/user demands.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

ORNL Contributions to the ENDF/B-VIII.1 Library [Slides]

For uranium and plutonium isotopes, evaluation work is converging to a stable configuration with an overall satisfactory benchmark performance (more information on the validation tests after CSEWG). Strontium evaluation was included in the repository library and successfully tested by processing codes relative to an alternative quantification of the external function. Cerium covariance matrices were updated to address review comments on the magnitude of the uncertainty. For future uncertainty quantification, theoretical observables should be properly decoupled from current measured guidelines. Tests to correctly include LP (from resonance parameters) fitted to measured energy-averaged data are in progress. Current ENDF files contains experimental LP data.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2025

Godiva IV Thermal Neutron Dosimetry Modeling and Variance Reduction

The transfer of the Godiva IV experiment from the Los Alamos Critical Experiments Facility (LACEF) to the National Critical Experiments Research Center (NCERC) introduced a vastly different experiment room return to the neutron flux. The contribution of the background to the burst neutron energy spectrum is significant in the thermal and epithermal neutron energies. Target materials may be placed in various locations in the Godiva room, or outside of the room, for thermal neutron activation. Modeling of this dosimetry problem in Monte Carlo N-Particle (MCNP) presented a novel challenge compared to previous Godiva IV glory hole irradiation simulations. An advanced dosimetry modeling framework for high efficiency calculations in locations far from the Godiva IV fission source was desired. The mesh-based weight windows and point detector advanced variance reduction techniques in MCNP were implemented and tested using adaptations of the critical experiment benchmark model of the Godiva IV problem. The models were validated against measured activations of Nickel, Indium, Scandium, and Cobalt foils at locations 2 meters from the Godiva IV core. Dosimetry measurements were performed in collaboration with Sandia National Laboratory. The weight windows and point detector variance reduction coupled method resulted in the highest problem efficiency.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Neutrinoless Double Beta Decay of 136 Xe and Related Nuclear Structure Studies (Final Scientific Report)

The search for neutrinoless double-beta decay (0nbb) explores new physics by directly probing the unknown mass scale and possible Majorana nature of the neutrino. The nEXO experiment, one of the two leading proposed ton-scale projects in the US, has a projected sensitivity to the 136 Xe 0nbb half-life of 10 28 years. The interpretation of possible signals in this next generation of experiments would be complicated by significant variations in theoretical calculations of the nuclear matrix elements (NMEs) of the decay, and nuclear structure measurements testing those theories can help address that uncertainty. We have 136 Xe(p,n) 136 Cs reaction at TUNL to deduce the level-scheme of states in 136 Cs through which the lowest-lying 1+ state decays. We find that over 99% of such decays will proceed through at least one isomeric state with a lifetime of order 100ns, which would enable large xenon detectors to employ a delayed-coincidence technique to search for charge-exchange processes including solar neutrino interactions. We have carried out measurements at TUNL of the 134 Xe( 3 He,n) 136 Ba and 136 Xe( 3 He,n) 138 Ba reactions to low-lying 0 + states in the residuals to probe the BCS assumption for QRPA NME calculations for the initial and final nuclei in 136 Xe 0nbb. While the analyses of these reactions is not yet complete and is ongoing, our initial results indicate tension with the BCS assumption. We have assembled a thermosyphon cooling R&D system for nEXO which has demonstrated up to 500 W of cooling in vacuum. We have developed a simple thermosyphon cooling simulation for nEXO and planned laboratory tests with the R&D system to benchmark it.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Battery Life Prediction Using Reduced-Order Physics Models and Machine Learning (CRADA Final Report)

Phase 1 (Original CRADA, plus no-cost extension modifications #1-3, 6/1/2017 to 3/13/2021): The Australian Department of Defence (AUDoD) is performing accelerated aging tests of Li-ion batteries to benchmark their reliability and degradation characteristics. Using its previously developed battery lifetime predictive model framework, the National Laboratory of the Rockies (NLR) will develop analytical models based the AUDoD data to predict lifetime of the multiple Li-ion battery chemistries under real-world use scenarios of interest to AUDoD. The NLR model is based on physical degradation mechanisms encountered by Li-ion batteries and has been previously validated. Phase 2 (CRADA modification #4, plus no-cost extension modification #5, 2/22/2021 to 3/30/2025): Train and support Australian Department of Defence personnel to use NLR software for model-based estimation of Li-ion battery lifetime using accelerated battery aging data collected by the Australian Department of Defence. Under separate DOE funding from 2019 to 2021, NLR enhanced its battery life-prediction software using machine learning algorithms to automate portions of the model-fitting process, requiring significantly less labor and expert judgment and also adding uncertainty quantification, increasing statistical rigor. Under Phase 2, NLR will customize NLR Software and provide it to AuDoD. NLR will enhance its NLR Model to capture aging modes of AuDoD's multi-cell modules, including cell-balancing effects. NLR will develop example single-cell and multi-cell models based on one AuDoD battery aging dataset. NLR will train AuDoD personnel on NLR Software. By the conclusion of the project, NLR will have provided AuDoD the training materials, a user manual and software needed to perform their own analysis of additional and/or future battery aging datasets.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2026

Accelerating computing for the future electric grid (CRADA Final Report)

As a participant in the Cyclotron Road Lab-Embedded Entrepreneurship Program (LEEP), Vellex Computing, Inc. has successfully validated the "Vellex Computing Stack," a breakthrough Analog Neural Computer (ANC) specifically designed for high-performance edge optimization. This project achieved critical milestones in mixed-signal circuit stability and software-hardware co-design, directly addressing national priorities in semiconductor resiliency. The success of this work is deeply rooted in the support from the Cyclotron Road LEEP, which provided the essential "hard tech" runway—funding, mentorship, and access to Lawrence Berkeley National Laboratory’s world-class characterization facilities—allowing Vellex to overcome the "Valley of Death" often faced by deep-tech hardware startups. By leveraging LBNL’s advanced testing infrastructure, Vellex was able to rigorously benchmark the ANC architecture against state-of-the-art digital solutions, a feat that would have been resource-prohibitive independently. This collaboration has not only advanced American leadership in analog computing but has also matured Vellex’s technology to a stage ripe for private sector commercialization.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Effect of Computational Schemes on Coupled Flow and Geo-Mechanical Modeling of CO 2 Leakage through a Compromised Well

Carbon capture, utilization, and storage (CCUS) describes a set of technically viable processes to separate carbon dioxide (CO 2 ) from industrial byproduct streams and inject it into deep geologic formations for long-term storage. Legacy wells located within the spatial domain of new injection and production activities represent potential pathways for fluids (i.e., CO 2 and aqueous phase) to leak through compromised components (e.g., through fractures or micro-annulus pathways). The finite element (FE) method is a well-established numerical approach to simulate the coupling between multi-phase fluid flow and solid phase deformation interactions that occur in a compromised well system. We assumed the spatial domain consists of a three-phases system: a solid, liquid, and gas phase. For flow in the two fluids phases, we considered two sets of primary variables: the first considering capillary pressure and gas pressure (PP) scheme, and the second considering liquid pressure and gas saturation (PS) scheme. Fluid phases were coupled with the solid phase using the full coupling (i.e., monolithic coupling) and iterative coupling (i.e., sequential coupling) approaches. The challenge of achieving numerical stability in the coupled formulation in heterogeneous media was addressed using the mass lumping and the upwinding techniques. Numerical results were compared with three benchmark problems to assess the performance of coupled FE solutions: 1D Terzaghi’s consolidation, Liakopoulos experiments, and the Kueper and Frind experiments. We found good agreement between our results and the three benchmark problems. For the Kueper and Frind test, the PP scheme successfully captured the observed experimental response of the non-aqueous phase infiltration, in contrast to the PS scheme. These exercises demonstrate the importance of fluid phase primary variable selection for heterogeneous porous media. We then applied the developed model to the hypothetical case of leakage along a compromised well representing a heterogeneous media. Considering the mass lumping and the upwinding techniques, both the monotonic and the sequential coupling provided identical results, but mass lumping was needed to avoid numerical instabilities in the sequential coupling. Additionally, in the monolithic coupling, the magnitude of primary variables in the coupled solution without mass lumping and the upwinding is higher, which is essential for the risk-based analyses.

deformation flow↗