Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Conflict Detection in Open RAN with Recurrent Neural Networks Using Geometric Manifolds

Allowing third-party applications on Radio Access Network (RAN) Intelligent Controllers (RICs) within the OpenRAN (O-RAN) framework introduces conflicting interactions that are often difficult to detect in advance. These conflicts, occurring between third-party applications in the Near RealTime RIC (Near-RT RIC), known as xApps, can lead to performance degradation and instability in O-RAN if not identified early. Existing conflict detection and mitigation solutions in the literature assume that the conflicts are known beforehand, which is not always accurate due to the complex and often hidden relationships between control parameters and Key Performance Indicators (KPIs). In this paper, we propose a novel Recurrent Neural Network (RNN) to detect both known and unknown conflicts in O-RAN xApps as specified in the O-RAN standards. We model the xApps, control parameters, and KPIs with nodes and edges to create graph structures and use the hidden nonEuclidean geometric properties of the Riemannian manifold to train the RNN model. The performance of this proposed model is validated using evaluation metrics and compared with benchmarks. Results demonstrate that the proposed RNN model, leveraging Riemannian geometric properties, can achieve 100% of the F1-score provided by an optimal solution in just 20 iterations.

5G↗
DOE OSTI2025

Conflict Detection in Open RAN with Recurrent Neural Networks Using Geometric Manifolds

Allowing third-party applications on Radio Access Network (RAN) Intelligent Controllers (RICs) within the OpenRAN (O-RAN) framework introduces conflicting interactions that are often difficult to detect in advance. These conflicts, occurring between third-party applications in the Near RealTime RIC (Near-RT RIC), known as xApps, can lead to performance degradation and instability in O-RAN if not identified early. Existing conflict detection and mitigation solutions in the literature assume that the conflicts are known beforehand, which is not always accurate due to the complex and often hidden relationships between control parameters and Key Performance Indicators (KPIs). In this paper, we propose a novel Recurrent Neural Network (RNN) to detect both known and unknown conflicts in O-RAN xApps as specified in the O-RAN standards. We model the xApps, control parameters, and KPIs with nodes and edges to create graph structures and use the hidden nonEuclidean geometric properties of the Riemannian manifold to train the RNN model. The performance of this proposed model is validated using evaluation metrics and compared with benchmarks. Results demonstrate that the proposed RNN model, leveraging Riemannian geometric properties, can achieve 100% of the F1-score provided by an optimal solution in just 20 iterations.

5G↗
DOE OSTI2022

Computational Identification and Experimental Demonstration of High‐Performance Methane Sorbents

Abstract Remarkable methane uptake is demonstrated experimentally in three metal‐organic frameworks (MOFs) identified by computational screening: UTSA‐76, UMCM‐152 and DUT‐23‐Cu. These MOFs outperform the benchmark sorbent, HKUST‐1, both volumetrically and gravimetrically, under a pressure swing of 80 to 5 bar at 298 K. Although high uptake at elevated pressure is critical for achieving this performance, a low density of high‐affinity sites (coordinatively unsaturated metal centers) also contributes to a more complete release of stored gas at low pressure. The identification of these MOFs facilitates the efficient storage of natural gas via adsorption and provides further evidence of the utility of computational screening in identifying overlooked sorbents.

Nath, Karabi↗
DOE OSTI2022

Computational Identification and Experimental Demonstration of High‐Performance Methane Sorbents

Abstract Remarkable methane uptake is demonstrated experimentally in three metal‐organic frameworks (MOFs) identified by computational screening: UTSA‐76, UMCM‐152 and DUT‐23‐Cu. These MOFs outperform the benchmark sorbent, HKUST‐1, both volumetrically and gravimetrically, under a pressure swing of 80 to 5 bar at 298 K. Although high uptake at elevated pressure is critical for achieving this performance, a low density of high‐affinity sites (coordinatively unsaturated metal centers) also contributes to a more complete release of stored gas at low pressure. The identification of these MOFs facilitates the efficient storage of natural gas via adsorption and provides further evidence of the utility of computational screening in identifying overlooked sorbents.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

TEX-Chlorine: Highly Enriched Uranium with Chloride Absorbers to Provide Validation Benchmarks for Y-12 Electrorefining Facility

This report documents the final benchmark of the TEX-Chlorine (IER-499) Thermal Epithermal eXperiments (TEX) with highly enriched uranium with chlorine absorbers and high-density polyethylene reflectors and moderators. TEX-Chlorine is a variation of the TEX-HEU baseline assembly with the addition of sodium chloride absorber plates. This evaluation contains three experimental configurations that were performed on Comet at NCERC between July and August 2024. The three configurations, which were acceptable as benchmark cases, spanned from thermal (first two configurations) to fast (third configuration). All three cases were reviewed and accepted by the ICSBEP TRG in April 2025 and was submitted to the ICSBEP in August 2025 after receiving subgroup approval.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Keeping LAMMPS cutting edge

Since its inception 30 years ago, LAMMPS has grown to be a world-class molecular dynamics code and a cornerstone of computational materials science research. This project aimed to keep LAMMPS at the forefront of molecular dynamics simulations by adapting LAMMPS to the latest developments in machine learning technology and hardware. Initially, the project set out to provide a unified implementation of active learning for efficient training data generation in LAMMPS, but the research trajectory pivoted to address more immediate and impactful opportunities. On the hardware side, recent record-breaking molecular dynamics simulations were developed on the Cerebras wafer-scale AI chip, and this project has developed an interface between LAMMPS and the hardware-specific molecular dynamics code to accelerate and simplify development and user adoption. On the software side, PyTorch’s Ahead-of-Time (AOT) compilation features promised increased performance for state-of-the-art equivariant neural network potentials, and this project laid the groundwork for their adoption in LAMMPS, resulting in a nearly 20x acceleration in extreme cases. Combined with a comprehensive benchmark study of LAMMPS across all current exascale systems, this project has reinforced LAMMPS’s role as a versatile, high-performance tool for current and future materials science applications.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Systematic benchmarking demonstrates large language models have not reached the diagnostic accuracy of traditional rare-disease decision support tools

Large language models (LLMs) show promise in supporting differential diagnosis, but their performance is challenging to evaluate due to the unstructured nature of their responses, and their accuracy compared to existing diagnostic tools is not well characterized. To assess the current capabilities of LLMs to diagnose genetic diseases, we benchmarked these models on 5213 previously published case reports using the Phenopacket Schema, the Human Phenotype Ontology and Mondo disease ontology. Prompts generated from each phenopacket were sent to seven LLMs, including four generalist models and three LLMs specialized for medical applications. The same phenopackets were used as input to a widely used diagnostic tool, Exomiser, in phenotype-only mode. The best LLM ranked the correct diagnosis first in 23.6% of cases, whereas Exomiser did so in 35.5% of cases. While the performance of LLMs for supporting differential diagnosis has been improving, it has not reached the level of commonly used traditional bioinformatics tools. Future research is needed to determine the best approach to incorporate LLMs into diagnostic pipelines.

Reese, Justin T. [Lawrence Berkeley National Labor↗
DOE OSTI2025

Benchmarking Monte Carlo codes for the modelling of low-energy neutron production target reactions

The increasing adoption of accelerator-based neutron sources (ABNS) for applications including neutron capture therapy (NCT) research has highlighted the need for accurate simulation tools. Precise modelling of the neutron production target is crucial to ensure that simulated predictions of neutron beam characteristics used for subsequent beam shaping assembly design are reliable. This work presents a comprehensive benchmarking of four widely-used Monte Carlo codes - Geant4, PHITS, FLUKA (CERN), and MCNP - for modelling low-energy neutron production target reactions. Using their recommended physics models and cross-section libraries, we evaluate each code’s performance in simulating four beam-target reactions: 7 Li(p,n) 7 Be, 9 Be(p,n) 9 B, 9 Be(d,n) 10 B, and C(d,n)N. Predictions of neutron yield, angular distributions, and energy spectra are compared against available thick target experimental data. Results show varying levels of agreement between the codes depending on the reaction type, energy range, and beam characteristics. Geant4, MCNP and PHITS are the overall best performing codes for the simulation of total neutron yield and yield in the forward direction across most reactions. Across energies where experimental benchmarks exist, inter-code discrepancies in total and forward-directed yield are typically 10 to 30%, with larger deviations at near-threshold incident ion energies. PHITS provides the best overall reproduction of experimental spectra, particularly for the 9 Be(p,n) 9 B reaction. Additionally, PHITS demonstrates superior computational performance for most reactions. These findings provide valuable guidance for ABNS design, highlighting the strengths and limitations of each code for the simulation of low-energy neutron production reactions.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Application-level benchmarking of quantum computers using nonlocal game strategies

In a nonlocal game, two noncommunicating players cooperate to convince a referee that they possess a strategy that does not violate the rules of the game. Quantum strategies allow players to optimally win some games by performing joint measurements on a shared entangled state, but computing these strategies can be challenging. We present a variational quantum algorithm to compute quantum strategies for nonlocal games by encoding the rules of a nonlocal game into a Hamiltonian. We show how this algorithm can generate a short-depth optimal quantum strategy for a graph coloring game with a quantum advantage. This quantum strategy is then evaluated on fourteen different quantum hardware platforms to demonstrate its utility as a benchmark. Finally, we discuss potential sources of errors that can explain the observed decreased performance of the executed task and derive an expression for the number of samples required to accurately estimate the win rate in the presence of noise.

nonlocal games↗
DOE OSTI2023

RELAP5-3D Solutions to Exercise 1 of the OECD-NEA HTTF Benchmark

Discussion of RELAP5-3D solutions to Exercise 1 of the OECD-NEA High Temperature Test Facility (HTTF) Benchmark. Results from ongoing benchmark activities supported by the Advanced Reactor Technologies - Gas-Cooled Reactor Program with use of the High Performance Computing Center at Idaho National Laboratory. Also discussion of Ultra Safe Nuclear Corporation and BWXT plans for the decade, and issues of modern thermal hydraulic tools for prismatic HTGR modeling. Open discussion of verification and validation and ART-GCR campaign to spearhead the development of an HTGR thermal hydraulics benchmark based on the HTTF.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

Toward Evaluating High-Level Synthesis Portability and Performance between Intel and Xilinx FPGAs

Offloading computation from a CPU to a hardware accelerator is becoming a more common solution for improving performance because traditional gains enabled by Moore’s law and Dennard scaling have slowed. GPUs are often used as hardware accelerators, but field-programmable gate arrays (FPGAs) are gaining traction. FPGAs are beneficial because they allow hardware specific to a particular application to be created. However, they are notoriously difficult to program. To this end, two of the main FPGA manufacturers, Intel and Xilinx, have created tools and frameworks that enable the use of higher level languages to design FPGA hardware. Although Xilinx kernels can be designed by using C/C++, both Intel and Xilinx support the use of OpenCL C to architect FPGA hardware. However, not much is known about the portability and performance between these two device families other than the fact that it is theoretically possible to synthesize a kernel meant for Intel to Xilinx and vice versa.In this work, we evaluate the portability and performance of Intel and Xilinx kernels. We use OpenCL C implementations of a subset of the Rodinia benchmarking suite that were designed for an Intel FPGA and make the necessary modifications to create synthesizable OpenCL C kernels for a Xilinx FPGA. We find that the difficulty of porting certain kernel optimizations varies, depending on the construct. Once the minimum amount of modifications is made to create synthesizable hardware for the Xilinx platform, more nontrivial work is needed to improve performance. However, we find that constructs that are known to be performant for an FPGA should improve performance regardless of the platform; the difficulty comes in deciding how to invoke certain kernel optimizations while also abiding by the constraints enforced by a given platform’s hardware compiler.

Cabrera, Anthony↗
DOE OSTI2024

QRCODE: Massively parallelized real-time time-dependent density functional theory for periodic systems

We present a new software module, QRCODE (Quantum Research for Calculating Optically Driven Excitations), for massively parallelized real-time time-dependent density functional theory (RT-TDDFT) calculations of periodic systems in the open-source Qbox software package. Our approach utilizes a custom implementation of a fast Fourier transformation scheme that significantly reduces inter-node message passing interface (MPI) communication of the major computational kernel and shows impressive scaling up to 16,344 CPU cores. In addition to improving computational performance, QRCODE contains a suite of various time propagators for accurate RT-TDDFT calculations. As benchmark applications of QRCODE, we calculate the current density and optical absorption spectra of hexagonal boron nitride (h-BN) and photo-driven reaction dynamics of the ozone-oxygen reaction. We also calculate the second and higher harmonic generation of monolayer and multi-layer boron nitride structures as examples of large material systems. Our optimized implementation of RT-TDDFT in QRCODE enables large-scale calculations of real-time electron dynamics of chemical and material systems with enhanced computational performance and impressive scaling across several thousand CPU cores.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Achieving GPT-4o level performance in astronomy with a specialized 8B-parameter large language model

AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained on the complete collection of astronomy-related arXiv papers from 2007 to 2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates remarkable proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9% on the AstroMLab-1 benchmark, greatly outperforming all models—proprietary and open-weight—in the 8-billion parameter class, and performing on par with GPT-4o. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely available, enabling widespread access to advanced AI capabilities for astronomical education and research.

AI assistant↗
DOE OSTI2023

Federated benchmarking of medical artificial intelligence with MedPerf

Medical artificial intelligence (AI) has tremendous potential to advance healthcare by supporting and contributing to the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving both healthcare provider and patient experience. Unlocking this potential requires systematic, quantitative evaluation of the performance of medical AI models on large-scale, heterogeneous data capturing diverse patient populations. Here, to meet this need, we introduce MedPerf, an open platform for benchmarking AI models in the medical domain. MedPerf focuses on enabling federated evaluation of AI models, by securely distributing them to different facilities, such as healthcare organizations. This process of bringing the model to the data empowers each facility to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status and real-world deployment, our roadmap and, importantly, the use of MedPerf with multiple international institutions within cloud-based technology and on-premises scenarios. Finally, we welcome new contributions by researchers and organizations to further strengthen MedPerf as an open benchmarking platform.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Towards Digital and Performance-Based Supervisory HVAC Control Delivery

Upgrading supervisory HVAC control in commercial buildings is one of the most attractive decarbonization tools at our disposal. Modern controls are software programs and can in theory be deployed at scale and with a low up-front carbon "pulse". In practice, however, control delivery is a disjointed and inefficient process, dominated by manual handoffs of imprecise English language documents. A particularly high barrier exists between control implementation and building energy modeling (BEM) which results in control sequences typically not being tested for correctness or performance before implementation. Together with industry partners, DOE and the national labs are developing an ecosystem of tools and standards that can support fully digital performance-based control delivery workflows. This paper describes this ecosystem, which consists of three mutually supportive efforts. Semantic models of buildings and their systems enable automatic configuration and installation of control software. Platform-neutral control descriptions separate control algorithms from control platforms and enable the creation of libraries of reference control implementations. Dynamic whole-building energy-control simulation that can execute physically realistic control sequences makes it possible to test and evaluate the performance of control sequences and then directly compile them for installation and execution in control systems. In addition to digitizing and streamlining project-level control delivery, these standards and related software support benchmarking of control algorithms, both rule-based and optimization-based, and help to both advance the state of the art and to implement ratings and programs that encourage the adoption of high-performance control.

building controls↗
DOE OSTI2023

Preliminary Design of Critical Experiments Involving Commercially Available B 4 C Neutron Absorber Plates with Low-Enriched UO 2 Fuel

The International Criticality Safety Benchmark Evaluation Project (ICSBEP) is an initiative to provide high-quality benchmark data in a standardized format for criticality safety analysts to use to validate calculational tools and nuclear data. In the last released 2022 version, the ICSBEP handbook contains over 5,000 critical experiment descriptions, results, and associated models. This paper describes the critical configurations proposed and calculations performed as part of the conceptual design phase of Integral Experiment Request (IER) 554 (IER-554). The goal of IER-554 is to add to the benchmark experiments available to the criticality safety community by designing critical experiments that can analyze how adding commercially available Boralcan plates to an assembly of low-enriched UO2 fuel rods affects the effective neutron multiplication factor (k eff ). Boralcan, a neutron absorber product developed by Rio Tinto, is made of B4C in an Al 1100 matrix and is commonly used for criticality suppression in fuel storage pools. In the current version of the ICSBEP handbook, only a few dated critical experiments involve B4C materials, and none involve Boralcan in a thin plate shape. The experiments designed as part of IER-554 are intended to be performed at the Sandia Pulsed Reactor Facility/Critical Experiments (SPRF/CX) apparatus at Sandia National Laboratories. SPRF/CX is a well-characterized assembly considered trustworthy by the benchmarking community because of the numerous high-quality evaluations with very low experimental uncertainties included in the ICSBEP handbook. Figure 1 illustrates the assembly for a particular configuration from LCT-078, one of the published benchmarks in the ICSBEP handbook. The experiments will be moderated and reflected by light water at ambient atmospheric pressure. Before the plates are inserted in the critical assembly, they will be characterized with x-ray computed tomography (XCT) to identify the sizes and distribution of the B 4 C powder particles inside the plates. The preliminary design calculations were performed with SCALE 6.3.0 using the KENO V.a sequence for the criticality calculations, the CE-TSUNAMI-3D sequence for the sensitivity and uncertainty studies, and the ENDF/B-VII.1 Continuous Energy cross section library. All the calculation results presented have 10 pcm statistical uncertainties. A set of critical experiments with Boralcan plates could increase the confidence of the criticality safety community in its modeling methods when using this type of neutron absorber material. Experiments would also help validate k eff calculations, and the industry could use these validations to change the B loading credit limits from the US Nuclear Regulatory Commission standard review plan for dry cask storage of spent nuclear fuel. This paper summarizes only part of the analysis documented in the preliminary design report.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

VERA neutronics high-fidelity benchmark for a modern PWR core design

Nearly 20 years since the origin of the infamous 'Kord Smith Challenge', and after 10 years and hundreds of millions of dollars invested by the US Department of Energy into the development of the Virtual Environment for Reactor Applications (VERA), the capability to accurately simulate a modern pressurized water reactor (PWR) fuel cycle at the fuel rod level with high-fidelity Monte Carlo (MC) stochastic transport methods still evades us. The deterministic methods in VERA make approximations that allow it to be successfully benchmarked against measured data from hundreds of PWR fuel cycles, but none of these benchmarks provided information at the local fuel rod level or allowed quantification of these approximations' impact for modern PWR reload designs. Using MPACT to establish the estimated isotopic distributions and thermal hydraulic conditions in a recent and challenging reactor design, a 2D quarter-core benchmark problem was established and solved consistently with both MPACT, the 51-energy group deterministic transport solver in VERA, and Shift, the continuous-energy MC transport solver in VERA. Impacts of energy groups, ray spacing, and scattering treatment were evaluated. Though MPACT has been previously shown to perform well compared to MC methods for initial Cycle 1 cores with only fresh fuel, this is the first benchmark of MPACT for a challenging modern reload core with depleted fuel and burnable absorbers. This work establishes additional credibility for the methods in MPACT and highlights the efficiency of these methods relative to those in use by MC-based tools. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗