Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,387 records · Page 77
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Too big, too small, or just right? A benchmark assessment of density functional theory for predicting the spatial extent of the electron density of small chemical systems

Multipole moments are the first-order responses of the energy to spatial derivatives of the electric field strength. The quality of density functional theory prediction of molecular multipole moments thus characterizes errors in modeling the electron density itself, as well as the performance in describing molecules interacting with external electric fields. However, only the lowest non-zero moment is translationally invariant, making the higher-order moments origin-dependent. Therefore, instead of using the 3 × 3 quadrupole moment matrix, we utilize the translationally invariant 3 × 3 matrix of second cumulants (or spatial variances) of the electron density as the quantity of interest (denoted by K). The principal components of K are the square of the spatial extent of the electron density along each axis. A benchmark dataset of the principal components of K for 100 small molecules at the coupled cluster singles and doubles with perturbative triples at the complete basis set limit is developed, resulting in 213 independent K components. The performance of 47 popular and recent density functionals is assessed against this Var213 dataset. Several functionals, especially double hybrids, and also SCAN and SCAN0 predict reliable second cumulants, although some modern, empirically parameterized functionals yield more disappointing performance. The H, Li, and Be atoms, in particular, are challenging for nearly all methods, indicating that future functional development could benefit from the inclusion of their density information in training or testing protocols.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Equation-of-motion coupled-cluster method with double electron-attaching operators: Theory, implementation, and benchmarks

We report a production-level implementation of the equation-of-motion (EOM) coupled-cluster (CC) method with double electron-attaching (DEA) EOM operators of 2 p and 3 p 1 h types, EOM-DEA-CCSD. This ansatz, suitable for treating electronic structure patterns that can be described as two-electrons-in-many orbitals, represents a useful addition to the EOM-CC family of methods. We analyze the performance of EOM-DEA-CCSD for energy differences and molecular properties. By considering reduced quantities, such as state and transition one-particle density matrices, we compare EOM-DEA-CCSD wave functions with wave functions computed by other EOM-CCSD methods. The benchmarks illustrate that EOM-DEA-CCSD is capable of treating diradicals, bond-breaking, and some types of conical intersections.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Machine Learning-enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. Here, we present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Testing and Expertise for Marine Energy (TEAMER) Program Support - Numerical Modeling Assistance for iProTech's "PIP" WEC Device: Cooperative Research and Development Final Report, CRADA Number CRD-20-17303

Develop a time-domain, 3 degree of freedom numerical model of IProTech's PIP device, including power take-off (PTO), that can be used to develop control algorithms and establish a baseline technology performance level (TPL). This is important to IProTech because having a good numerical model allows IProTech to optimize design and control algorithms on an ongoing basis. A baseline TPL enables the PIP technology to be benchmarked against other wave energy converter (WEC) concepts, providing (i) justification (or not) for investment in wave tank model testing, (ii) identifying optimum design parameters and control algorithms for the test model, (iii) specifying test conditions and (iv) interpreting wave tank test results. An accredited WEC-Sim model of the PIP device is an essential tool for further development of the technology. 1) The development of numerical models of the PIP concept will enable iProTech to evaluate the potential performance of this concept, and to investigate how sensitive the device is to different variables within the system (e.g. hydraulic components, geometry dimensions, etc.) 2) The numerical models were developed with WEC-Sim and PTO-Sim -- the de-facto industry standard WEC numerical modeling tools. A detailed model of the system's hydraulic circuit was developed with PTO-Sim. However, WEC-Sim is based on linear hydrodynamics and has several limitations; validation against high fidelity models/physical scale models would help to build confidence in the results.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗
DOE OSTI2022

Evaluation of Oak Ridge National Laboratory Health Physics Research Reactor Operation Data for Critical Benchmark Creation

The Oak Ridge National Laboratory (ORNL) Health Physics Research Reactor (HPRR) was a research reactor designed and built at ORNL in 1961. The critical assembly used a highly enriched uranium and molybdenum alloy as the fuel and could be operated in steady-state or burst modes. The HPRR has recently been the object of an investigation to create a criticality benchmark. Such benchmarks are very important, as they are used primarily to show the accuracy of newly developed modeling codes and to help experimental validation and reactor licensing. The evaluated experiments considered in this paper were carried out between 1974 and 1986 from various HPRR activities such as steady-state subcritical, steady-state critical, and burst prompt super-critical operations of the reactor for dosimetry, irradiation, or training purposes. By using the HPRR experimental logbook information and the as-built drawings of the critical assembly, a highly detailed model of the HPRR was created with SCALE 6.2.4/KENO-VI, and a first version of a critical benchmark of the HPRR was developed following the International Criticality Safety Benchmark Evaluation Project (ICSBEP) guidelines for thorough description and uncertainty/sensitivity quantification. Unfortunately, in most of the evaluated experiments, the obtained difference between calculated and experimental k eff is around 1,000 pcm, corresponding to a relative error of approximately 1%, beyond the quality standards of the ICSBEP recommending a relative error below 0.1%. Moreover, the derived experimental uncertainty is high, around 4% relative, mainly due to the U-Mo fuel density uncertainty, but also from numerous other factors. For these reasons, the creation of a valuable critical benchmark from HPRR operation data is thus far compromised. In this paper, the different steps of the experiments’ evaluation are summarized, and the reasons for the experimental/calculation discrepancies and potential ways to solve them are explored. This paper also aims to remind us always to exercise considerable care when performing experimental work, and to record all the data possible for potential future uses.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2026

A Full-Induction Magnetohydrodynamics Solver for Liquid Metal Fusion Blankets in Vertex-CFD

Multiphysics modeling of liquid metal fusion blankets, which produce tritium and convert energy of neutrons created via fusion reactions into heat, is crucial for predicting performance, ensuring structural integrity, and optimizing energy production. While traditional blanket modeling of liquid metal flows during normal steady operating conditions commonly employs the inductionless approximation of the magnetohydrodynamics (MHD) equations, transient scenarios, when the plasma-confining magnetic field varies on millisecond time scales, require a full-induction MHD approach that dynamically evolves the magnetic field via the time-dependent induction equation. This paper presents the formulation, implementation, and initial verification of a full-induction MHD solver integrated within the open-source Vertex-CFD framework, which aims to achieve tight multiphysics coupling, a flexible software design enabling easy extension and addition of physics models, and performance portability across computing platforms. The solver utilizes finite element spatial discretization, implicit Runge–Kutta time integration, and an inexact Newton method to solve the resulting discrete nonlinear system, leveraging Trilinos packages for efficient computation. Verification against selected benchmark problems demonstrates accuracy and robustness of the solver. Furthermore, when the solver is applied to an idealized blanket model in 2.5D and full 3D, results obtained with Vertex-CFD are in good agreement with recently published quasi-2D simulations. These findings establish a computational foundation for future simulations of transient MHD phenomena in liquid metal blankets with Vertex-CFD, and open avenues for future extensions and performance optimizations.

Endeve, Eirik [ORNL] (ORCID:0000000312519507)↗
DOE OSTI2026

Crowdsourcing the Frontier: Advancing Hybrid Physics‐ML Climate Simulation via a $\$$50,000 Kaggle Competition

Subgrid machine-learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.

Environmental sciences↗
DOE OSTI2022

Sizing ramping reserve using probabilistic solar forecasts: A data-driven method

Ramping products have been introduced or proposed in several U.S. power markets to mitigate the impact of load and renewable uncertainties on market efficiency and reliability. Current methods often rely on historical data to estimate the requirements of ramping products and fail to take into account the effects of the latest weather conditions and their uncertainties, which could lead to overly conservative or insufficient requirements. This study proposes a k-nearest-neighbor-based method to give weather-informed estimates of ramping needs based on short-term probabilistic solar irradiance forecasts. Forecasts from multiple sites are employed in conjunction with principal component analysis to derive numerical classifiers to characterize system-level weather conditions. In addition, we develop a data-driven method to optimize the model parameters in a rolling-forward manner. By using real-world data from the California Independent System Operator, we design two metrics to evaluate method performance: 1) frequency of shortage and 2) oversupply of ramping product. Our proposed method presents advantages in comparison with the baseline and a set of benchmark methods: without compromising system reliability, it reduces system ramping requirements by up to 25%, therefore improving both system reliability and economics.

14 SOLAR ENERGY↗
DOE OSTI2026

Enabling topography-resolving structural dynamic contact simulation

Damping of structures and systems is often dominated by frictional dissipation in connections, the prediction of which remains a longstanding scientific challenge. Previous studies have shown that the actual topography of contact interfaces may have a strong effect, especially in the partial slip/liftoff regime. We recently proposed a multi-scale method, which couples finite element and boundary element modeling. The primary benefit of this approach is that it permits to analyze the effect of the actual contact topography on the dynamics of jointed structures. While this multi-scale modeling method was initially developed for quasi-static analysis, we demonstrate herein how it can be used for time step integration and Harmonic Balance analysis. We cross-verify those fully dynamic analysis methods against each other and quasi-static results, for the S4 Beam benchmark. We compare the multi-scale method against state-of-the-art full-FE analysis, in terms of numerical damping and computational performance. Some discrepancy is found to be of physical origin. Depending on the load history, it is shown that the system settles to a slightly different equilibrium. Finally, transient multi-scale simulations enable the prediction of this interesting phenomenon, for the first time, for a structure with bolted joints.

Frictional-unilateral contact↗
DOE OSTI2024

Iridium Surface Oxide Affects the Nafion Interface in Proton-Exchange-Membrane Water Electrolysis

Proton-exchange-membrane water electrolyzer (PEMWE) catalyst layers consist of aggregates of catalyst particles (typically iridium) and ionomer (typically Nafion). Prior work suggests that the oxide form of Ir affects the kinetics of the oxygen-evolution reaction. However, because most catalyst-benchmarking studies are conducted ex situ in liquid electrolytes, it remains unclear how the ionomer is influenced by the catalyst oxide and affects overall cell performance. Using a suite of experimental techniques, we conduct fundamental investigations into model ink (catalyst and ionomer dispersed in solution) and thin-film systems to inform cell-level overpotential analysis as a function of three forms of Ir (metallic Ir m , oxyhydroxide IrOOH, and oxide IrO 2 ). Furthermore, nafion on Ir m has a high degree of phase separation and higher swelling, likely improving the ionic conductivity. Additionally, Nafion binds most strongly to IrOOH, likely yielding reduced kinetic overpotentials. These findings highlight the intricacies of the ionomer/Ir interface and provide insight into all catalyst-layer systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

sciCAN: single-cell chromatin accessibility and gene expression data integration via cycle-consistent adversarial network

The boom in single-cell technologies has brought a surge of high dimensional data that come from different sources and represent cellular systems from different views. With advances in these single-cell technologies, integrating single-cell data across modalities arises as a new computational challenge. Here, we present an adversarial approach, sciCAN, to integrate single-cell chromatin accessibility and gene expression data in an unsupervised manner. We benchmarked sciCAN with 5 existing methods in 5 scATAC-seq/scRNA-seq datasets, and we demonstrated that our method dealt with data integration with consistent performance across datasets and better balance of mutual transferring between modalities than the other 5 existing methods. We further applied sciCAN to 10X Multiome data and confirmed that the integrated representation preserves biological relationships within the hematopoietic hierarchy. Finally, we investigated CRISPR-perturbed single-cell K562 ATAC-seq and RNA-seq data to identify cells with related responses to different perturbations in these different modalities.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

An open source knowledge graph ecosystem for the life sciences

Translational research requires data at multiple scales of biological organization. Advancements in sequencing and multi-omics technologies have increased the availability of these data, but researchers face significant integration challenges. Knowledge graphs (KGs) are used to model complex phenomena, and methods exist to construct them automatically. However, tackling complex biomedical integration problems requires flexibility in the way knowledge is modeled. Moreover, existing KG construction methods provide robust tooling at the cost of fixed or limited choices among knowledge representation models. PheKnowLator (Phenotype Knowledge Translator) is a semantic ecosystem for automating the FAIR (Findable, Accessible, Interoperable, and Reusable) construction of ontologically grounded KGs with fully customizable knowledge representation. The ecosystem includes KG construction resources (e.g., data preparation APIs), analysis tools (e.g., SPARQL endpoint resources and abstraction algorithms), and benchmarks (e.g., prebuilt KGs). We evaluated the ecosystem by systematically comparing it to existing open-source KG construction methods and by analyzing its computational performance when used to construct 12 different large-scale KGs. With flexible knowledge representation, PheKnowLator enables fully customizable KGs without compromising performance or usability.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2026

A curated benchmark for cofolding models on kinase conformational states

Abstract Protein kinases are critical drug targets, requiring therapeutics that can modulate their active and inactive conformational states. While cofolding models can generate global folds directly from kinase sequences and ligand SMILES strings, these models have not yet been tested on their ability to recover ligand-induced-fit conformational states of the kinase proteins. Here, we introduce KinConfBench, a curated benchmark of 2225 high-quality human kinase chains to evaluate the ability of four state-of-the-art cofolding models—Boltz-2, Chai-1, Protenix, and RoseTTAFold-All-Atom—to recover both canonical and rare conformational states. We show that geometric success metrics of a ligand pose in the active site do not correlate strongly with the correct kinase conformational state, motivating a new set of dynamical benchmarks for assessing cofolding models. While all four cofolding models achieve ~60–80% prediction accuracy for kinase conformational classification, they exhibit severe mode collapse when performing multiple inferences, show negligible structural diversity in sampling induced-fit motions, and display a prevalent “apo-drift” in which most cofolding models predominantly predict the kinase to be in its ligand-free state. Our results highlight that capturing ligand-induced protein conformational diversity, not just geometric fit, is critical for next-generation structure-based drug discovery.

Sun, Kunyang↗
DOE OSTI2025

RELAP5-3D Capabilities for High-Temperature Gas-Cooled Reactor Analysis

RELAP5-3D is the premier systems thermal-hydraulic code in the world. The code has been used significantly for high-temperature gas-cooled reactor (HTGR) analyses over the last 10 years, but there has not been a comprehensive summary of RELAP5-3D capabilities and methods for HTGR modeling. In this paper, we provide a guide for how novice users can build models for prismatic HTGRs. This guide includes best practices for developing heat structures and setting up radiation and conduction heat transfer. We also provide a summary of relevant validation studies. These studies are divided into three categories: separate effects tests, integral effects tests, and nuclear reactor studies. We present a comprehensive review of RELAP5-3D HTGR applications for reactor modeling over the last 10 years. These applications include work for the Modular High-Temperature Gas-Cooled Reactor benchmark, design activities for a prismatic HTGR as part of the U.S. Department of Energy’s Advanced Demonstration and Test Reactor Options Study, and studies performed to design gas-cooled microreactors. Following the comprehensive literature review, we present a gap analysis highlighting capabilities of RELAP5-3D that are available and missing for a complete analysis of HTGR thermal hydraulics. We also discuss the limitations of existing code capabilities and potential improvements that could improve analyses. Current research efforts have yet to fully validate RELAP5-3D for a prismatic HTGR analysis; however, some of this is attributable to confounding factors in the available experimental data as opposed to code capabilities. In conclusion, this paper presents the most comprehensive review to date on RELAP5-3D capabilities, limitations, and applications for HTGR analysis.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

First spectroscopic study of odd-odd Cu 78

Nuclei in the vicinity of 78 Ni are important benchmarks for nuclear structure, which can reveal changes in the shell structure far from stability. Spectroscopy of the odd-odd isotope 78 CU was performed for the first time in an experiment with the EURICA setup at the Radioactive Isotope Beam Factory at RIKEN Nishina Center. Excited states in the neutron-rich isotope were populated following the β decay of 78 Ni produced by in-flight fission and separated by the BigRIPS separator. A level scheme based on the analysis of $\gamma$-$\gamma$ coincidences is presented. Tentative spin and parity assignments were made when possible based on the β -decay feeding intensities and $\gamma$-decay properties of the excited states. Time correlations between β and $\gamma$ decay show clear indications of an isomeric state with a half-life of 3.8(4) ms. Large-scale Monte Carlo shell-model calculations were performed using the A3DA-m interaction and a valence space comprising the full f p shell and the 1⁢g 9/2 and 2⁢d 5/2 orbitals for both protons and neutrons. The comparison of the experimental results with the shell-model calculations allows interpreting the excited states in terms of spin multiplets arising from the proton-neutron interaction. In conclusion, the results provide further insight into the evolution of the proton single-particle orbitals as a function of neutron number, and quantitative information about the proton-neutron interaction outside the doubly magic 78 Ni core.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics With Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis

Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns’ similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and assess the performance, efficiency, and consistency of latent maps generated by VAE, which have been utilized in prior studies for latent space cartography and used as a benchmark in this study, and the emerging TFT architecture under various configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.

Explainable AI↗