Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Quantum Computing for AI-based Design and Optimization of Electric Motors

Knowledge-based artificial intelligence and hierarchical fuzzy logic offer an interpretable framework for electricvehicle motor preliminary design, but their computational burden grows with linguistic granularity and coupled design-space size. This paper presents a reduced quantum reformulation of the hierarchical fuzzy inference of air-gap flux density, a representative level-one motor-design parameter. Starting from the published electric-vehicle motor-design framework, a three-term fuzzy prototype is constructed from the original inference structure. The reduced model is then reformulated as a modular quantum register-oracle system, in which each hierarchical subrelation is encoded as a block oracle and evaluated through superpositionbased candidate-label testing. The proposed modular quantum formulation reproduces the reduced classical prototype after block fusion. A resource analysis shows that the reduced modular system requires seven qubits per block and twenty-two qubits in a straightforward four-block implementation. Finally, a crossovercomplexity model is derived to identify the regime in which quantum candidate search may become favorable relative to hierarchical fuzzy inference. The results show that no quantum advantage should be claimed for the present one-output reduced benchmark, but that a plausible crossover emerges for larger joint candidate spaces and higher linguistic granularity. The work therefore establishes a technically consistent starting point for future quantum-assisted electric-vehicle motor-design optimization.

Kumar, Praveen [ORNL] (ORCID:0000000291877857)↗
DOE OSTI2022

Integrated Capture and Conversion of CO 2 to Methanol (ICCCM) Process Technology - CRADA 449 (Final Report)

The goal of this project is to develop a prototype system that integrates the capture and catalytic hydrogenation of CO 2 into methanol in the same solvent. Ultimately, this integrated approach could provide a technology with projected reductions to capital and operating costs by at least 20% relative to a benchmark methanol synthesis via gas-phase CO 2 hydrogenation. There were five primary objectives for the project: 1. Optimize the selections of both the catalysts and the CO 2 capture solvents to achieve high conversion (>80%) and high selectivity (>90%) toward methanol in the targeted processing ranges (100-180°C, 1-60 bar). 2. Design a reactor system that provides a fixed catalyst surface area and high-efficiency heat exchange to assure compact modular processing. 3. Build and test a reactor system and demonstrate in a bench scale continuous flow system, and use the operating data to refine the comparison to existing conversion technologies. 4. Refine techno economic (TEA) and Tech-to-Market (TTM) analyses using the experimental data obtained from the bench-scale system. 5. Use the experimental data and TEA/TTM projections to develop a conceptual design for a pre-commercial system that targets a specific market opportunity.

09 BIOMASS FUELS↗
DOE OSTI2022

Performance improvements of the windowed multipole formalism using a rational fraction approximation of the Faddeeva function

The windowed multipole (WMP) formalism was introduced as a way to calculate Doppler broadened cross sections on the fly during Monte Carlo simulations. While more arithmetic is needed compared to point-wise cross section look-ups, performance remained competitive from the large memory reductions and sequential data access. The single most expensive function call in a depleted fuel assembly problem using WMP comes from the evaluation of the Faddeeva function, which previously relied on a highly accurate, highly-branching algorithm. This paper explores the use of rational fraction approximations tailored to the domain interest of reactor physics applications and the development of lower accuracy approximations sufficient for our application. The rational approximations were implemented and tested in OpenMC on an infinite medium problem to stress the cross section calculation routine and a PWR assembly problem. In both cases, the rational approximation nearly eliminated the ∼ 20% penalty previously observed when comparing to point-wise libraries. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Algal Biomass Conversion to Fuels via Combined Algae Processing (CAP): 2022 State of Technology and Future Research

The annual State of Technology (SOT) assessment is an essential activity for platform research conducted under the Bioenergy Technologies Office (BETO). It allows for the impact of research progress (both directly achieved in-house at NREL and furnished by partner organizations) to be quantified in terms of economic improvements in the overall biofuel production process for a particular biomass processing pathway, whether based on terrestrial or algal biomass feedstocks. As such, initial benchmarks can be established for currently demonstrated performance, and progress can be tracked toward out-year goals to ultimately demonstrate economically viable biofuel technologies. NREL's algae SOT benchmarking efforts focus both on front-end algal biomass production and separately on back-end conversion to fuels through NREL's "combined algae processing" (CAP) pathway. The production model is based on outdoor long-term cultivation data, enabled by comprehensive algal biomass production trials conducted under Development of Integrated Screening, Cultivar Optimization, and Verification Research (DISCOVR) consortium efforts and driven by data furnished by Arizona State University's (ASU's) Arizona Center for Algae Technology and Innovation (AzCATI) test bed site. The CAP model is primarily based on experimental efforts conducted under NREL research and development projects, with some process parameters provided by partner organizations. Assumptions regarding the wet storage of algae use data provided by Idaho National Laboratory (INL), while parts of the polyurethane production process leverage BETO-funded research from collaborators at Algenesis and the University of California, San Diego (UCSD). This report focuses on back-end conversion of algal biomass through the CAP pathway, highlighting the 2022 updates to minimum fuel selling price (MFSP). This update incorporates improvements to fermentation performance for two biological pathways through carboxylic acid and 2,3-butanediol (BDO) intermediates, as demonstrated through parallel research on the biochemical conversion of corn stover. Improvements are applied to the glucose fraction of the biomass only, while parameters regarding the conversion of the mannose fraction (not a significant component in corn stover) are maintained consistently with prior CAP SOTs. Additional parameters are also updated to reflect the most current understanding of each pathway, including an increase in the catalyst loading requirement in the ketonization step of the acids pathway and a decrease in the fermentation productivity in the BDO pathway. Additionally, the biomass feedstock costs (minimum biomass selling price [MBSP]), yields, and seasonal variability from the upstream cultivation SOT model were also incorporated into downstream Aspen Plus CAP models.

09 BIOMASS FUELS↗
DOE OSTI2023

Quantum Simulation of an Open System: A Dissipative 1+1D Ising Model

The 1+1D Ising model is an ideal benchmark for quantum algorithms, as it is very well understood theoretically. This is true even when expanding the model to include complex coupling constants. In this work, we implement quantum algorithms designed for the simulation of open or complex coupling quantum field theories on IBM devices with a focus on the measurement of the Lee-Yang edge singularity. This feature corresponds (at large volumes) to a phase transition, and our successful reproduction of the transition represents a non-trivial test for current hardware and its ability to distinguish features of interest in quantum field theories.

Gustafson, Erik↗
DOE OSTI2024

A comprehensive guide to CAN IDS data and introduction of the ROAD dataset

Although ubiquitous in modern vehicles, Controller Area Networks (CANs) lack basic security properties and are easily exploitable. A rapidly growing field of CAN security research has emerged that seeks to detect intrusions or anomalies on CANs. Producing vehicular CAN data with a variety of intrusions is a difficult task for most researchers as it requires expensive assets and deep expertise. To illuminate this task, we introduce the first comprehensive guide to the existing open CAN intrusion detection system (IDS) datasets. We categorize attacks on CANs including fabrication (adding frames, e.g., flooding or targeting and ID), suspension (removing an ID’s frames), and masquerade attacks (spoofed frames sent in lieu of suspended ones). We provide a quality analysis of each dataset; an enumeration of each datasets’ attacks, benefits, and drawbacks; categorization as real vs. simulated CAN data and real vs. simulated attacks; whether the data is raw CAN data or signal-translated; number of vehicles/CANs; quantity in terms of time; and finally a suggested use case of each dataset. State-of-the-art public CAN IDS datasets are limited to real fabrication (simple message injection) attacks and simulated attacks often in synthetic data, lacking fidelity. In general, the physical effects of attacks on the vehicle are not verified in the available datasets. Only one dataset provides signal-translated data but is missing a corresponding “raw” binary version. This issue pigeon-holes CAN IDS research into testing on limited and often inappropriate data (usually with attacks that are too easily detectable to truly test the method). The scarcity of appropriate data has stymied comparability and reproducibility of results for researchers. As our primary contribution, we present the Real ORNL Automotive Dynamometer (ROAD) CAN IDS dataset, consisting of over 3.5 hours of one vehicle’s CAN data. ROAD contains ambient data recorded during a diverse set of activities, and attacks of increasing stealth with multiple variants and instances of real (i.e. non-simulated) fuzzing, fabrication, unique advanced attacks, and simulated masquerade attacks. To facilitate a benchmark for CAN IDS methods that require signal-translated inputs, we also provide the signal time series format for many of the CAN captures. Our contributions aim to facilitate appropriate benchmarking and needed comparability in the CAN IDS research field.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Metals and Quantum Materials with Spin-orbit Interactions by Quantum Monte Carlo methods

The key goals of this project were as follows: 1) Analysis and benchmarks of electron correlation effects recovered in the fixed-node approximation that is inherent to quantum Monte Carlo (QMC) method as applied to metallic states; 2) development of new algorithms for electron spin-degrees of freedom to be treated as explicit quantum variables; 3) designing electronic structure QMC algorithm for efficient evaluation of spin-orbit effects in systems with heavy atoms; 4) adapting the algorithm to complex wave functions and developing corresponding fixed-phase approximation; 5) design and testing of algorithm for valence-only non-local spin-orbit operators; 6) analysis of fixed-node vs fixed-phase errors and their comparisons. The key accomplishments: i) We carried out a systematic study of Li systems by the fixed-node diffusion Monte Carlo method. This involved Li atom, molecule, cluster and solid calculated by the full range of QMC methods including fixed-node QMC.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2026

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multiple efforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680,000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin [Fermilab] (ORCID:0000000157000288↗
DOE OSTI2025

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multipleefforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of synthesized ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680 000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin G. [Fermilab]↗
DOE OSTI2026

Hydride and Seek: Comparing Crystallographic Hydride Placement Techniques with an Open-Shell Cobalt Complex

Locating hydrides is crucial in organometallic chemistry but difficult to do accurately using X-ray diffraction. Electron diffraction has been proposed as a way to overcome this problem but has not been systematically compared to neutron diffraction and to quantum crystallography (Hirshfeld atom refinement, HAR) to test this hypothesis. Here, we present a comparative analysis of methods for a terminal cobalt hydride complex by comparing a single-crystal neutron diffraction reference structure to results from single-crystal X-ray diffraction with and without Hirshfeld atom refinement (HAR, NoSpherA2), density functional theory (DFT), and electron diffraction (3D-ED/MicroED) refined under kinematical and dynamical formalisms. Conventional X-ray diffraction gives lower precision than neutron diffraction as expected. Despite expected improvements, HAR gives systematic deviation from the neutron benchmark. Interestingly, optimized DFT equilibrium geometries are closer to the neutron value than the value from HAR. On the other hand, electron diffraction with a high-quality data set coupled with dynamical refinement localizes the hydride in difference maps and gives excellent agreement with the neutron data. Dynamical refinement is crucial, as kinematical refinement does not allow assignment of a hydride peak. This cross-modal comparison defines the conditions under which 3D-ED/MicroED delivers high-precision metal–hydride distances for this open-shell cobalt hydride.

anions↗
DOE OSTI2023

Improved quality metrics for association and reproducibility in chromatin accessibility data using mutual information

Correlation metrics are widely utilized in genomics analysis and often implemented with little regard to assumptions of normality, homoscedasticity, and independence of values. This is especially true when comparing values between replicated sequencing experiments that probe chromatin accessibility, such as assays for transposase-accessible chromatin via sequencing (ATAC-seq). Such data can possess several regions across the human genome with little to no sequencing depth and are thus non-normal with a large portion of zero values. Despite distributed use in the epigenomics field, few studies have evaluated and benchmarked how correlation and association statistics behave across ATAC-seq experiments with known differences or the effects of removing specific outliers from the data. Here, we developed a computational simulation of ATAC-seq data to elucidate the behavior of correlation statistics and to compare their accuracy under set conditions of reproducibility. Using these simulations, we monitored the behavior of several correlation statistics, including the Pearson’s R and Spearman’s ρ coefficients as well as Kendall’s τ and Top–Down correlation. We also test the behavior of association measures, including the coefficient of determination R 2 , Kendall’s W, and normalized mutual information. Our experiments reveal an insensitivity of most statistics, including Spear man’s ρ, Kendall’s τ, and Kendall’s W, to increasing differences between simulated ATAC-seq replicates. The removal of co-zeros (regions lacking mapped sequenced reads) between simulated experiments greatly improves the estimates of correlation and association. After removing co-zeros, the R 2 coefficient and normalized mutual information display the best performance, having a closer one-to-one relationship with the known portion of shared, enhanced loci between simulated replicates. When comparing values between experimental ATAC-seq data using a random forest model, mutual information best predicts ATAC-seq replicate relationships. Collectively, this study demonstrates how measures of correlation and association can behave in epigenomics experiments. We provide improved strategies for quantifying relationships in these increasingly prevalent and important chromatin accessibility assays.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Breaking the reproducibility barrier with standardized protocols for plant–microbiome research

Inter-laboratory replicability is crucial yet challenging in microbiome research. Leveraging microbiomes to promote soil health and plant growth requires understanding underlying molecular mechanisms using reproducible experimental systems. In a global collaborative effort involving five laboratories, we aimed to help advance reproducibility in microbiome studies by testing our ability to replicate synthetic community assembly experiments. Our study compared fabricated ecosystems constructed using two different synthetic bacterial communities, the model grass Brachypodium distachyon, and sterile EcoFAB 2.0 devices. All participating laboratories observed consistent inoculum-dependent changes in plant phenotype, root exudate composition, and final bacterial community structure, where Paraburkholderia sp. OAS925 could dramatically shift microbiome composition. Comparative genomics and exudate utilization linked the pH-dependent colonization ability of Paraburkholderia, which was further confirmed with motility assays. The study provides detailed protocols, benchmarking datasets, and best practices to help advance replicable science and inform future multi-laboratory reproducibility studies.

Novak, Vlastimil↗
DOE OSTI2020

Predicting Elastic Properties of Materials from Electronic Charge Density Using 3D Deep Convolutional Neural Networks

Materials representation plays a key role in machine learning-based prediction of materials properties and new materials discovery. Currently both graph and three-dimensional (3D) voxel representation methods are based on the heterogeneous elements of the crystal structures. Here, we propose to use electronic charge density (ECD) as a generic unified 3D descriptor for materials property prediction with the advantage of possessing close relation with the physical and chemical properties of materials. We developed an ECD-based 3D convolutional neural networks (CNNs) for predicting the elastic properties of materials, in which CNNs can learn effective hierarchical features with multiple convolving and pooling operations. Extensive benchmark experiments over 2170 $Fm\bar3m$ face-centered-cubic materials show that our ECD-based CNNs can achieve good performance for elasticity prediction. Especially, our CNN models based on the fusion of elemental Materials-Agnostic Platform for Informatics and Exploration features and ECD descriptors achieved the best fivefold cross-validation performance. More importantly, we showed that our ECD-based CNN models can achieve significantly better extrapolation performance when evaluated over nonredundant data sets, where there are few neighbor-training samples around test samples. As an additional validation, we evaluated the predictive performance of our models on 329 materials of space group $Fm\bar3m$ by comparing to density functional theory calculated values, which shows a better prediction power of our model for bulk modulus than shear modulus. Because of the unified representation power of ECD, it is expected that our ECD-based CNN approach can also be applied to predict other physical and chemical properties of crystalline materials.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Fractional occupation numbers and self-interaction correction-scaling methods with the Fermi-Löwdin orbital self-interaction correction approach

In this work, we present a new assessment of the Fermi-Löwdin orbital self-interaction correction (FLO-SIC) approach with an emphasis on its performance for predicting energies as a function of fractional occupation numbers (FONs) for various multielectron systems. Our approach is implemented in the massively parallelized NWChem quantum chemistry software package and has been benchmarked on the prediction of total energies, atomization energies, and ionization potentials of small molecules and relatively large aromatic systems. Within our study, we also derive an alternate expression for the FLO-SIC energy gradient expressed in terms of gradients of the Fermi-orbital eigenvalues and revisit how the FLO-SIC methodology can be seen as a constrained unitary transformation of the canonical Kohn–Sham orbitals. Finally, we conclude with calculations of energies as a function of FONs using various SIC-scaling methods to test the limits of the FLO-SIC formalism on a variety of multielectron systems. We find that these relatively simple scaling methods do improve the prediction of total energies of atomic systems as well as enhance the accuracy of energies as a function of FONs for other multielectron chemical species.

fractional occupation number↗
DOE OSTI2024

Extending deterministic transport capabilities for very-high and ultra-high energy electron beams

Focused Very-High Energy Electron (VHEE, 50–300 MeV) and Ultra-High Energy Electron (UHEE, > 300 MeV) beams can accurately target both large and deeply seated human tumors with high sparing properties, while avoiding the spatial requirements and cost of proton and heavy ion facilities. Advanced testing phases are underway at the CLEAR facilities at CERN (Switzerland), NLCTA at Stanford (USA), and SPARC at INFN (Italy), aiming to accelerate the transition to clinical application. Currently, Monte Carlo (MC) transport is the sole paradigm supporting preclinical trials and imminent clinical deployment. In this paper, we propose an alternative: the first extension of the nuclear-reactor deterministic chain Njoy-Dragon for VHEE and UHEE applications. We have extended the Boltzmann-Fokker-Planck (BFP) multigroup formalism and validated it using standard radio-oncology benchmarks, complex assemblies with a wide range of atomic numbers, and comprehensive irradiation of the entire periodic table. We report that 99% of water voxels exhibit a BFP-MC deviation below 2% for electron energies under 1.5 GeV. Additionally, we demonstrate that at least 97% of voxels of bone, lung, adipose tissue, muscle, soft tissue, tumor, steel, and aluminum meet the same criterion between 50 MeV and 1.5 GeV. For water, the thorax, and the breast intra-operative benchmark, typical average BFP-MC deviations of 0.3% and 0.4% were observed at 300 MeV and 1 GeV, respectively. By irradiating the entire periodic table, we observed similar performance between lithium (Z = 3) and cerium (Z = 58). Deficiencies observed between praseodymium (Z = 59) and einsteinium (Z = 99) have been reported, analyzed, and quantified, offering critical insights for the ongoing development of the Evaluated Nuclear Data File mode in NJOY.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Design of an Integrated Solids Handling System to Maximize Syngas Process Reliability (Cooperative Research and Development Final Report)

The National Renewable Energy Laboratory (NREL), in partnership with Wonderful Renewable Energy (WRE) and Idaho National Laboratory (INL), plans to develop a general methodology for designing integrated biorefinery solids preprocessing, handling, and feeding systems based on the chemical, physical, and mechanical attributes of the starting biomass material. This attribute-driven approach will include detailed feedstock property measurements, iterative computational modeling, and bench-scale testing to design systems for preprocessing, handling, and reactor in-feed, up to and including the selection of the conversion reactor. The initial tests and system design will be conducted using waste material from almond and pistachio growing and production operations (shells, hulls, and wood), targeting the conversion of this material to syngas for electricity production. The methodology will then be generalized to other feedstocks. The purpose of this project is to design an integrated solids handling system to maximize the process reliability of converting almond and pistachio waste to electricity. The design methodology and workflow developed from this example will then be applied to the Feedstock Conversion Interface Consortium (FCIC) benchmark loblolly pine residues, thus demonstrating the robustness of the overall design approach and providing insight and guidance for future conversion systems. V-Grid Energy Systems was brought on as a subcontractor to provide gasifiers and labor to complete gasifier runs.

09 BIOMASS FUELS↗
DOE OSTI2025

Review of Grey Box/Black Box Data Contamination Metrics on Open and Commercial Models

Dataset contamination is a problem where benchmarks and tasks used to evaluate the capabilities of Large Language Models (LLMs) have been incorporated into the training dataset of the models. This gives a false sense of performance that can overestimate how these models will function on truly unseen data. This problem becomes worse with commercial LLMs with larger and non-accessible training data, so techniques have been developed to try to measure the degree to which a model is contaminated with a benchmark’s data. To understand the effectiveness of these techniques, particularly when evaluating contamination on coding tasks, we review trends and categorize techniques by the degree of access to the model that is required. The research literature on this topic has reported mixed effectiveness of these techniques, so we select a set of black box (text access only) and grey box (access to model loss/probabilities required) techniques and apply them to both commercial and non-commercial models. We implement these metrics as part of a framework to test the contamination of Python code in LLMs to see to what extent we can replicate the effectiveness (or ineffectiveness) of these contamination detection techniques. Though we find mixed results in the capabilities of these metrics to identify contamination, we do observe evidence that they can identify contamination (broadly) in fine-tuned models when both a baseline and fine-tuned model is present. Additionally, similarity metrics were able to identify between contaminated and uncontaminated data even in situations where the data is distributionally similar (e.g., drawn from the same set of code projects).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Status and performance of the GlueX DIRC

In support of the future physics program at Ce+BAF, we propose a prototype positron target. The prototype target consists of a tungsten disk mounted on a water-cooled copper support structure to enhance heat removal. The primary goals of this work are to benchmark CFD simulations, ensure vacuum integrity at the 10^-6 Pa level, achieve stable target rotation up to 10 Hz, and implement efficient water-based cooling for sustained high-power operation. To manage the thermal load, the target is designed to rotate, allowing the deposited heat to spread over a larger region, and thereby reducing localized thermal stress while maintaining the maximum temperature below 1000 K. To experimentally evaluate the thermal performance and structural stability of the target, we plan to conduct high-power laser heating tests at Laser Lab 5, LERF Building, JLab. These tests will serve as a stepping-stone for the positron production target at CEBAF.

Stevens, J.R. [College of William and Mary, Willia↗