Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarking Software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Design and Operability of a Pressurized Oxy Combustion System

Fossil fuel powers more than two-thirds of the world’s electricity, a significant share of which is derived from coal power plants. Despite coal being an abundant and energy-rich fuel, the major halt to its progress is greenhouse gas emissions. Pressurized oxy-combustion cycles can achieve theoretically high thermal efficiencies along with a 90% carbon capture rate. Additionally, the high energy density allows smaller turbomachinery and save capital cost. Thus, the primary objective of this dissertation is to present the design and operability of a pressurized oxy-combustor system. The initial part of the dissertation investigates different thermodynamic cycles and presents a model that can be adapted for existing power plants. The thermodynamic model will be used in the later part to design the proposed combustor. The two cycles analyzed are ENEL and TIPS. This study focuses on qualitative analyses of the cycles using a commercially available software called Aspen Plus®. A detailed benchmark study has been performed to validate the modeling process. ENEL and TIPS cycles are designed in the software, adopting the same principles. Recirculation ratio and pressures are varied to find the efficiency range of the cycles. Power calculations are done to find overall and net efficiency for a fixed recirculation ratio. The efficiencies of the cycles are compared to select an optimum cycle. A recirculation ratio of 50% is selected for implementation. The comparison shows that ENEL is marginally efficient over TIPS, at the cost of a pressure difference of about 70bars. Technology Readiness Level analysis is performed to present the availability of the specialized equipment for the cycles. From the TRL analysis, it is seen that the cost and availability of high viii pressure equipment surmount the edge of TIPS over ENEL. Thus, ENEL is chosen as the better cycle for investigating a scaled experiment considering efficiency and viability. The later part of the dissertation focuses on developing the combustor's design for a pressurized oxy-combustion cycle. The proposed combustor is a powerhead-mounted design aimed to produce 1MW of thermal outputThe different aspects of the design of a down-fired pressurized swirl combustor are presented in this dissertation.

Chowdhury, Mehrin↗
DOE OSTI2025

Binder-benchmarking

SAND2025-07593O Binder-benchmarking evaluates the speed and memory impacts of C++, Python, and Matlab code binders. As a repository, it provides a way to locally run computation-based and memory-based benchmark suites on pybind11 and nanobind-based code in a Docker image. The software runs simple-speed and memory benchmarks on primitive navigation and integration exemplar algorithms. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Walker II, Michael [Sandia National Lab. (SNL-CA),↗
DOE OSTI2025

Generalizable, fast, and accurate DeepQSPR with fastprop

Abstract Quantitative Structure–Property Relationship studies (QSPR), often referred to interchangeably as QSAR, seek to establish a mapping between molecular structure and an arbitrary target property. Historically this was done on a target-by-target basis with new descriptors being devised to specifically map to a given target. Today software packages exist that calculate thousands of these descriptors, enabling general modeling typically with classical and machine learning methods. Also present today are learned representation methods in which deep learning models generate a target-specific representation during training. The former requires less training data and offers improved speed and interpretability while the latter offers excellent generality, while the intersection of the two remains under-explored. This paper introduces , a software package and general Deep-QSPR framework that combines a cogent set of molecular descriptors with deep learning to achieve state-of-the-art performance on datasets ranging from tens to tens of thousands of molecules. provides both a user-friendly Command Line Interface and highly interoperable set of Python modules for the training and deployment of feedforward neural networks for property prediction. This approach yields improvements in speed and interpretability over existing methods while statistically equaling or exceeding their performance across most of the tested benchmarks. is designed with Research Software Engineering best practices and is free and open source, hosted at github.com/jacksonburns/fastprop.

Burns, Jackson W. (ORCID:0000000206579426)↗
DOE OSTI2025

Benchmark Tracking System for Performance Monitoring

Benchmarking is essential for high-performance software development, particularly for monitoring performance across code iterations. This project focused on enhancing the benchmarking process for Lamellar, an asynchronous runtime for High-Performance Computing (HPC) systems developed at Pacific Northwest National Laboratory. Prior to this work, benchmark results were difficult to track and compare across code versions, presenting significant challenges in identifying performance regressions and long-term trends. The primary objective was to establish a systematic, reproducible approach for measuring performance and detecting regressions following code commits. Our methodology involved three key components: standardizing benchmark outputs, implementing data versioning, and developing analysis tools. We standardized the benchmark output format to JSON Line records containing specific fields (execution time, hardware specifications, and environmental variables). To address data management challenges, we evaluated several options and eventually chose a git repository dedicated to benchmark data. We developed a suite of Python tools that processed benchmark results, enriched them with metadata, and facilitated search in the repository. The resulting system enables more efficient filtering and comparison of performance metrics across commit histories, hardware configurations, and benchmark variants through a unified query interface. Our implementation reduces computational overhead by first checking for existing results through configuration matching before initiating new benchmark runs, thereby conserving resources. The system has been validated by Lamellar developers. It organizes results by benchmark type and build configurations for efficient retrieval. Future developments include a planned Large Language Model interface for predicting benchmark performance, incorporating the criterion package for statistical analysis, which will enable automated detection of statistically significant performance changes, and integration with continuous integration pipelines. Despite these enhancements being reserved for future work, this project has successfully provided the Lamellar development team with a framework for maintaining consistent performance standards and identifying optimization opportunities across workloads and hardware environments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Fractional occupation numbers and self-interaction correction-scaling methods with the Fermi-Löwdin orbital self-interaction correction approach

In this work, we present a new assessment of the Fermi-Löwdin orbital self-interaction correction (FLO-SIC) approach with an emphasis on its performance for predicting energies as a function of fractional occupation numbers (FONs) for various multielectron systems. Our approach is implemented in the massively parallelized NWChem quantum chemistry software package and has been benchmarked on the prediction of total energies, atomization energies, and ionization potentials of small molecules and relatively large aromatic systems. Within our study, we also derive an alternate expression for the FLO-SIC energy gradient expressed in terms of gradients of the Fermi-orbital eigenvalues and revisit how the FLO-SIC methodology can be seen as a constrained unitary transformation of the canonical Kohn–Sham orbitals. Finally, we conclude with calculations of energies as a function of FONs using various SIC-scaling methods to test the limits of the FLO-SIC formalism on a variety of multielectron systems. We find that these relatively simple scaling methods do improve the prediction of total energies of atomic systems as well as enhance the accuracy of energies as a function of FONs for other multielectron chemical species.

fractional occupation number↗
DOE OSTI2025

Development, validation, and verification of multi-pass thermo-mechanical welding simulations using the open-source MOOSE framework: NeT TG4 benchmark weldment

This study develops and validates a sequentially coupled thermo-mechanical welding simulation for the three-pass 316L stainless steel NeT TG4 benchmark weldment using the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) and the Nuclear Engineering Material model Library (NEML). A diffused ellipsoidal heat source was calibrated against thermocouple data and weld macrographs to accurately model the fusion zone geometry and transient thermal fields. Material hardening is represented using the Lemaitre-Chaboche mixed isotropic-kinematic hardening model, while four annealing models - no annealing, single-stage at 1050 °C and 1300 °C, and two-stage at 800 °C/1300 °C - were implemented to assess the impact of annealing models on the accuracy of the predicted welding-induced plasticity, distortions, and residual stresses. The predictions were validated against experimental measurements and benchmarked against results from commercial software, demonstrating that thermo-mechanical MOOSE welding simulations achieve comparable accuracy with enhanced computational efficiency. This work highlights the potential of using open-source finite element frameworks like MOOSE for advanced manufacturing simulations.

Ji, Wendy [Australian Nuclear Science and Technolo↗
DOE OSTI2026

Reassessment of PHDS Fulcrum40h High Purity Germanium (HPGe) Detector System Performance

This report presents a comprehensive evaluation of the updated PHDS Fulcrum40h High Purity Germanium (HPGe) detector system, benchmarking its performance, usability, and software capabilities against the current National Nuclear Security Administration (NNSA) Nuclear Emergency Support Team (NEST) HPGe Detector System Requirements Document. Systematic measurements were conducted using a single Fulcrum40h detector to assess key parameters including gamma efficiency and resolution, neutron detection efficiency, gamma pulse-pileup response, and gamma-to-neutron crosstalk. The Fulcrum40h system, acquired in August 2022, has undergone recent updates by PHDS to address deficiencies identified following the initial NEST requirements release. The results provide critical insights into the detector’s operational capabilities and compliance with NNSA NEST standards.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Sensitivity-based Similarity Metrics for New Experiment Design Optimization

The nuclear data used in advanced reactor simulations requires validation. Data from nuclear criticality experiments can provide this validation. New nuclear criticality experiment design requires extensive knowledge and expert judgement such that the experimental design parameters are selected in such a way to keep the experiment subcritical. To aide in this experimental design process, professionals can utilize sensitivity and uncertainty analysis. Sensitivity and uncertainty analysis relies on matching new application experiments with currently existing benchmark experiments. Currently, there is functionality in the Whisper 1.1 software package to calculate a similarity metric based on neutron multiplication factor sensitivity coefficients between a new application designed by the user and existing International Criticality Safety Benchmark Experiment Project (ICSBEP) benchmarks. The Whisper 1.1 software package is included in Monte Carlo N-Particle ® Code Version 6.21 (MCNP ® 6.2). This work is geared toward expanding this capability to new similarity metrics based on beta-effective sensitivity coefficients and reactivity coefficient sensitivity coefficients. While the investigation of these sensitivity coefficients is presented in detail in separate works at this same conference, this work will be primarily focused on studying the similarity metrics in more detail. These similarity metrics will then be incorporated into the optimization algorithms used for experiment design in EUCLID (Experiments Underpinned by Computational Learning for Improvements in nuclear Data), which is a Los Alamos National Laboratory (LANL) project designed to constrain nuclear data of interest, such that adjustments can be made to possible inaccuracies. A more detailed optimization can be subsequently performed by breaking down these similarity metrics by isotope, reaction, and energy.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Intern-Artificial Intelligence Benchmarking

Benchmarks provide a standardized method for evaluating different AI models, enabling reproducibility and comparison between models, and facilitating scientific progress. As AI models continue to develop rapidly, incorporating new datasets, capabilities, and architectures becomes more complicated. Therefore, the current static benchmarks become increasingly irrelevant. The MLCommons team argues that to make AI benchmarks more relevant, it involves making the benchmarks themselves more dynamic, as well as technical innovations that make it easier for scientists and researchers at all levels to use and contribute to the benchmarks. The current progress in technical innovation is a software that allows for a detailed view of a collection of AI benchmarks to be output in various formats that are easily readable and accessible.

Krishnan, Anjay [Fermilab]↗
DOE OSTI2025

qcal v0.0.1

qcal is a software package for calibration, characterization, and benchmarking of quantum gates. It was developed to operate full-stack superconducting quantum systems at the Advanced Quantum Testbed.

Hashim, Akel [Lawrence Berkeley National Laborator↗
DOE OSTI2022

Anomaly Detection in Gamma Spectra Using Hopfield Neural Network with B-SAT and Grover’s Algorithm on a Quantum Computing Simulator

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. In principle, gamma radiation sources can be detected and identified by their unique spectral lines. However, detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. In recent prior work, we have developed a Hopfield Neural Network (HNN) in conjunction with an image processing algorithm to detect a weak signal anomaly hidden among the highly fluctuating background spectra. The objective of this work is to explore quantum computing methods to increase the speed of HNN. The approach is based on the Grover’s search algorithm in conjunction with a 3-SAT problem formalism. The Grover’s algorithm is implemented on a quantum computing simulator using Qiskit software. Performance of HNN algorithm is benchmarked using search data from an environmental screening campaign, where the anomaly is a subset of measurements containing a 137 Cs source. Results indicate that using Grover’s algorithm on a quantum simulator reduces runtime of HNN by two orders of magnitude.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2025

Invertible Design Manifolds for Heat Transfer Surfaces (INVERT) (Final Technical Report)

This final report briefly reviews the main technical accomplishments of the INVERT award, summarizes existing or planned publications or transitions from the effort, and lastly reviews T2M strategies resulting from the program. Specifically, under the award, our team studied three main technical areas and performed one preliminary T2M study on the cost-benefit analysis of Inverse Design Methods and one major software release (the Maryland Inverse Design Benchmark Suite).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

OpenMxP-Opensource Mixed Precision Computing

This is an opensource library for benchmarking the system's GPU mixed precision capabilities. The software calculates solution of the system of linear equation in 64bit accuracy using mixed precision techniques and iterative refinement. Original benchmark designed is done by ICL, and it is name HPL-MxP (HPL-AI)

Lu, Hao↗
DOE OSTI2023

Opportunities for enhancing MLCommons efforts while leveraging insights from educational MLCommons earthquake benchmarks efforts

MLCommons is an effort to develop and improve the artificial intelligence (AI) ecosystem through benchmarks, public data sets, and research. It consists of members from start-ups, leading companies, academics, and non-profits from around the world. The goal is to make machine learning better for everyone. In order to increase participation by others, educational institutions provide valuable opportunities for engagement. In this article, we identify numerous insights obtained from different viewpoints as part of efforts to utilize high-performance computing (HPC) big data systems in existing education while developing and conducting science benchmarks for earthquake prediction. As this activity was conducted across multiple educational efforts, we project if and how it is possible to make such efforts available on a wider scale. This includes the integration of sophisticated benchmarks into courses and research activities at universities, exposing the students and researchers to topics that are otherwise typically not sufficiently covered in current course curricula as we witnessed from our practical experience across multiple organizations. As such, we have outlined the many lessons we learned throughout these efforts, culminating in the need for benchmark carpentry for scientists using advanced computational resources. The article also presents the analysis of an earthquake prediction code benchmark while focusing on the accuracy of the results and not only on the runtime; notedly, this benchmark was created as a result of our lessons learned. Energy traces were produced throughout these benchmarks, which are vital to analyzing the power expenditure within HPC environments. Additionally, one of the insights is that in the short time of the project with limited student availability, the activity was only possible by utilizing a benchmark runtime pipeline while developing and using software to generate jobs from the permutation of hyperparameters automatically. It integrates a templated job management framework for executing tasks and experiments based on hyperparameters while leveraging hybrid compute resources available at different institutions. The software is part of a collection called cloudmesh with its newly developed components, cloudmesh-ee (experiment executor) and cloudmesh-cc (compute coordinator).

58 GEOSCIENCES↗
DOE OSTI2022

FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning

Applications of machine learning (ML) are growing by the day for many unique and challenging scientific applications. However, a crucial challenge facing these applications is their need for ultra low-latency and on-detector ML capabilities. Given the slowdown in Moore's law and Dennard scaling, coupled with the rapid advances in scientific instrumentation that is resulting in growing data rates, there is a need for ultra-fast ML at the extreme edge. Fast ML at the edge is essential for reducing and filtering scientific data in real-time to accelerate science experimentation and enable more profound insights. To accelerate real-time scientific edge ML hardware and software solutions, we need well-constrained benchmark tasks with enough specifications to be generically applicable and accessible. These benchmarks can guide the design of future edge ML hardware for scientific applications capable of meeting the nanosecond and microsecond level latency requirements. To this end, we present an initial set of scientific ML benchmarks, covering a variety of ML and embedded system techniques.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

WfBench: Automated Generation of Scientific Workflow Benchmarks

The prevalence of scientific workflows with high computational demands calls for their execution on various distributed computing platforms, including large-scale leadership-class high-performance computing (HPC) clusters. To handle the deployment, monitoring, and optimization of workflow executions, many workflow systems have been developed over the past decade. There is a need for workflow benchmarks that can be used to evaluate the performance of workflow systems on current and future software stacks and hardware platforms.We present a generator of realistic workflow benchmark specifications that can be translated into benchmark code to be executed with current workflow systems. Our approach generates workflow tasks with arbitrary performance characteristics (CPU, memory, and I/O usage) and with realistic task dependency structures based on those seen in production workflows. We present experimental results that show that our approach generates benchmarks that are representative of production workflows, and conduct a case study to demonstrate the use and usefulness of our generated benchmarks to evaluate the performance of workflow systems under different configuration scenarios.

Coleman, Taina↗
DOE OSTI2023

Distributed Software-Defined Network Architecture for Smart Grid Resilience to Denial-of-Service Attacks

An important challenge for smart grid security is designing a secure and robust smart grid communications architecture to protect against cyber-threats, such as Denial-of-Service (DoS) attacks, that can adversely impact the operation of the power grid. Researchers have proposed using Software Defined Network frameworks to enhance cybersecurity of the smart grid, but there is a lack of benchmarking and comparative analyses among the many techniques. In this work, a distributed three-controller software-defined networking (D3-SDN) architecture, benchmarking, and comparative analysis with other techniques is presented. The selected distributed flat SDN architecture divides the network horizontally into multiple areas or clusters, where each cluster is handled by a single Open Network Operating System (ONOS) controller. A case study using the IEEE 118-bus system is provided to compare the performance of the presented ONOS-managed D3-SDN, against the POX controller. In addition, the proposed architecture outperforms a single SDN controller framework by a tenfold increase in throughput; a reduction in latency of > 20%; and an increase in throughput of approximately 11% during the DoS attack scenarios.

Agnew Jr., Dennis↗