Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarking Software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Modeling fission product diffusion in TRISO fuel particles with BISON

Diffusion of fission products in intact TRISO particles depends on particle geometry, fission product source rates, time, temperature, and temperature-dependent diffusion coefficients. Simulating this diffusion process requires models for source rates and diffusion coefficients, plus computation of the temperature field if not prescribed. In addition, simulation quality depends on discretization of the geometry, appropriate time stepping, and the accuracy of the solution method. In this paper, we explore the simulation of fission product diffusion in TRISO fuel particles using the finite element method via the fuel performance code Bison. Recent material model development has occurred in Bison for each material present in tri-structural isotropic (TRISO) fuel particles: the buffer, inner pyrolytic carbon, silicon carbide, and outer pyrolytic carbon layers, as well as the fuel kernel. Also, new mesh generation and fission product release fraction capabilities have been added. Diffusion capabilities are shown to converge to the correct solution via formal verification tests. A large number of code benchmarking problems are also given, with good results, showing that Bison’s computed release fractions closely match those of other software tools. Finally, a significant validation effort is detailed in which fission product release, measured as part of the AGR-1 capsule experiments, is compared to Bison outputs. Bison outputs compare very well to the experimental data and to PARFUME results.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

PRISMS-Fatigue computational framework for fatigue analysis in polycrystalline metals and alloys

Abstract The PRISMS-Fatigue open-source framework for simulation-based analysis of microstructural influences on fatigue resistance for polycrystalline metals and alloys is presented here. The framework uses the crystal plasticity finite element method as its microstructure analysis tool and provides a highly efficient, scalable, flexible, and easy-to-use ICME community platform. The PRISMS-Fatigue framework is linked to different open-source software to instantiate microstructures, compute the material response, and assess fatigue indicator parameters. The performance of PRISMS-Fatigue is benchmarked against a similar framework implemented using ABAQUS. Results indicate that the multilevel parallelism scheme of PRISMS-Fatigue is more efficient and scalable than ABAQUS for large-scale fatigue simulations. The performance and flexibility of this framework is demonstrated with various examples that assess the driving force for fatigue crack formation of microstructures with different crystallographic textures, grain morphologies, and grain numbers, and under different multiaxial strain states, strain magnitudes, and boundary conditions.

Chemistry↗
DOE OSTI2025

Monte Carlo N-Particle Transport Performance of Predicting Digital Radiographic IQI Inspection

The identification of porosity, geometric noncompliance, and other defect types are critical to the qualification of materials and components. X-ray radiographic nondestructive testing is a common industrial inspection method for process quality control and component qualification and certification. Digital radiography provides a quick and efficient alternative when compared to traditional film-based inspection. The quality of radiographic inspection is dependent on equipment specifications, such as the source spot size and detector pixel size, and the specific parameters selected for use for the radiographic technique. To evaluate if an x-ray system and technique is sufficient for a given requirement, a radiographic image quality indicator (IQI) can be used. Radiographic IQIs in hard to machine materials or hard to manufacture defects can be time consuming and expensive to manufacture. This study was conducted to evaluate current Savannah River National Laboratory (SRNL) x-ray imaging systems with a custom tantalum IQI and using Monte Carlo simulations to predict the performance of future systems. The tantalum IQI was tested using a Siefert Isovolt 420 keV x-ray tube with a Perkin Elmer XRD 1611 flat panel with 100-micron pixels. Using the Monte Carlo N-Particle transport software, the radiographic tally was used to simulate the photon flux through an identical tantalum IQI. These simulations provided a benchmark as to the best theoretical identification on a given system using our tantalum IQI. The simulations were refined to match SRNL’s current systems’ noise levels, leading to confidence in their ability to predict the performance of other systems that may be purchased and deployed in the future at the Savannah River Site. Future studies will be conducted to prove this research can be extended to artificially evaluate the ability for systems to identify critical defect sizes through x-ray radiographic inspection, drastically reducing the cost and time burdens of producing high-fidelity radiographic test articles.

digital X-ray radiography↗
DOE OSTI2020

Turbo FRMAC Implemetation of IAEA Radiological Assessment Methodologies for Nuclear and Radiological Emergencies.

This report documents the findings of an assessment of the Turbo FRMAC software's ability to implement International Atomic Energy Agency (IAEA) guidance for calculating Operational Intervention Levels (OIL) 1 & 2 for nuclear and radiological emergencies. The IAEA OIL and U.S. Federal Radiological Monitoring and Assessment Center (FRMAC) Derived Response Level methodology and implementation in respective tools were compared, as demonstrated through benchmarking activities for a nuclear power plant source term and potential radionuclides of concern for radiological dispersal devices. This comparison revealed some shortcomings in Turbo FRMACs ability to perform IAEA OIL calculations and resulted in recommended software modifications to be considered for future development.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2022

Simulated 5g Network Traffic Dataset

This is a dataset of 5G network traffic for use with machine learning tools to benchmark attack detection capabilities for multiple different models. The dataset contains simulated normal and attack 5G network traffic. There is no software in this dataset, only simulated network traffic data.

Anderson, MatthewW↗
DOE OSTI2024

Computer Vision on Edge Devices for the Short Term Prediction of Cloud Cover

Edge Computing and IoT are important pieces of today's technological landscape. Here, we build a low-cost IoT sensor for sky imaging and program it using AWS GreenGrass, one of the leading IoT platforms. We demonstrate remote reprogramming of this device to load software that predicts sun shading events through the linear advection method, which is a baseline algorithm that can be used to benchmark algorithmic improvements in future work. Some future directions for sky imaging research are enumerated.

14 SOLAR ENERGY↗
DOE OSTI2024

GradDFT. A software library for machine learning enhanced density functional theory

Density functional theory (DFT) stands as a cornerstone method in computational quantum chemistry and materials science due to its remarkable versatility and scalability. Yet, it suffers from limitations in accuracy, particularly when dealing with strongly correlated systems. To address these shortcomings, recent work has begun to explore how machine learning can expand the capabilities of DFT: an endeavor with many open questions and technical challenges. In this work, we present GradDFT a fully differentiable JAX-based DFT library, enabling quick prototyping and experimentation with machine learning-enhanced exchange–correlation energy functionals. GradDFT employs a pioneering parametrization of exchange–correlation functionals constructed using a weighted sum of energy densities, where the weights are determined using neural networks. Moreover, GradDFT encompasses a comprehensive suite of auxiliary functions, notably featuring a just-in-time compilable and fully differentiable self-consistent iterative procedure. To support training and benchmarking efforts, we additionally compile a curated dataset of experimental dissociation energies of dimers, half of which contain transition metal atoms characterized by strong electronic correlations. The software library is tested against experimental results to study the generalization capabilities of a neural functional across potential energy surfaces and atomic species, as well as the effect of training data noise on the resulting model accuracy.

Chemistry↗
DOE OSTI2021

Open-Source PSCAD Grid-Following and Grid-Forming Inverters and a Benchmark for Zero-Inertia Power System Simulations: Preprint

This paper presents open-source, flexible, and easily-scalable models of grid following and grid forming inverters for the PSCAD software platform. The models are intended for system integration studies, particularly stability analyses of power systems with high penetration of inverter-based generation. To verify the model functionality, they are implemented in a IEEE9-bus system in a zero-inertia operational scenario of 100% inverter-based generation. The models have been made available open source at the PyPSCAD NREL GitHub page.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2025

Time-resolved electrical potential pump – X-ray photoelectron spectroscopy probe developments for investigating dynamic processes occurring at electrochemical interfaces

Electrode–electrolyte interfaces are of critical importance in several fields, including renewable energy, corrosion, and environmental chemistry. However, investigating these interfaces under operational conditions poses considerable challenges due to the limitations of the instrumentation employed. While recent advancements in in situ and operando techniques have enhanced our comprehension of the steady-state properties of solid-liquid interfaces, the dynamic behaviors of these systems remain inadequately explored. This study introduces a time-resolved X-ray photoelectron spectroscopy (XPS) technique designed to capture transient reaction intermediates and charging dynamics at electrified interfaces. The presented proof-of-principle study demonstrates that electrochemical processes, represented by an equivalent electrical circuit (EEC) model, can be probed and understood using square wave voltage pulses of a potentiostat synchronized to the modified data acquisition of an XPS setup. This method offers a valuable alternative to traditional pump–probe techniques, facilitating the investigation of a broader range of electrochemical systems. A dedicated software package for analyzing time- and energy-resolved XPS with a focus on extracting parameters of the EEC is geared towards benchmarking different EECs in future real-world electrochemical experiments.

Electrochemistry↗
DOE OSTI2023

Ultra-sensitive isotope probing to quantify activity and substrate assimilation in microbiomes

Abstract Background Stable isotope probing (SIP) approaches are a critical tool in microbiome research to determine associations between species and substrates, as well as the activity of species. The application of these approaches ranges from studying microbial communities important for global biogeochemical cycling to host-microbiota interactions in the intestinal tract. Current SIP approaches, such as DNA-SIP or nanoSIMS allow to analyze incorporation of stable isotopes with high coverage of taxa in a community and at the single cell level, respectively, however they are limited in terms of sensitivity, resolution or throughput. Results Here, we present an ultra-sensitive, high-throughput protein-based stable isotope probing approach (Protein-SIP), which cuts cost for labeled substrates by 50–99% as compared to other SIP and Protein-SIP approaches and thus enables isotope labeling experiments on much larger scales and with higher replication. The approach allows for the determination of isotope incorporation into microbiome members with species level resolution using standard metaproteomics liquid chromatography-tandem mass spectrometry (LC–MS/MS) measurements. At the core of the approach are new algorithms to analyze the data, which have been implemented in an open-source software ( https://sourceforge.net/projects/calis-p/ ). We demonstrate sensitivity, precision and accuracy using bacterial cultures and mock communities with different labeling schemes. Furthermore, we benchmark our approach against two existing Protein-SIP approaches and show that in the low labeling range used our approach is the most sensitive and accurate. Finally, we measure translational activity using 18 O heavy water labeling in a 63-species community derived from human fecal samples grown on media simulating two different diets. Activity could be quantified on average for 27 species per sample, with 9 species showing significantly higher activity on a high protein diet, as compared to a high fiber diet. Surprisingly, among the species with increased activity on high protein were several Bacteroides species known as fiber consumers. Apparently, protein supply is a critical consideration when assessing growth of intestinal microbes on fiber, including fiber-based prebiotics. Conclusions We demonstrate that our Protein-SIP approach allows for the ultra-sensitive (0.01 to 10% label) detection of stable isotopes of elements found in proteins, using standard metaproteomics data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

DeepBench: A simulation package for physical benchmarking data

We introduce **DeepBench**, a python library that generates simple simulated image data from first principles, such as basic geometric shapes and astronomical objects. These data are highly valuable for developing (calibration, testing, and benchmarking) statistical and machine learning models because they make it possible to connect the final data product to physically interpretable inputs. This software includes tools to curate and store the datasets to maximize reproducibility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Methods and Experiences for Developing Abstractions for Data-intensive, Scientific Applications

Developing software for scientific applications that require the integration of diverse types of computing, instruments, and data present challenges that are distinct from commercial software. These applications require scale, and the need to integrate various programming and computational models with evolving and heterogeneous infrastructure. Pervasive and effective abstractions for distributed infrastructures are thus critical; however, the process of developing abstractions for scientific applications and infrastructures is not well understood. While theory-based approaches for system development are suited for well-defined, closed environments, they have severe limitations for designing abstractions for scientific systems and applications. The design science research (DSR) method provides the basis for designing practical systems that can handle real-world complexities at all levels. In contrast to theory-centric approaches, DSR emphasizes both practical relevance and knowledge creation by building and rigorously evaluating all artifacts. In this work, we show how DSR provides a well-defined framework for developing abstractions and middleware systems for distributed systems. Specifically, we address the critical problem of distributed resource management on heterogeneous infrastructure over a dynamic range of scales, a challenge that currently limits many scientific applications. We use the pilot-abstraction, a widely used resource management abstraction for high-performance, high throughput, big data, and streaming applications, as a case study for evaluating the DSR activities. For this purpose, we analyze the research process and artifacts produced during the design and evaluation of the pilot-abstraction. We find DSR provides a concise framework for iteratively designing and evaluating systems. Finally, we capture our experiences and formulate different lessons learned.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Understanding HPC Benchmark Performance on Intel Broadwell and Cascade Lake Processors

Hardware platforms in high performance computing are constantly getting more complex to handle even when considering multicore CPUs alone. Numerous features and configuration options in the hardware and the software environment that are relevant for performance are not even known to most application users or developers. Microbenchmarks, i.e., simple codes that fathom a particular aspect of the hardware, can help to shed light on such issues, but only if they are well understood and if the results can be reconciled with known facts or performance models. The insight gained from microbenchmarks may then be applied to real applications for performance analysis or optimization. In this paper we investigate two modern Intel x86 server CPU architectures in depth: Broadwell EP and Cascade Lake SP. We highlight relevant hardware configuration settings that can have a decisive impact on code performance and show how to properly measure on-chip and off-chip data transfer bandwidths. The new victim L3 cache of Cascade Lake and its advanced replacement policy receive due attention. Finally we use DGEMM, sparse matrix-vector multiplication, and the HPCG benchmark to make a connection to relevant application scenarios.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Advanced Computing is at the Forefront of a New “Moonshot” Revolutionizing the North American Power Grid

In the 50+ years since the first humans landed on the moon, computing has grown at breakneck speed. We are faced with another challenge that is just as daunting, and just as important to overcome-modernizing the North American electric power grid-and high-performance computing (HPC) systems with specialized software will be an important element in rising to this challenge. We describe at a high level how software developed in the ExaSGD project addresses this "moonshot" goal by utilizing exascale computing and a novel high performance solver software stack to support the mission of decarbonizing power grid operations in an environment of uncertain weather and climate. To reach the exascale benchmark the team has made a number of first-of-their-kind innovations, including novel method for stochastic optimization, fine grained parallel methods for modeling power systems, and GPU resident sparse numerical linear solvers.

17 WIND ENERGY↗
DOE OSTI2023

Quarknet Project: Simulating Double-Source Plane Lenses and Einstein Rings for Domain Adaptation

Double-Source Plane Lenses (DSPLs) are a form of strong gravitational lensing that can be used to uniquely constrain cosmological parameters like the dark energy equation of state. These objects are exceedingly rare: to date, only four have been discovered in astronomical surveys, like the Sloan Digital Sky Survey and the HyperSuprimeCam Survey. Additionally, the main features of these objects are typically co-located sets of arcs, which look like Einstein Rings of single-source galaxy-galaxy lenses. Larger astronomical surveys typically have low image resolution, which significantly exacerbates issues when trying to find DSPLs. Future surveys are likely to include many more of these objects and have higher image resolution. Traditional methods for finding strong lenses struggle to identify simple galaxy-galaxy lenses. Recently, neural networks have become the standard for lens identification, but these tools have not yet been used to search for DSPLs. In this work, we use the software deeplenstronomy to simulate varied images of DSPLs and Einstein Rings – with and without noise and artifacts of observational surveys, forming a comprehensive benchmark dataset of 40,000 images. We utilise this dataset to perform experiments with convolutional neural networks to discern between the two phenomena. Sofia Grimm and Jerry Zhou are co-first authors on this work.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Battle of the Defaults: Extracting Performance Characteristics of HDF5 under Production Load

Popular parallel I/O libraries, such as HDF5, provide tuning parameters to obtain superior performance. However, the selection of effective parameters on production systems is complex due to the interdependence of I/O software and file system layers. Hence, application developers typically use the default parameters and often experience poor I/O performance. This work conducts a benchmarking-based analysis on the HDF5 behaviors with a wide variety of I/O patterns to extract performance characteristics under the production workload. To make the analysis well controlled, we exercise I/O benchmarks on POSIX-IO, MPI-IO, and HDF5 using the same I/O patterns and in the same jobs. To address high performance variability in production environments, we repeat the benchmarks across I/O patterns, storage devices, and time intervals. Based on the results, we identified consistent HDF5 behaviors that appropriate configurations and operations on dataset layout and file-metadata placement can improve performance significantly. We apply our findings and evaluate the tuned I/O library on two supercomputers: Summit and Cori. The results show that our tuned parameters can achieve more than 10× I/O performance speedup than that with default parameters on both systems, suggesting the effectiveness, stability, and generality of our solution.

Xie, Bing↗
DOE OSTI2023

Signal Processing Based Method for Real-Time Anomaly Detection in High-Performance Computing

Performance anomalies can manifest as irregular execution times or abnormal execution events for many reasons, including network congestion and resource contention. Detecting such anomalies in real-time by analyzing the details of performance traces at scale is impractical due to the sheer volume of data High-Performance Computing (HPC) applications produce. In this paper, we propose formulating HPC performance anomaly detection as a signal-processing problem where anomalies can be treated as noise. We evaluate our proposed method in comparison with two other commonly used anomaly detection techniques of varying complexity based on their detection accuracy and scalability. Since real-time in-situ anomaly detection at a large scale requires lightweight methods that can handle a large volume of streaming data, we find that our proposed method provides the best trade-off. We then implement the proposed method in Chimbuko, the first online, distributed, and scalable workflow-level performance trace analysis framework. We compare our proposed signal-based anomaly detection algorithm with two other methods using a function of their accuracy, F1 score, and detection overhead. Our experiments demonstrate that our proposed approach achieves a 99% improvement for the benchmark datasets and a 93% improvement with Chimbuko traces.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation

The rapid development of quantum computing (QC) in the NISQ era urgently demands a low-level benchmark suite and insightful evaluation metrics for characterizing the properties of prototype NISQ devices, the efficiency of QC programming compilers, schedulers and assemblers, and the capability of quantum system simulators in a classical computer. In this work, we fill this gap by proposing a low-level, easy-to-use benchmark suite called QASMBench based on the OpenQASM assembly representation. It consolidates commonly used quantum routines and kernels from a variety of domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, and so on, trading-off between generality and usability. To analyze these kernels in terms of NISQ device execution, in addition to circuit width and depth, we propose four circuit metrics including gate density, retention lifespan, measurement density, and entanglement variance, to extract more insights about the execution efficiency, the susceptibility to NISQ error, and the potential gain from machine-specific optimizations. Applications in QASMBench can be launched and verified on several NISQ platforms, including IBM-Q, Rigetti, IonQ and Quantinuum. For evaluation, we measure the execution fidelity of a subset of QASMBench applications on 12 IBM-Q machines through density matrix state tomography, comprising 25K circuit evaluations. In addition we also compare the fidelity of executions among the IBM-Q machines, the IonQ QPU and the Rigetti Aspen M-1 system.

97 MATHEMATICS AND COMPUTING↗