Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarking Software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Hybrid Solid Oxide Fuel Cell/Gas Turbine Model Development for Electric Aviation

A thermodynamic model was developed and validated to analyze a high-performance solid oxide fuel cell and gas turbine (SOFC-GT) hybrid power system for electric aviation. This study used a process simulation software package (ProMax) to study the role of SOFC design and operation on the feasibility and performance of the hybrid system. Standard modules, including compressor, turbine, heat exchanger, reforming reactor, and combustor were used from the ProMax tool suite while a custom module was created to simulate the SOFC stack. The model used an SOFC test data set as an input. Additional SOFC stack performance effects, such as pressure, temperature, and utilization of air and fuel, were added from open source data. System performance predictors were SOFC specific power, fuel-to-electricity conversion efficiency, and hybrid system efficiency. Using these input data and predictors, a static thermodynamic performance model was created that can be modified for different system configurations and operating conditions. Prior to creating the final aircraft performance model, initial demonstration models were developed to validate output results. We used the NASA SOFC model as a benchmark, which was created with their Numerical Propulsion System Simulator (NPSS) software framework. Our output results matched within 1% of both the NASA model and open source SOFC performance data. With confidence gained in the accuracy of this model, a 1-MW SOFC-GT hybrid power system was constructed for an aircraft propulsion concept. Overall hybrid system efficiencies of > 75% FTE were observed during standard 36,000 feet cruise flight conditions.

30 DIRECT ENERGY CONVERSION↗
DOE OSTI2022

GriddingMachine, a database and software for Earth system modeling at global and regional scales

Land and Earth system modeling is moving towards more explicit biophysical representations, requiring increasing variety of datasets for initialization and benchmarking. However, researchers often have difficulties in identifying and integrating non-standardized datasets from various sources. We aim towards a standardized database and one-stop distribution method of global datasets. Here, we present the GriddingMachine as (1) a database of global-scale datasets commonly used to parameterize or benchmark the models, from plant traits to vegetation indices and geophysical information and (2) a cross-platform open source software to download and request a subset of datasets with only a few lines of code. The GriddingMachine datasets can be accessed either manually through traditional HTTP, or automatically using modern programming languages including Julia, Matlab, Octave, Python, and R. The GriddingMachine collections can be used for any land and Earth modeling framework and ecological research at the regional and global scales, and the number of datasets will continue to grow to meet the increasing needs of research communities.

58 GEOSCIENCES↗
NASA NTRSDate not supplied

TPSAS-NF1676L-13122-DND

An approach for assessing the delamination propagation and growth capabilities in commercial finite element codes was developed and demonstrated for the Virtual Crack Closure Technique (VCCT) implementations in ABAQUS?. The Double Cantilever Beam (DCB) specimen was chosen as an example. First, benchmark results to assess delamination propagation capabilities under static loading were created using models simulating specimens with different delamination lengths. Second, starting from an initially straight front, the delamination was allowed to propagate based on the algorithms implemented in the commercial finite element software. The load-displacement relationship obtained from the propagation analysis results and the benchmark results were compared. Good agreements could be achieved by selecting the appropriate input parameters, which were determined in an iterative procedure. A benchmark case to assess the fatigue growth prediction capabilities was also created. Input control parameters were varied to study the effect on the computed delamination growth. The benchmarks enabled the selection of the appropriate input parameters that yielded good agreements between the results obtained from the automated analysis and the benchmark results. Overall, the results are encouraging but further assessment for mixed-mode delamination is required.

Ronald Krueger↗
DOE OSTI2022

[Presentation Slides] Code-to-Code Benchmark Study for Thermal Stress Modeling and Preliminary Analysis of the High-temperature Single Heat-Pipe Experiment

In the heat-pipe-cooled microreactor, heat pipes remove heat from the reactor core as a passive heat-transfer device, so the fluid circulation is not required for cooling, which can substantially simplify the overall reactor design. However, given the extremely high temperatures in the core region and potentially large temperature gradients across the structure materials, thermal stresses need to be well-analyzed to ensure structural integrity during normal operations and accident scenarios. This presentation slides discuss finite element method-based thermal-stress analysis for the high-temperature single heat-pipe test article in the Single Primary Heat Extraction and Removal Emulator (SPHERE) facility at Idaho National Laboratory (INL), using two commercial software packages, Abaqus and Star-CCM+. A code-to-code benchmark study was performed to crosscheck the model setup and capability of each code and to gain preliminary insights into the potential thermal stress concerns from the current experimental setup. It is observed the significant thermal stresses happen at the inner surface of the heat pipe hole surrounded by electric heaters where the largest temperature gradients appear. The temperature fields have good agreements between Abaqus and Star-CCM+, while the induced thermal stresses show modest deviations probably due to differences of meshing engines used in these two codes. It is found that the local maximum thermal stresses may reach close to the ultimate tensile strength and yield strength of structural material depending on the heater power. Ultimately, the coupled thermal-structural analysis will help guide the current experimental plan and ensure the facility safety for future experimental study.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2012

Validation and Verification of Operational Land Analysis Activities at the Air Force Weather Agency

The NASA developed Land Information System (LIS) is the Air Force Weather Agency's (AFWA) operational Land Data Assimilation System (LDAS) combining real time precipitation observations and analyses, global forecast model data, vegetation, terrain, and soil parameters with the community Noah land surface model, along with other hydrology module options, to generate profile analyses of global soil moisture, soil temperature, and other important land surface characteristics. (1) A range of satellite data products and surface observations used to generate the land analysis products (2) Global, 1/4 deg spatial resolution (3) Model analysis generated at 3 hours. AFWA recognizes the importance of operational benchmarking and uncertainty characterization for land surface modeling and is developing standard methods, software, and metrics to verify and/or validate LIS output products. To facilitate this and other needs for land analysis activities at AFWA, the Model Evaluation Toolkit (MET) -- a joint product of the National Center for Atmospheric Research Developmental Testbed Center (NCAR DTC), AFWA, and the user community -- and the Land surface Verification Toolkit (LVT), developed at the Goddard Space Flight Center (GSFC), have been adapted to operational benchmarking needs of AFWA's land characterization activities.

Shaw, Michael↗
NASA NTRS1993

Liquid rocket propulsion impeller CFD modeling

Steady-state impeller geometric modeling and typical Navier-Stokes CFD algorithm analysis procedures are assessed using two benchmark quality impeller data sets. Two geometric modeling and grid generation software packages, ICEM-CFD and PATRAN, are considered. Results show that a significant advantage of PATRAN's open-ended architecture is the potential interaction between CFD and structural/thermal analysts inside the mechanical computer-aided engineering environment. However the time required to construct the inducer grid would be unacceptable in a design and engineering environment. The ICEM-CFD package is considered to be more appropriate for structural grid generation but lacks the mature link to structural/thermal analysis arena as compared to PATRAN.

Ratcliff, Mark L.↗
NASA NTRS2015

The SeaHorn Verification Framework

In this paper, we present SeaHorn, a software verification framework. The key distinguishing feature of SeaHorn is its modular design that separates the concerns of the syntax of the programming language, its operational semantics, and the verification semantics. SeaHorn encompasses several novelties: it (a) encodes verification conditions using an efficient yet precise inter-procedural technique, (b) provides flexibility in the verification semantics to allow different levels of precision, (c) leverages the state-of-the-art in software model checking and abstract interpretation for verification, and (d) uses Horn-clauses as an intermediate language to represent verification conditions which simplifies interfacing with multiple verification tools based on Horn-clauses. SeaHorn provides users with a powerful verification tool and researchers with an extensible and customizable framework for experimenting with new software verification techniques. The effectiveness and scalability of SeaHorn are demonstrated by an extensive experimental evaluation using benchmarks from SV-COMP 2015 and real avionics code.

Model Checking↗
DOE OSTI2022

CODE-TO-CODE BENCHMARK STUDY FOR THERMAL STRESS MODELING AND PRELIMINARY ANALYSIS OF THE HIGH-TEMPERATURE SINGLE HEAT PIPE EXPERIMENT

Microreactors are very small nuclear reactors with typical thermal-energy output of up to 20 MWth. The microreactor concept is gaining more and more attention for safe, robust, and reliable supply of electricity for remote locations and to meet industrial process heat needs. Heat-pipe-cooled microreactor is one of the microreactors being investigated at Idaho National Laboratory. In the heat-pipe-cooled microreactor, heat pipes remove heat from the reactor core as a passive heat-transfer device, so the fluid circulation is not required for cooling, which can substantially simplify the overall reactor design. However, given the extremely high temperatures in the core region and potentially large temperature gradients across the structure materials, thermal stresses need to be well-analyzed to ensure structural integrity during normal operations and accident scenarios for the heat-pipe-cooled microreactor designs. This paper discusses finite element method-based thermal-stress analysis for the high-temperature single heat-pipe test article in the Single Primary Heat Extraction and Removal Emulator (SPHERE) facility at Idaho National Laboratory, using two commercial software packages, Abaqus and STAR-CCM+. A code-to-code benchmark study was performed to crosscheck the model setup and capability of each code and to gain insights into the potential thermal stress concerns from the current experimental setup. It is observed the significant thermal stresses happen at the locations where the largest temperature gradients appeared between heat pipe and electric heater. The temperature fields have good agreements between Abaqus and STAR-CCM+, while the induced thermal stresses show modest deviations probably due to differences of meshing engines used in these two codes.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

hypredrive: high-level interface for solving linear systems with hypre

This software introduces a high-level interface designed to simplify solving linear systems using hypre, a renowned library for such computational challenges. It is crafted to be accessible and user-friendly, making the powerful capabilities of hypre available to a broader audience without requiring in-depth technical knowledge. The interface is characterized by its use of YAML for input, a format celebrated for its structured yet straightforward readability. This choice ensures that users can easily configure the software to meet their specific needs. Additionally, the software boasts an intuitive API that encapsulates hypre's functionalities, making it easier for users to interact with the process of solving linear systems. It is particularly beneficial for prototyping, offering a quick and efficient means to test various solver and preconditioner configurations. Furthermore, the software allows for the creation of an offline testing framework in which predefined linear systems are read from files and benchmarked with user-defined solution strategies. This makes it an invaluable tool for developers and researchers exploring and validating their computational models. Overall, the software serves as a bridge, bringing the advanced computational capabilities of hypre closer to users who may need more specialized technical expertise, thereby facilitating innovation and exploration in the field of numerical linear algebra.

Paludetto Magri, Victor↗
DOE OSTI2020

Gaussian Process Optimization of Sensitivity-Based Similarity Metrics between New Nuclear Applications and New/Existing Benchmarks [Slides]

This presentation discusses Nuclear Criticality Safety (NCS) and how designing safe, new nuclear criticality experiments requires expert judgement, which could take years of experience. Sensitivity/uncertainty (S/U) analysis can be utilized by less experienced individuals to conservatively estimate uncertainties in important parameters, such as k eff , in newly proposed nuclear experiments. The presentation poses the question of how this analysis can be performed and states that the answer lies in matching new nuclear experiments with existing benchmark experiments using similarity metrics. By increasing the criticality safety of the application in this work, higher mass limits could be used in PF-4 operations. Additionally, the presentation discusses MCNP6.2®, Whisper-1.1, the software that can be used in this analysis. Also discussed is the fact that International Criticality Safety Benchmark Evaluation Project (ICSBEP) benchmarks rarely match new nuclear applications and that there are significant differences in given set of materials and/or geometry. If there are no benchmarks that match the application, the presentation discusses the possibility of creating new benchmarks. In conclusion, this work presents a Gaussian process (GP) optimization scheme that was used to generate new benchmarks with the highest sensitivity-based similarity metrics to user-defined nuclear applications. The Gaussian process optimization successfully designed 3 new experimental benchmarks that were highly correlated to the application of interest and had k eff values near critical. Optimization over c k,i-r has shown that investigating specific isotope reactions for different applications is crucial to designing benchmark experiments. Partial contribution from Pu dominates c k similarity metric. Future work includes testing new stand-alone similarity metrics or new combinations of similarity metrics as the design criterion of this optimization – design criterion is application dependent.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

AMG2023

The AMG2023 benchmark solves two diffusion problems with a linear solver preconditioned with algebraic multigrid. The code only contains a driver, a Makefile, and a documentation file. It requires an installation of the open source software library hypre that needs to be downloaded elsewhere and is not included here. Its purpose is to benchmark linear solver performance on high performance computers.

Li, Ruipeng↗
NASA NTRS2012

Application of Benchmark Examples to Assess the Single and Mixed-Mode Static Delamination Propagation Capabilities in ANSYS

The application of benchmark examples for the assessment of quasi-static delamination propagation capabilities is demonstrated for ANSYS. The examples are independent of the analysis software used and allow the assessment of the automated delamination propagation in commercial finite element codes based on the virtual crack closure technique (VCCT). The examples selected are based on two-dimensional finite element models of Double Cantilever Beam (DCB), End-Notched Flexure (ENF), Mixed-Mode Bending (MMB) and Single Leg Bending (SLB) specimens. First, the quasi-static benchmark examples were recreated for each specimen using the current implementation of VCCT in ANSYS . Second, the delamination was allowed to propagate under quasi-static loading from its initial location using the automated procedure implemented in the finite element software. Third, the load-displacement relationship from a propagation analysis and the benchmark results were compared, and good agreement could be achieved by selecting the appropriate input parameters. The benchmarking procedure proved valuable by highlighting the issues associated with choosing the input parameters of the particular implementation. Overall the results are encouraging, but further assessment for three-dimensional solid models is required.

Krueger, Ronald↗
DOE OSTI2024

The ATLAS experiment software on ARM

With an increased dataset obtained during the Run 3 of the LHC at CERN and the even larger expected increase of the dataset by more than one order of magnitude for the HL-LHC, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM architecture is becoming a competitive and energy efficient alternative. Some surveys indicate its increased presence in HPCs and commercial clouds, and some WLCG sites have expressed their interest. Chip makers are also developing their next generation solutions on ARM architectures, sometimes combining ARM and GPU processors in the same chip. Consequently it is important that the ATLAS software embraces the change and is able to successfully exploit this architecture. We report on the successful porting to ARM of the Athena software framework, which is used by ATLAS for both online and offline computing operations. Furthermore we report on the successful validation of simulation workflows running on ARM resources. For this we have set up an ATLAS Grid site using ARM compatible middleware and containers on Amazon Web Services (AWS) ARM resources. The ARM version of Athena is fully integrated in the regular software build system and distributed in the same way as other software releases. In addition, the workflows have been integrated into the HEPscore benchmark suite which is the planned WLCG wide replacement of the HepSpec06 benchmark used for Grid site pledges. In the overall porting process we have used resources on AWS, Google Cloud Platform (GCP) and CERN. A performance comparison of different architectures and resources will be discussed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Validation of SAS4A/SASSYS-1 for Steady-State Single-Phase Natural Circulation

The development of numerical software for engineering applications requires the validation of the code against experimental benchmark datasets. To support the validation of SAS4A/SASSYS-1 in simulating the single-phase natural circulation within nuclear reactor systems, high-precision steady-state experiments are performed to capture single-phase natural circulation phenomena on an existing scaled facility with comprehensive instrumentation. Forced convection tests are designed precisely capturing the facility’s critical thermal-hydraulic parameters required for one-dimensional modeling, and a comprehensive single-phase natural circulation dataset is obtained with well-documented experimental uncertainty and facility description. Analysis and discussion based on the finalized dataset confirm the dataset’s ability in capturing dominant phenomena and address important physical interpretation of parameters. The dataset reported in this project provides a valuable benchmark resource for the validation of system analysis codes under single-phase natural circulation inside nuclear reactors. Validation of SAS4A/SASSYS-1 is then performed against the obtained benchmark dataset to confirm the capability of its physics model in simulating the steady-state single-phase natural circulation. The experimental facility is modeled in the candidate code. Solution verification is performed using Richardson-extrapolation-based estimators to quantify and restrict numerical errors from discretization. Numerical uncertainty originating from finite maximum pseudo-transient time is also quantified and restricted. Input uncertainty provided by the benchmark dataset is forward propagated through the candidate code, directly quantifying the simulation output in a Monte Carlo approach. The composition of the output uncertainty is further determined by the estimators of Sobol’ indices through a variance-based sensitivity analysis. With uncertainty quantified for each individual condition, detailed comparison between the simulation results and experimental data is performed covering the whole dataset, which shows consistent agreement for all important quantities. The validation activity reported in this project demonstrates that SAS4A/SASSYS-1 can predict the primary parameters with satisfactorily accuracy under steady-state single-phase natural circulation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Machine Learning Classification of Molten Salt Heat Exchanger Channel Plugging using Synthetic Data

This report addresses the requirements of Milestone M3.4 AI capability to identify and predict maintenance events. Development of digital twins (DT) for molten salt reactor (MSR) components is crucial for reducing operating and maintenance costs (O&M) and ensuring commercial viability of these reactors. Our focus is on development of DT for MSR primary system heat exchanger (HX), a critical component, the fault in which can reduce operating efficiency and force reactor shutdown. We are investigating the feasibility of a conceptual DT of HX consisting of internal distributed temperature sensing with fiber optics and machine learning (ML) algorithms to detect and localize faults. To determine the optimal approach to detection and localization of channel plugging, we benchmark seven different ML models: Logistic Regression, K-Nearest Neighbors (KNN), Gaussian Naïve Bayes, Support Vector Machines (SVM), Decision Tree Classifier, Random Forest Tree Classifier, and Feed-Forward Neural Network. ML algorithms are benchmarked using synthetic HX plugging data generated with computational fluid dynamics COMSOL software, with added brown noise to represent experimental noise. We show that the best performance is obtained with the Decision Tree classifier.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2009

Systematic Benchmarking of Diagnostic Technologies for an Electrical Power System

Automated health management is a critical functionality for complex aerospace systems. A wide variety of diagnostic algorithms have been developed to address this technical challenge. Unfortunately, the lack of support to perform large-scale V&V (verification and validation) of diagnostic technologies continues to create barriers to effective development and deployment of such algorithms for aerospace vehicles. In this paper, we describe a formal framework developed for benchmarking of diagnostic technologies. The diagnosed system is the Advanced Diagnostics and Prognostics Testbed (ADAPT), a real-world electrical power system (EPS), developed and maintained at the NASA Ames Research Center. The benchmarking approach provides a systematic, empirical basis to the testing of diagnostic software and is used to provide performance assessment for different diagnostic algorithms.

Kurtoglu, Tolga↗
DOE OSTI2020

Computing Bottleneck Structures at Scale for High-Precision Network Performance Analysis

The Theory of Bottleneck Structures is a recently-developed framework for studying the performance of data networks. It describes how local perturbations in one part of the network propagate and interact with others. This framework is a powerful analytical tool that allows network operators to make accurate predictions about network behavior and thereby optimize performance. Previous work implemented a software package for bottleneck structure analysis, but applied it only to toy examples. In this work, we introduce the first software package capable of scaling bottleneck structure analysis to production-size networks. Here, we benchmark our system using logs from ESnet, the Department of Energy's high-performance data network that connects research institutions in the U.S. Using the previously published tool as a baseline, we demonstrate that our system achieves vastly improved performance, constructing the bottleneck structure graphs in 0.21 s and calculating link derivatives in 0.09 s on average. We also study the asymptotic complexity of our core algorithms, demonstrating good scaling properties and strong agreement with theoretical bounds. These results indicate that our new software package can maintain its fast performance when applied to even larger networks. They also show that our software is efficient enough to analyze rapidly changing networks in real time. Overall, we demonstrate the feasibility of applying bottleneck structure analysis to solve practical problems in large, real-world data networks.

benchmark↗
DOE OSTI2022

IER-517: Molybdenum Optimized Benchmark System Demonstrating Integral Correlations (MOBY DICK)

Nuclear criticality experiments are essential to the validation of nuclear data used in simulation software. The quality of nuclear data becomes paramount as simulation software becomes more relied upon for criticality safety studies and designs of nuclear systems. To improve the quality of nuclear data, experimenters can design critical experiments that are sensitive to isotope reaction pairs in materials of interest. The efforts conducted by the Organisation for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA) Working Party on Nuclear Criticality Safety (WPNCS) Subgroup 8: Preservation of Expert Knowledge and Judgement Applied to Criticality Benchmarks (SG8) to categorize benchmarks according to their usefulness for nuclear data validation have been of great importance. Based on the OECD studies benchmark experiments included in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook are concisely used by nuclear data evaluators, criticality safety engineers and others to validate nuclear data and simulation results. A lack of benchmarks sensitive to molybdenum in the (ICSBEP), particularly in the intermediate range, was noted by Los Alamos National Laboratory (LANL), the French Institut de Radioprotection et de Sûreté Nucléaire (IRSN), and Y-12 National Security Site prompting them to submit a joint integral experiment request to the Nuclear Criticality Safety Program (NCSP) in 2019. The request included both HEU and Plutonium systems in order to validate differential nuclear data focusing on the intermediate energy range but also includes thermal and fast configurations. This document represents the preliminary design work for a series of molybdenum integral experiments known as Molybdenum Optimized Benchmark System Demonstrating Integral Correlations (MOBY DICK).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗