Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarking Software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Evaluating Performance of Components

Parallel Component Performance Benchmarks is a computer program developed to aid the evaluation of the Common Component Architecture (CCA) - a software architecture, based on a component model, that was conceived to foster high-performance computing, including parallel computing. More specifically, this program compares the performances (principally by measuring computing times) of componentized versus conventional versions of the Parallel Pyramid 2D Adaptive Mesh Refinement library - a software library that is used to generate computational meshes for solving physical problems and that is typical of software libraries in use at NASA s Jet Propulsion Laboratory.

Katz, Daniel↗
DOE OSTI2026

Hardware Aware Mitigation of Timing Side-Channel Vulnerabilities in Critical Infrastructure Software

Program runtime/timing attacks exploit variations in a program’s execution times to extract sensitive information from the program (e.g. encryption keys, sensitive variable data, intellectual property). State-of-the-art solutions to runtime sidechannel attacks attempt to balance the execution time of the sensitive code for different control flow paths to eliminate the timing leakage. However, during the mitigation process, most techniques do not consider the underlying hardware/device on which the target program is supposed to run on. This can lead to over-fixing (unnecessary extra operations), under-fixing (not solving the imbalance properly), and even failures. We propose DISARM, a joint hardware-software methodology (unlike any existing solution) for mitigating runtime side-channel vulnerabilities that utilizes timing values from real embedded devices to generate targeted software fixes. We implement DISARM to support C/C++/Java source codes and validate it across 22 standard benchmarks. DISARM outperforms state-of-the-art solutions such as PENDULUM and DifFuzzAR in terms of execution time overhead (up to −46%), code size overhead (up to −10%), and correctness (no failures) on five different embedded/edge devices.

Suha, Tasneem [University of Maine]↗
DOE OSTI2022

UPDATES FROM THE INVOLUTE WORKING GROUP

The HFIR, RHF, and FRM II reactors represent a particular class of Research and Test Reactors that provide some of the most intense and continuous neutron fluxes for science, industry, and medical applications. These high-performance reactors have achieved compact cores by operating with Highly Enriched Uranium fuel (HEU, 235U/U ≥ 20 wt. %) and utilizing fuel plates curved as an involute. Due to the proliferation risks, the international community aims to reduce or eliminate, when possible, the use of HEU fuel in civilian facilities by converting them to a Low-Enriched Uranium fuel (LEU, 235U/U < 20 wt. %). Conversion of these reactors without significantly compromising their performance or safety is a challenging endeavor that can tremendously benefit from advanced computational tools and thus, eliminate unnecessary conservatism to ensure sufficient thermal margins. Therefore, models are being developed using modern Computational Fluid Dynamics (CFD) and Computational Structural Mechanics (CSM) software to evaluate the steady-state safety margins of various LEU designs instead of being reliant on the more traditional, conservative methods. To gain the confidence and acceptance of high-fidelity modeling by the nuclear regulators, Argonne National Laboratory (ANL) and the involute reactors have formed an informal scientific group, the Involute Working Group (IWG). The IWG facilitates inter-organizational collaboration on experimental benchmarking, code-to-code comparisons, and Verification and Validation (V&V). This paper describes some of the recent IWG efforts in validating software against the existing experimental data, as well as code-to-code comparisons of different software used by the IWG members.

Bergeron, Aurelien↗
DOE OSTI2021

Validating Intra-Plate Power Distributions in the Advanced Test Reactor

We report that the neutronics software, HELIOS, was validated in 2015 for performing core reload design and safety analysis of the Advanced Test Reactor. However, when HELIOS was benchmarked against historic fission-wire measurements (i.e., zero-power full-core measurements) a statistically resolved calculation-to-measurement bias was discovered. The azimuthal power along each fuel plate computed by HELIOS has consistently shown to under-predict measurements made by fission-wires in historic zero-power tests near the fuel element side-plates. It was hypothesized during the HELIOS software validation work, that this bias is attributable to local moderation in coolant vents in the side-plates axially just above and below the fission-wires on the fuel-plate edges. This work used detailed MCNP and MC21 models of the side-plate vents to test this hypothesis. By comparing the average azimuthal biases between HELIOS, and 2D and 3D MCNP models, and a 3D MC21 model, it was found that the HELIOS azimuthal bias is not due to the measurement.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS2012

Development and Application of Benchmark Examples for Mixed-Mode I/II Quasi-Static Delamination Propagation Predictions

The development of benchmark examples for quasi-static delamination propagation prediction is presented and demonstrated for a commercial code. The examples are based on finite element models of the Mixed-Mode Bending (MMB) specimen. The examples are independent of the analysis software used and allow the assessment of the automated delamination propagation prediction capability in commercial finite element codes based on the virtual crack closure technique (VCCT). First, quasi-static benchmark examples were created for the specimen. Second, starting from an initially straight front, the delamination was allowed to propagate under quasi-static loading. Third, the load-displacement relationship from a propagation analysis and the benchmark results were compared, and good agreement could be achieved by selecting the appropriate input parameters. Good agreement between the results obtained from the automated propagation analysis and the benchmark results could be achieved by selecting input parameters that had previously been determined during analyses of mode I Double Cantilever Beam and mode II End Notched Flexure specimens. The benchmarking procedure proved valuable by highlighting the issues associated with choosing the input parameters of the particular implementation. Overall the results are encouraging, but further assessment for mixed-mode delamination fatigue onset and growth is required.

Krueger, Ronald↗
DOE OSTI2025

Logical error rates for the surface code under a mixed coherent and stochastic circuit-level noise model inspired by trapped ions

With fault-tolerant quantum computing (FTQC) on the horizon, it is critical to understand sources of logical errors in plausible hardware implementations of quantum error-correcting codes. Detailed error modeling of computational instructions on particular FTQC architectures will enable the better prediction of error propagation in FT-encoded quantum circuits while revealing where greater attention is needed in hardware design. In this work, we consider logical error rates for the surface code implemented on a hypothetical grid-based trapped-ion quantum charge-coupled device architecture. Specifically, we construct logical channels for the idling surface code and examine its diamond error under a mixed coherent and stochastic circuit-level noise model inspired by trapped ions. We include the coherent dephasing noise that is known to accumulate during physical qubit idling and transport in these systems, determining idling and transport durations using the time-resolved output of an open-source trapped-ion surface code compiler. To estimate expectation values of logical Pauli observables following hardware circuits containing non-Clifford sources of noise, we utilize a Monte Carlo technique to sample from an underlying quasiprobability distribution of Clifford circuits that we independently simulate in a phase-sensitive fashion. We verify error suppression up to code distance 𝑑 = 11 at coherent dephasing rates near and below those of current-generation trapped-ion quantum computers and find that logical error rates align with those of analogous fully stochastic simulations in this regime. Exploring higher dephasing rates at 𝑑 = 3−5, we find evidence for growing coherent rotations about all three logical Pauli axes, increased diagonal logical error process matrix elements relative to those of stochastic simulations, and a reduced dephasing rate threshold. Overall, our work paves a way toward realistic hardware emulation of small fault-tolerant quantum processes, e.g., members of an FTQC instruction set.

Quantum benchmarking↗
NASA NTRS2011

Advanced Technology Subsonic Transport Study: N+3 Technologies and Design Concepts

Conceptual Research Corporation, the Science of the Possible, has completed a two-year study of concepts and technologies for future airliners in the 180-passenger class. This NASA-funded contract was primarily focused on the ambitious goal of a 70 percent reduction in fuel consumption versus the market-dominating Boeing 737-800. The study is related to the N+3 contracts awarded in 2008 by NASA s Aeronautics Research Mission Directorate to teams led by Boeing, GE Aviation, MIT, and Northrop Grumman, but with more modest goals and funding. CRC s contract featured a predominant emphasis on propulsion and fuel consumption, but since fuel consumption depends upon air vehicle design as much as on propulsion technology, the study included notional vehicle design, analysis, and parametric studies. Other NASA goals including NOx and noise reduction are of long-standing interest but were not highlighted in this study, other than their inclusion in the propulsion system provided to CRC by NASA. The B-737-800 was used as a benchmark, parametric tool, and design point of departure. It was modeled in the RDS-Professional aircraft design software then subjected to extensive parametric variations of parasitic drag, drag-due-to-lift, specific fuel consumption, and unsized empty weight. These studies indicated that the goal of a 70 percent reduction in fuel consumption could be attained with roughly a 30 percent improvement in all four parameters. The results were then fit to a Response Surface and coded for ease of use in subsequent trade studies. Potential technologies to obtain such savings were identified and discussed. More than 16 advanced concept designs were then prepared, attempting to investigate almost every possible emerging concept for application to this class airliner. A preliminary assessment of these concepts was done based on their total wetted area after design normalization of trimmed maximum lift. This assessment points towards a Tailless Airliner concept which was designed and analyzed in some detail. To make it work, a retracting canard and an all-moving chin rudder were employed, along with the use of the Active Aeroelastic Wing technology. Results indicate that a 60 percent savings in fuel burn may be credibly attained, but this depends upon a lot of technology maturation, concept development, and risk reduction. This should be expected-such a dramatic reduction in fuel consumption is a "game changer" in the world of commercial aviation. It won t be easy.

Raymer, Daniel P.↗
NASA NTRS2013

The GLAS Science Algorithm Software (GSAS) Detailed Design Document Version 6

The Geoscience Laser Altimeter System (GLAS) is the primary instrument for the ICESat (Ice, Cloud and Land Elevation Satellite) laser altimetry mission. ICESat was the benchmark Earth Observing System (EOS) mission for measuring ice sheet mass balance, cloud and aerosol heights, as well as land topography and vegetation characteristics. From 2003 to 2009, the ICESat mission provided multi-year elevation data needed to determine ice sheet mass balance as well as cloud property information, especially for stratospheric clouds common over polar areas. It also provided topography and vegetation data around the globe, in addition to the polar-specific coverage over the Greenland and Antarctic ice sheets.This document describes the detailed design of GLAS Science Algorithm Software (GSAS). The GSAS is used to create the ICESat GLAS standard data products. The National Snow and Ice Data Center (NSDIC) distribute these products. The document contains descriptions, flow charts, data flow diagrams, and structure charts for each major component of the GSAS. The purpose of this document is to present the detailed design of the GSAS. It is intended as a reference source to assist the maintenance programmer in making changes that fix or enhance the documented software.

Lee, Jeffrey E.↗
NASA NTRS2013

The GLAS Science Algorithm Software (GSAS) User's Guide Version 7

The Geoscience Laser Altimeter System (GLAS) is the primary instrument for the ICESat (Ice, Cloud and Land Elevation Satellite) laser altimetry mission. ICESat was the benchmark Earth Observing System (EOS) mission for measuring ice sheet mass balance, cloud and aerosol heights, as well as land topography and vegetation characteristics. From 2003 to 2009, the ICESat mission provided multi-year elevation data needed to determine ice sheet mass balance as well as cloud property information, especially for stratospheric clouds common over polar areas. It also provided topography and vegetation data around the globe, in addition to the polar-specific coverage over the Greenland and Antarctic ice sheets.This document is the final version of the GLAS Science Algorithm Software Users Guide document. It contains the instructions to install the GLAS Science Algorithm Software (GSAS) in the production environment that was used to create the standard data products. It also describes the usage of each GSAS program in that environment with their required inputs and outputs. Included are a number of utility programs that are used to create ancillary data files that are used in the processing but generally are not distributed to the public as data products. Of importance is the values for the large number of constants used in the GSAS algorithm during processing are provided in an appendix.

Lee, Jeffrey E.↗
NASA NTRS1986

Ada education in a software life-cycle context

Some of the experience gained from a comprehensive educational program undertaken at The Charles Stark Draper Lab. to introduce the Ada language and to transition modern software engineering technology into the development of Ada and non-Ada applications is described. Initially, a core group, which included manager, engineers and programmers, received training in Ada. An Ada Office was established to assume the major responsibility for training, evaluation, acquisition and benchmarking of tools, and consultation on Ada projects. As a first step in this process, and in-house educational program was undertaken to introduce Ada to the Laboratory. Later, a software engineering course was added to the educational program as the need to address issues spanning the entire software life cycle became evident. Educational efforts to date are summarized, with an emphasis on the educational approach adopted. Finally, lessons learned in administering this program are addressed.

Clough, Anne J.↗
NASA NTRS2011

Development of Benchmark Examples for Static Delamination Propagation and Fatigue Growth Predictions

The development of benchmark examples for static delamination propagation and cyclic delamination onset and growth prediction is presented and demonstrated for a commercial code. The example is based on a finite element model of an End-Notched Flexure (ENF) specimen. The example is independent of the analysis software used and allows the assessment of the automated delamination propagation, onset and growth prediction capabilities in commercial finite element codes based on the virtual crack closure technique (VCCT). First, static benchmark examples were created for the specimen. Second, based on the static results, benchmark examples for cyclic delamination growth were created. Third, the load-displacement relationship from a propagation analysis and the benchmark results were compared, and good agreement could be achieved by selecting the appropriate input parameters. Fourth, starting from an initially straight front, the delamination was allowed to grow under cyclic loading. The number of cycles to delamination onset and the number of cycles during stable delamination growth for each growth increment were obtained from the automated analysis and compared to the benchmark examples. Again, good agreement between the results obtained from the growth analysis and the benchmark results could be achieved by selecting the appropriate input parameters. The benchmarking procedure proved valuable by highlighting the issues associated with the input parameters of the particular implementation. Selecting the appropriate input parameters, however, was not straightforward and often required an iterative procedure. Overall, the results are encouraging but further assessment for mixed-mode delamination is required.

Kruger, Ronald↗
NASA NTRS2011

Development and Application of Benchmark Examples for Mode II Static Delamination Propagation and Fatigue Growth Predictions

The development of benchmark examples for static delamination propagation and cyclic delamination onset and growth prediction is presented and demonstrated for a commercial code. The example is based on a finite element model of an End-Notched Flexure (ENF) specimen. The example is independent of the analysis software used and allows the assessment of the automated delamination propagation, onset and growth prediction capabilities in commercial finite element codes based on the virtual crack closure technique (VCCT). First, static benchmark examples were created for the specimen. Second, based on the static results, benchmark examples for cyclic delamination growth were created. Third, the load-displacement relationship from a propagation analysis and the benchmark results were compared, and good agreement could be achieved by selecting the appropriate input parameters. Fourth, starting from an initially straight front, the delamination was allowed to grow under cyclic loading. The number of cycles to delamination onset and the number of cycles during delamination growth for each growth increment were obtained from the automated analysis and compared to the benchmark examples. Again, good agreement between the results obtained from the growth analysis and the benchmark results could be achieved by selecting the appropriate input parameters. The benchmarking procedure proved valuable by highlighting the issues associated with choosing the input parameters of the particular implementation. Selecting the appropriate input parameters, however, was not straightforward and often required an iterative procedure. Overall the results are encouraging, but further assessment for mixed-mode delamination is required.

Krueger, Ronald↗
DOE OSTI2022

SBbadger: biochemical reaction networks with definable degree distributions

Abstract Motivation An essential step in developing computational tools for the inference, optimization and simulation of biochemical reaction networks is gauging tool performance against earlier efforts using an appropriate set of benchmarks. General strategies for the assembly of benchmark models include collection from the literature, creation via subnetwork extraction and de novo generation. However, with respect to biochemical reaction networks, these approaches and their associated tools are either poorly suited to generate models that reflect the wide range of properties found in natural biochemical networks or to do so in numbers that enable rigorous statistical analysis. Results In this work, we present SBbadger, a python-based software tool for the generation of synthetic biochemical reaction or metabolic networks with user-defined degree distributions, multiple available kinetic formalisms and a host of other definable properties. SBbadger thus enables the creation of benchmark model sets that reflect properties of biological systems and generate the kinetics and model structures typically targeted by computational analysis and inference software. Here, we detail the computational and algorithmic workflow of SBbadger, demonstrate its performance under various settings, provide sample outputs and compare it to currently available biochemical reaction network generation software. Availability and implementation SBbadger is implemented in Python and is freely available at https://github.com/sys-bio/SBbadger and via PyPI at https://pypi.org/project/SBbadger/. Documentation can be found at https://SBbadger.readthedocs.io. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Performance Improvements for the Griffin Transport Solvers

Griffin is a Multiphysics Object-Oriented Simulation Environment based reactor multiphysics analysis application jointly developed by Idaho National Laboratory and Argonne National Laboratory. Griffin includes a variety of deterministic radiation transport solvers for fixed source, k-eigenvalue, adjoint, and subcritical multiplication, as well as transient solvers for point-kinetics, improved quasi-static, and spatial dynamics. A code assessment performed in FY-20 identified two significant issues with the transport solvers in Griffin: first, the primary heterogeneous SN (discrete ordinates) transport solver based on continuous finite element methods required significant mesh refinement and higher memory usage compared to solvers based on the method of characteristic for equivalent accuracy. Second, the homogeneous PN (spherical harmonics expansion) transport solver did not adequately support polynomial refinement, which is a feature usually required for problems with spatial homogenization and pronounced streaming, typical in fast or gas-cooled reactor systems. To address the first issue, the development effort focused on the more promising discontinuous finite element method (DFEM)-based SN transport solver in Griffin. The addition of an asynchronous parallel transport sweeper and coarse mesh finite difference (CMFD) acceleration have rendered a superior heterogeneous SN transport capability for multiphysics problems that requires far less computing resources in terms of both CPU time and memory usage. This is demonstrated with typical thermal- and fast-spectrum reactor benchmark problems, including 2D Transient Reactor Test, 3D Advanced Burner Test Reactor (ABTR), and 2D and 3D Empire microreactor. For the second issue, the development effort focused on a new transport solver based on the hybrid finite element PN method (HFEM-PN), equivalent to the variational nodal method, as well as a new diffusion solver based on HFEM-Diffusion. This solver is intended for homogenized domains with multiphysics coupling (i.e., supports mesh displacement, seamless temperature feedback, etc.). Initial calculations with the HFEM-Diffusion implementation show very good parallel efficiency for the residual evaluations with the 2D ABTR benchmark. A future development effort will be centered on further improvements to the CMFD, HFEM-PN, and DFEM diffusion solvers to ensure Griffin meets performance and software quality assurance requirements for advanced reactor design and analysis.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Integrating quantum computing resources into scientific HPC ecosystems

Quantum Computing (QC) offers significant potential to enhance scientific discovery in fields such as quantum chemistry, optimization, and artificial intelligence. Yet QC faces challenges due to the noisy intermediate-scale quantum era’s inherent external noise issues. Here, this paper discusses the integration of QC as a computational accelerator within classical scientific high-performance computing (HPC) systems. By leveraging a broad spectrum of simulators and hardware technologies, we propose a hardware-agnostic framework for augmenting classical HPC with QC capabilities. Drawing on the HPC expertise of the Oak Ridge National Laboratory (ORNL) and the HPC lifecycle management of the Department of Energy (DOE), our approach focuses on the strategic incorporation of QC capabilities and acceleration into existing scientific HPC workflows. This includes detailed analyses, benchmarks, and code optimization driven by the needs of the DOE and ORNL missions. Our comprehensive framework integrates hardware, software, workflows, and user interfaces to foster a synergistic environment for quantum and classical computing research. This paper outlines plans to unlock new computational possibilities, driving forward scientific inquiry and innovation in a wide array of research domains.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

Solving complex eigenvalue problems on a quantum annealer with applications to quantum scattering resonances

Quantum computing is a new and rapidly evolving paradigm for solving chemistry problems. In previous work, we developed the Quantum Annealer Eigensolver (QAE) and applied it to the calculation of the vibrational spectrum of a molecule on the D-Wave quantum annealer. However, the original QAE methodology was applicable to real symmetric matrices only. For many physics and chemistry problems, the diagonalization of complex matrices is required. For example, the calculation of quantum scattering resonances can be formulated as a complex eigenvalue problem where the real part of the eigenvalue is the resonance energy and the imaginary part is proportional to the resonance width. In the present work, we generalize the QAE to treat complex matrices: first complex Hermitian matrices and then complex symmetric matrices. These generalizations are then used to compute a quantum scattering resonance state in a 1D model potential for O + O collisions. These calculations are performed using both a software (classical) annealer and hardware annealer (the D-Wave 2000Q). The results of the complex QAE are also benchmarked against a standard linear algebra library (LAPACK). Furthermore, this work presents the first numerical solution of a complex eigenvalue problem of any kind on a quantum annealer, and it is the first treatment of a quantum scattering resonance on any quantum device.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Equilipy: a python package for calculating phase equilibria

The CALPHAD (CALculation of PHAse Diagram) approach (Nigel Saunders & Miodownik, 1998) provides predictions for thermodynamically stable phases in multicomponent-multiphase materials across a wide range of temperatures. Consequently, the CALPHAD calculations became an essential tool in materials and process design (Luo, 2015). Such design tasks frequently require navigating a high-dimensional space due to multiple components involved in the system. This increasing complexity demands high-throughput CALPHAD calculations, especially in the rapidly evolving field of alloy design. In response to the need, we developed Equilipy an open-source Python package designed for calculating phase equilibria of multicomponent-multiphase systems. Equilipy is specifically tailored for high-throughput CALPHAD calculations, offering parallel computations across multiple processors and nodes with the given NPT input conditions namely elemental compositions (N), pressure (P), and temperature (T). Equilipy utilizes the program structure and Gibbs energy functions from the Fortran-based program, Thermochimica (Piro et al., 2013), with incorporating a new Gibbs energy minimization algorithm. This algorithm, originally developed by Capitani and Brown in 1987 (Capitani & Brown, 1987), has been revised and implemented to enhance the stability and performance of calculations. The Fortran codes are precompiled and interfaced with Python via F2PY, ensuring high computation speed. Benchmark tests shown in Figure 1 demonstrate that Equilipy’s computation speed is comparable to those of established commercial software, TC-Python and PanPython. This result highlights its efficiency and potential applications in various scientific and industrial fields.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2009

A New Monte Carlo Filtering Method for the Diagnosis of Mission-Critical Failures

Testing large-scale systems is expensive in terms of both time and money. Running simulations early in the process is a proven method of finding the design faults likely to lead to critical system failures, but determining the exact cause of those errors is still time-consuming and requires access to a limited number of domain experts. It is desirable to find an automated method that explores the large number of combinations and is able to isolate likely fault points. Treatment learning is a subset of minimal contrast-set learning that, rather than classifying data into distinct categories, focuses on finding the unique factors that lead to a particular classification. That is, they find the smallest change to the data that causes the largest change in the class distribution. These treatments, when imposed, are able to identify the settings most likely to cause a mission-critical failure. This research benchmarks two treatment learning methods against standard optimization techniques across three complex systems, including two projects from the Robust Software Engineering (RSE) group within the National Aeronautics and Space Administration (NASA) Ames Research Center. It is shown that these treatment learners are both faster than traditional methods and show demonstrably better results.

Gay, Gregory↗