Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarking Software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Bison Verification and Validation Activities for TRISO

Numerical modeling and simulation (M&S) tools play a key role in the research, development, and overall safety assessments of next-generation nuclear energy systems. One such tool, Bison, is a nuclear fuel performance code that is applicable to many fuel forms (e.g., light-water reactor fuel, oxide and metallic fuel for fast reactors, tri-structural isotropic (TRISO) fuel, and plate fuel), and it uses the finite element method to model the thermo- mechanical response of nuclear fuels. One fuel form widely utilized in Generation-IV high-temperature gas-cooled and fluoride- salt-cooled nuclear reactor concepts is TRISO fuel. Recently, Bison’s capabilities were significantly expanded to enable it to model the performance of TRISO particles and compacts. It is important that Bison’s computational results be reliable and predictive, since this code is used to inform high-consequence decisions. The various processes developed to address this issue generally entail two fundamental steps: verification and validation (V&V). Verification ensures that the code functions correctly and is reliable. Code/solution verification, code benchmark, and software quality assurance exercises are examples of verification activities. On the other hand, validation is the process of assessing a code’s capability to accurately model physical problems. Comparisons between code results and experiments quantify the validation level. Application of V&V procedures is crucial to the development of computational tools that are free of coding mistakes and can accurately represent reality. The current study presents an overview of Bison V&V activities relevant to the TRISO fuel concept, which include code/solution verification exercises, CRP-6 Benchmark—a Coordinated Research Program through the International Atomic Energy Agency (IAEA)—exercises, and validation exercises with the Advanced Gas Reactor (AGR)- 1/2/3/4 experiment series.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Noisy intermediate-scale quantum algorithms

A universal fault-tolerant quantum computer that can efficiently solve problems such as integer factorization and unstructured database search requires millions of qubits with low error rates and long coherence times. While the experimental advancement toward realizing such devices will potentially take decades of research, noisy intermediate-scale quantum (NISQ) computers already exist. These computers are composed of hundreds of noisy qubits, i.e., qubits that are not error corrected, and therefore perform imperfect operations within a limited coherence time. In the search for achieving quantum advantage with these devices, algorithms have been proposed for applications in various disciplines spanning physics, machine learning, quantum chemistry, and combinatorial optimization. The overarching goal of such algorithms is to leverage the limited available resources to perform classically challenging tasks. In this review, a thorough summary of NISQ computational paradigms and algorithms is provided. The key structure of these algorithms and their limitations and advantages are discussed. Finally, a comprehensive overview of various benchmarking and software tools useful for programming and testing NISQ devices is additionally provided.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics

Abstract We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of $$C^{\infty }$$ C ∞ bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at https://github.com/MathBioCU/WENDy .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Benchmarking second and third-generation sequencing platforms for microbial metagenomics

Shotgun metagenomic sequencing is a common approach for studying the taxonomic diversity and metabolic potential of complex microbial communities. Current methods primarily use second generation short read sequencing, yet advances in third generation long read technologies provide opportunities to overcome some of the limitations of short read sequencing. Here, we compared seven platforms, encompassing second generation sequencers (Illumina HiSeq 300, MGI DNBSEQ-G400 and DNBSEQ-T7, ThermoFisher Ion GeneStudio S5 and Ion Proton P1) and third generation sequencers (Oxford Nanopore Technologies MinION R9 and Pacific Biosciences Sequel II). We constructed three uneven synthetic microbial communities composed of up to 87 genomic microbial strains DNAs per mock, spanning 29 bacterial and archaeal phyla, and representing the most complex and diverse synthetic communities used for sequencing technology comparisons. Our results demonstrate that third generation sequencing have advantages over second generation platforms in analyzing complex microbial communities, but require careful sequencing library preparation for optimal quantitative metagenomic analysis. Our sequencing data also provides a valuable resource for testing and benchmarking bioinformatics software for metagenomics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

NMSBA: Continuous Application Benchmarking & Analysis – CABA

The CABA project is a test of a new tool, Survey,developed by Trenza to be used not only for benchmarking or profiling programs but also to allow incorporation of the information provided by Survey to be utilized in a CI, continuous integration,tool such as GitLab CI.Survey is foremost a means of assessing code performance in terms of time and operations which for computer programmers is known as benchmarking.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1994

NASA High Performance Computing and Communications program

The National Aeronautics and Space Administration's HPCC program is part of a new Presidential initiative aimed at producing a 1000-fold increase in supercomputing speed and a 1(X)-fold improvement in available communications capability by 1997. As more advanced technologies are developed under the HPCC program, they will be used to solve NASA's 'Grand Challenge' problems, which include improving the design and simulation of advanced aerospace vehicles, allowing people at remote locations to communicate more effectively and share information, increasing scientists' abilities to model the Earth's climate and forecast global environmental trends, and improving the development of advanced spacecraft. NASA's HPCC program is organized into three projects which are unique to the agency's mission: the Computational Aerosciences (CAS) project, the Earth and Space Sciences (ESS) project, and the Remote Exploration and Experimentation (REE) project. An additional project, the Basic Research and Human Resources (BRHR) project, exists to promote long term research in computer science and engineering and to increase the pool of trained personnel in a variety of scientific disciplines. This document presents an overview of the objectives and organization of these projects, as well as summaries of early accomplishments and the significance, status, and plans for individual research and development programs within each project. Areas of emphasis include benchmarking, testbeds, software and simulation methods.

Holcomb, Lee↗
NASA NTRS2010

Permissible Exposure Level for Lunar Dusts: Gaps are Closing

Space faring nations plan to return human explorers to the moon within the next decade. Experience during the Apollo flights suggests that lunar dust will invariably get into the habitat where the finest portion (less than 5 micrometers) could be inhaled by the crew before it is cleared from the atmosphere. NASA is developing a database from which a 6-month, episodic exposure standard for lunar dust can be set. Three kinds of moon dust were prepared from a parent sample of Apollo 14 regolith #14003,96. Our goal was to prepare each type of dust sample with a mean diameter less than 2 m, which is suitable for instillation into the lungs of rats. The three samples were prepared as follows: separation from the parent sample using a fluidized bed, grinding using a jet mill grinder, or grinding with a ball-mill grinder. Grinding simulated restoration of surface activation of dust expected to occur at the surface of the moon on native lunar dust. We used two grinding methods because they seemed to produce different modes of activation. The effects of grinding were preserved by maintaining the dust in ultra-pure nitrogen until immediately before it was placed in suspension for administration to rats. The dust was suspended in physiological saline with 10% Survanta, a lung surfactant. Rats were given intratrachael instillations of the dust suspension at three doses. In addition to the three moon dusts (A, C and E), we instilled the same amount of a negative control (TiO2, B) and a highly-toxic, positive control (quartz, D). These additional mineral dusts were selected because they have well-established and very different permissible exposure levels (PELs). Our goal was to determine where lunar dusts fit between these extremes, and then estimate a PEL for each lunar dust. We evaluated many indices of toxicity to the lung. The figure shows the changes in lactate dehydrogenase (LDH), a marker of cell death, for the five dusts. Benchmark dose software (Version 2.1.2) from the Environmental Protection Agency was used to estimate the 10% effect levels (BMD(sub 10)) using five models. The best-fitting model was used to estimate the optimal BMD(sub 10) (table)

James, John T.↗
NASA NTRS2010

Modeling Respiratory Toxicity of Authentic Lunar Dust

The lunar expeditions of the Apollo operations from the 60 s and early 70 s have generated awareness about lunar dust exposures and their implication towards future lunar explorations. Critical analyses on the reports from the Apollo crew members suggest that lunar dust is a mild respiratory and ocular irritant. Currently, NASA s space toxicology group is functioning with the Lunar Airborne Dust Toxicity Assessment Group (LADTAG) and the National Institute for Occupational Safety and Health (NIOSH) to investigate and examine toxic effects to the respiratory system of rats in order to establish permissible exposure levels (PELs) for human exposure to lunar dust. In collaboration with the space toxicology group, LADTAG and NIOSH the goal of the present research is to analyze dose-response curves from rat exposures seven and twenty-eight days after intrapharyngeal instillations, and model the response using BenchMark Dose Software (BMDS) from the Environmental Protection Agency (EPA). Via this analysis, the relative toxicities of three types of Apollo 14 lunar dust samples and two control dust samples, titanium dioxide (TiO2) and quartz will be determined. This will be executed for several toxicity endpoints such as cell counts and biochemical markers in bronchoaveolar lavage fluid (BALF) harvested from the rats.

Santana, Patricia A.↗
DOE OSTI2021

Mesh independency in topology optimization

The topology optimization community has regularly employed optimization algorithms from the operations research community. However, these algorithms are implemented in the sequence space l2 instead of the proper function space where the design variable resides. In this software, we benchmark the mesh independent implementation of pyMMAopt with three common problems in topology optimization. We show how the volume fraction variable discretization on non-uniform meshes affects the convergence of l2 based optimization algorithms and how pyMMAopt successfully achieves mesh independent convergence.

Salazar De Troya, Miguel↗
DOE OSTI2024

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

Jarmusch, Aaron↗
NASA NTRSDate not supplied

Modelling Sonic Boom Propagation through Planetary Boundary Layer Turbulence near the Lateral Extent of the Carpet

Nonlinear acoustic propagation of sonic booms in the atmospheric boundary layer is considered in the context of a one-way solution to a third order wave equation. A split-step approach is utilized to compute different physical effects efficiently. Unlike previous approaches, the diffraction effects are computed exactly in the forward direction with no restriction on the angle of propagation. This results in more accurate modelling of sonic boom near the carpet edge. Heterogenous flow effects are incorporated with a wide-angle parabolic approximation. Nonlinear propagation is computed with a Burgers-Hayes method. Turbulence in the medium is constructed by the method of random Fourier modes. The turbulence spectra are constructed using an altitude dependent 3-dimensional von Karman spectrum. The turbulence field is considered frozen, as the eddy turnover time in the atmospheric boundary layer is generally much larger than the propagation time of the acoustic wave. Comparisons with benchmark predictions from the PCBoom software are conducted. The benchmark cases provide insight into the accuracy of the current prediction code for non-turbulent atmospheres. Future work is discussed regarding the prediction of shaped booms, boundary conditions, and predictions near the lateral extent of the boom carpet where previous approaches fail to capture the diffraction effects.

sonic boom↗
NASA NTRS2002

Multirate Flutter Suppression System Design for the Benchmark Active Controls Technology Wing: Methodology Application Software Toolbox - Part 2

To study the effectiveness of various control system design methodologies, the NASA Langley Research Center initiated the Benchmark Active Controls Project. In this project, the various methodologies were applied to design a flutter suppression system for the Benchmark Active Controls Technology (BACT) Wing. This report describes the user's manual and software toolbox developed at the University of Washington to design a multirate flutter suppression control law for the BACT wing.

Mason, Gregory S.↗
DOE OSTI2021

Benchmark Comparison of HyRAM and ALDEA Software for Hydrogen Release Behavior

There are several different calculation approaches and tools that can be used to evaluate the risk of hydrogen energy applications. A comparative study of Air Liquide’s ALDEA (Air Liquide Dispersion and Explosion Assessment) tools suite and Sandia’s HyRAM (Hydrogen Risk Assessment Models) toolkit has been conducted. The purpose of this study was to understand and evaluate the differences between the two calculation approaches, and identify areas for model improvements. There were several scenarios examined in this effort regarding hydrogen release dynamics. These scenarios include free jet release cases at varying pressures, vessel blowdown, and hydrogen build-up scenarios with and without ventilation. For each scenario, the input and output of the HyRAM calculations are documented, along with a comparison to the ALDEA results. Generally, the results from the two different tools were reasonably aligned. However, there were fundamental differences in evaluation methodology and functional limitations in HyRAM that caused discrepancies in some calculations.

08 HYDROGEN↗
NASA NTRSDate not supplied

Space Flown Rodent Liver RNA Sequencing Data for Machine Learning in Space Biology Research

High-throughput nucleic acid sequencing (DNA-seq, RNA-seq) has become widespread in biomedical research due to the growing availability and affordability of these assays. Data analysis has been accelerated in recent years by the adoption of artificial intelligence (AI) and machine learning (ML) techniques by biomedical researchers. In space biology research, RNAseq datasets from space-flown experimental samples are critical for characterizing the gene expression aberrations associated with exposure to spaceflight stressors. However, space biological experiments tend to be very low sample size, so identifying proper AI/ML algorithms for sequencing data analysis is an ongoing challenge since these algorithms typically require large sample size. The NASA Science Mission Directorate (SMD) has started the “Benchmark Initiative for AI/ML”, focused on creating datasets meant for three main applications: 1) scientific benchmarking, which finds the best algorithm for a specific problem; 2) application benchmarking, which measures algorithm performance against a set of parameters; and 3) system benchmarking, which evaluates performance of hardware and software architecture. These scientific benchmarks consist of an AI-ready dataset and a reference implementation on a specific scientific question. In this work, we focused on generating standardized datasets to allow the scientific community to benchmark AI/ML algorithms in the domain of space biology. We present here a standardized, AI-ready, publicly available benchmark dataset for space biology RNA-seq data as a collaboration between the NASA AI4LS (Artificial Intelligence for Life Sciences) working group. and NASA’s SMD. This dataset consists of space-flown and ground control mouse liver found in the NASA GeneLab omics database. However, to amplify the small sample number (n=112 samples) for ML purposes, we employ Gaussian noise and a generative adversarial network to extend this dataset to 6,000 synthetic samples, matching the original gene expression characteristics.

James Casaletto↗
DOE OSTI2024

HamLib: A library of Hamiltonians for benchmarking quantum algorithms and hardware

In order to characterize and benchmark computational hardware, software, and algorithms, it is essential to have many problem instances on-hand. This is no less true for quantum computation, where a large collection of real-world problem instances would allow for benchmarking studies that in turn help to improve both algorithms and hardware designs. To this end, here we present a large dataset of qubit-based quantum Hamiltonians. The dataset, called HamLib (for Hamiltonian Library), is freely available online and contains problem sizes ranging from 2 to 1000 qubits. HamLib includes problem instances of the Heisenberg model, Fermi-Hubbard model, Bose-Hubbard model, molecular electronic structure, molecular vibrational structure, MaxCut, Max- k -SAT, Max- k -Cut, QMaxCut, and the traveling salesperson problem. The goals of this effort are (a) to save researchers time by eliminating the need to prepare problem instances and map them to qubit representations, (b) to allow for more thorough tests of new algorithms and hardware, and (c) to allow for reproducibility and standardization across research studies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

An HPC benchmark survey and taxonomy for characterization

The field of High-Performance Computing (HPC) is defined by providing computing devices with highest performance for a variety of demanding scientific users. The tight co-design relationship between HPC providers and users propels the field forward, paired with technological improvements, achieving continuously higher performance and resource utilization. A key device for system architects, architecture researchers, and scientific users are benchmarks, allowing for well-defined assessment of hardware, software, and algorithms. Many benchmarks exist in the community, from individual niche benchmarks testing specific features, to large-scale benchmark suites for whole procurements. We survey the available HPC benchmarks, summarizing them in table form with key details and concise categorization, also through an interactive website. For categorization, we present a benchmark taxonomy for well-defined characterization of benchmarks.

Benchmarking↗
DOE OSTI2025

Benchmarking quantum computers

The rapid pace of development in quantum computing technology has sparked a proliferation of benchmarks to assess the performance of quantum computing hardware and software. However, not all benchmarks are of equal merit. Good ones empower scientists, engineers, programmers and users to understand the power of a computing system, whereas bad ones can misdirect research and inhibit progress. In this Perspective, we survey the science of quantum computer benchmarking. Here, we discuss the role of benchmarks and benchmarking and how good benchmarks can drive and measure progress towards the long-term goal of useful quantum computations, known as quantum utility. We explain how different kinds of benchmark quantify the performance of different parts of a quantum computer, discuss existing benchmarks, examine recent trends in benchmarking, and highlight important open research questions in this field.

Proctor, Timothy James [Sandia National Laboratori↗