Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Best practices and benchmarks for intact protein analysis for top-down mass spectrometry

One gene can give rise to many functionally distinct proteoforms, each of which has a characteristic molecular mass. Many laboratories, however, forgo the challenging mass analysis of intact proteins. Instead, these researchers resort to well-established methods for proteolytic digests. In this work, the Consortium for Top-Down Proteomics (CTDP) addresses the unmet need for robust protocols for obtaining intact protein mass spectra. We quantify and rationalize the signal suppression associated with common interfering substances. A decision tree, based upon sample composition and experimental goals, guides researchers to a given protocol. Cross-platform analytical benchmarks are provided to allow users to gauge their proficiency and to promote method standardization. This work should permit users at any experience level to obtain high-quality intact mass spectra of native and denatured proteins from mixtures of varying complexity, including intact antibodies and membrane proteins.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

A multi-scale time-series dataset with benchmark for machine learning in decarbonized energy grids

The electric grid is a key enabling infrastructure for the ambitious transition towards carbon neutrality as we grapple with climate change. With deepening penetration of renewable resources, the reliable operation of the electric grid becomes increasingly challenging. In this paper, we present PSML, a first-of-its-kind open-access multi-scale time-series dataset, to aid in the development of data-driven machine learning (ML)-based approaches towards reliable operation of future electric grids. The dataset is synthesized from a joint transmission and distribution electric grid to capture the increasingly important interactions and uncertainties of the grid dynamics, containing power, voltage and current measurements over multiple spatio-temporal scales. Using PSML, we provide state-of-the-art ML benchmarks on three challenging use cases of critical importance to achieve: (i) early detection, accurate classification and localization of dynamic disturbances; (ii) robust hierarchical forecasting of load and renewable energy; and (iii) realistic synthetic generation of physical-law-constrained measurements. We envision that this dataset will provide use-inspired ML research in safety-critical systems, while simultaneously enabling ML researchers to contribute towards decarbonization of energy sectors.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Benchmarking second and third-generation sequencing platforms for microbial metagenomics

Shotgun metagenomic sequencing is a common approach for studying the taxonomic diversity and metabolic potential of complex microbial communities. Current methods primarily use second generation short read sequencing, yet advances in third generation long read technologies provide opportunities to overcome some of the limitations of short read sequencing. Here, we compared seven platforms, encompassing second generation sequencers (Illumina HiSeq 300, MGI DNBSEQ-G400 and DNBSEQ-T7, ThermoFisher Ion GeneStudio S5 and Ion Proton P1) and third generation sequencers (Oxford Nanopore Technologies MinION R9 and Pacific Biosciences Sequel II). We constructed three uneven synthetic microbial communities composed of up to 87 genomic microbial strains DNAs per mock, spanning 29 bacterial and archaeal phyla, and representing the most complex and diverse synthetic communities used for sequencing technology comparisons. Our results demonstrate that third generation sequencing have advantages over second generation platforms in analyzing complex microbial communities, but require careful sequencing library preparation for optimal quantitative metagenomic analysis. Our sequencing data also provides a valuable resource for testing and benchmarking bioinformatics software for metagenomics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Increasing the hardness of posiform planting using random QUBOs for programmable quantum annealer benchmarking

Posiform planting is a method for constructing QUBO instances with a unique planted solution that can be tailored to arbitrary connectivity graphs. In this study we investigate making posiform planted QUBOs computationally harder by fusing many smaller random Ising models, whose global minimum is computed classically, with posiform planted QUBOs. The unique ground state of the resulting QUBO is the concatenation of (exactly one of) the ground states of each smaller problem. Our method generates QUBO instances that have a unique solution, are native to the hardware graph, and have tunable computational hardness. We use our QUBOs to benchmark three D-Wave quantum annealing processors (with 563–5627 qubits), and compare them against simulated annealing and Gurobi. Surprisingly, we find that the D-Wave ground state sampling success rate is not dependent on the glued random QUBO size, and that some QUBO classes are solved at high success rates at short annealing times on the Zephyr processors.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2018

Unified field studies of the algae testbed public-private partnership as the benchmark for algae agronomics

National scale agronomic projections are an important input for assessing potential benefits of algae cultivation on the future of innovative agriculture. The Algae Testbed Public-Private Partnership was established with the goal of investigating open pond algae cultivation across different geographic, climatic, seasonal, and operational conditions while setting the benchmark for quality data collection, analysis, and dissemination. Identical algae cultivation systems and data analysis methodologies were established at testbed sites across the continental United States and Hawaii. Within this framework, the Unified Field Studies were designed for algae cultivation during all 4 seasons across the testbed network. With increasingly diverse algae research and development, and field deployment strategies, the challenges associated with data collection, quality, and dissemination increase dramatically. The dataset presented here is the complete, curated, climatic, cultivation, harvest, and biomass composition data for each season at each site. Furthermore, these data enable others to do in-depth cultivation, harvest, techno-economic, life cycle, resource, and predictive growth modelling analysis, as well as development of crop protection strategies throughout the algae cultivation industry.

09 BIOMASS FUELS↗
DOE OSTI2022

Inverse deep learning methods and benchmarks for artificial electromagnetic material design

In this work we investigate the use of deep inverse models (DIMs) for designing artificial electromagnetic materials (AEMs) – such as metamaterials, photonic crystals, and plasmonics – to achieve some desired scattering properties (e.g., transmission or reflection spectrum). DIMs are deep neural networks (i.e., deep learning models) that are specially-designed to solve ill-posed inverse problems. There has recently been tremendous growth in the use of DIMs for solving AEM design problems however there has been little comparison of these approaches to examine their absolute and relative performance capabilities. In this work we compare eight state-of-the-art DIMs on three unique AEM design problems, including two models that are novel to the AEM community. Our results indicate that DIMs can rapidly produce accurate designs to achieve a custom desired scattering on all three problems. Although no single model always performs best, the Neural-Adjoint approach achieves the best overall performance across all problem settings. As a final contribution we show that not all AEM design problems are ill-posed, and in such cases a conventional deep neural network can perform better than DIMs. We recommend that a deep neural network is always employed as a simple baseline approach when addressing AEM design problems. Furthermore, we publish python code for our AEM simulators and our DIMs to enable easy replication of our results, and benchmarking of new DIMs by the AEM community.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Benchmark relativistic delta-coupled-cluster calculations of K-edge core-ionization energies of third-row elements

Here a benchmark computational study of K-edge core-ionization energies of third-row elements using relativistic delta-coupled-cluster (ΔCC) methods and a revised core-valence separation (CVS) scheme is reported. High-level relativistic (HLR) corrections beyond the spin-free exact two-component theory in its one-electron variant (SFX2C-1e), including the contributions from two-electron picture-change effects, spin–orbit coupling, the Breit term, and quantum electrodynamics effects, have been taken into account and demonstrated to play an important role. Relativistic ΔCC calculations are shown to provide accurate results for core-ionization energies of third-row elements. The SFX2C-1e-CVS-ΔCC results augmented with HLR corrections show a maximum deviation of less than 0.5 eV with respect to experimental values.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Packaging HEP Heterogeneous Mini-apps for Portable Benchmarking and Facility Evaluation on Modern HPCs

High Energy Physics (HEP) experiments are making increasing use of GPUs and GPU dominated High Performance Computer facilities. Both the software and hardware of these systems are rapidly evolving, creating challenges for experiments to make informed decisions as to where they wish to devote resources. In its first phase, the High Energy Physics Center for Computational Excellence (HEP-CCE) produced portable versions of a number of heterogeneous HEP mini-apps, such as p2r, FastCaloSim, Patatrack and the WireCell Toolkit, that exercise a broad range of GPU characteristics, enabling cross platform and facility benchmarking and evaluation. However, these miniapps still require a significant amount of manual intervention to deploy on a new facility. We present our work in developing turn-key deployments of these mini-apps, where by means of containerization and automated configuration and build techniques such as Spack, we are able to quickly test new hardware, software, environments and entire facilities with minimal user intervention, and then track performance metrics over time.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

The 0E2 benchmarks for PWR UO 2 decay heat: an analysis from the NEA WPNCS

This paper presents the work performed in the subgroup 16 of the Working Party for Nuclear Criticality Safety (WPNCS) of the OECD Nuclear Energy Agency. The main goal was to define two decay heat benchmarks for Spent Nuclear Fuel (one pincell and one assembly), perform calculations and compare and analyze the results in light of existing calorimetric measurements. The selected case is the PWR UO2 assembly 0E2, irradiated at the Ringhals-3 reactor and measured at the Clab facility in Sweden. In total, 21 institutes worldwide participated to the exercise, leading to 55 calculated results (named C). It was found that the measured decay heat values (E) can be satisfactorily reproduced with two-dimensional assembly calculations, leading to an average C/E value of 0.99, with an uncertainty (or one standard deviation) of ±0.01.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2016

Comparing the accuracy of high-dimensional neural network potentials and the systematic molecular fragmentation method: A benchmark study for all-trans alkanes

Many approaches, which have been developed to express the potential energy of large systems, exploit the locality of the atomic interactions. A prominent example is the fragmentation methods in which the quantum chemical calculations are carried out for overlapping small fragments of a given molecule that are then combined in a second step to yield the system’s total energy. Here we compare the accuracy of the systematic molecular fragmentation approach with the performance of high-dimensional neural network (HDNN) potentials introduced by Behler and Parrinello. HDNN potentials are similar in spirit to the fragmentation approach in that the total energy is constructed as a sum of environment-dependent atomic energies, which are derived indirectly from electronic structure calculations. As a benchmark set, we use all-trans alkanes containing up to eleven carbon atoms at the coupled cluster level of theory. These molecules have been chosen because they allow to extrapolate reliable reference energies for very long chains, enabling an assessment of the energies obtained by both methods for alkanes including up to 10 000 carbon atoms. We find that both methods predict high-quality energies with the HDNN potentials yielding smaller errors with respect to the coupled cluster reference.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Benchmarking semiclassical approaches to strong-field QED: Nonlinear Compton scattering in intense laser pulses

The recoil associated with photon emission is key to the dynamics of ultrarelativistic electrons in strong electromagnetic fields, as found in high-intensity laser-matter interactions and astrophysical environments such as neutron star magnetospheres. When the energy of the photon becomes comparable to that of the electron, it is necessary to use quantum electrodynamics (QED) to describe the dynamics accurately. However, computing the appropriate scattering matrix element from strong-field QED is not generally possible due to multiparticle effects and the complex structure of the electromagnetic fields. Therefore, these interactions are treated semiclassically, coupling probabilistic emission events to classical electrodynamics using rates calculated in the locally constant field approximation. Here, we provide comprehensive benchmarking of this approach against the exact QED calculation for nonlinear Compton scattering of electrons in an intense laser pulse. We find agreement at the percentage level between the photon spectra, as well as between the models' predictions of absorption from the background field, for normalized amplitudes a0 > 5. We discuss possible routes towards improved numerical methods and the implications of our results for the study of QED cascades.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2019

Implementation of analytic gradients for CCSD and EOM-CCSD using Cholesky decomposition of the electron-repulsion integrals and their derivatives: Theory and benchmarks

We present a general formulation of analytic nuclear gradients for the coupled-cluster with single and double substitution (CCSD) and equation-of-motion (EOM) CCSD energies computed using Cholesky decomposition (CD) representations of the electron repulsion integrals. By rewriting the correlated energy and response equations such that the storage of the largest four-index intermediates is eliminated, CD leads to a significant reduction in disk storage requirements, reduced I/O penalties, and an improved parallel performance. CD thus extends the scope of the systems that can be treated by (EOM-)CCSD methods, although analytic gradients in the framework of CD are needed to extend the applicability of (EOM-)CCSD methods in the context of geometry optimizations. This paper presents a formulation of analytic (EOM-)CCSD gradient within the CD framework and reports on the salient details of the corresponding implementation. The accuracy and the capabilities of analytic CD-based (EOM-)CCSD gradients are illustrated by benchmark calculations and several illustrative examples.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Exploring Hilbert space on a budget: Novel benchmark set and performance metric for testing electronic structure methods in the regime of strong correlation

This work explores the ability of classical electronic structure methods to efficiently represent (compress) the information content of full configuration interaction (FCI) wave functions. We introduce a benchmark set of four hydrogen model systems of different dimensionalities and distinctive electronic structures: a 1D chain, a 1D ring, a 2D triangular lattice, and a 3D close-packed pyramid. To assess the ability of a computational method to produce accurate and compact wave functions, we introduce the accuracy volume, a metric that measures the number of variational parameters necessary to achieve a target energy error. Using this metric and the hydrogen models, we examine the performance of three classical deterministic methods: (i) selected configuration interaction (sCI) realized both via an a posteriori (ap-sCI) and variational selection of the most important determinants, (ii) an a posteriori singular value decomposition (SVD) of the FCI tensor (SVD-FCI), and (iii) the matrix product state representation obtained via the density matrix renormalization group (DMRG). We find that the DMRG generally gives the most efficient wave function representation for all systems, particularly in the 1D chain with a localized basis. For the 2D and 3D systems, all methods (except DMRG) perform best with a delocalized basis, and the efficiency of sCI and SVD-FCI is closer to that of DMRG. For larger analogs of the models, the DMRG consistently requires the fewest parameters but still scales exponentially in 2D and 3D systems, and the performance of SVD-FCI is essentially equivalent to that of ap-sCI.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

3D simulations of vertical displacement events in tokamaks: A benchmark of M3D-C 1 , NIMROD, and JOREK

In recent years, the nonlinear 3D magnetohydrodynamic codes JOREK, M3D-C1, and NIMROD developed the capability of modeling realistic 3D vertical displacement events (VDEs) including resistive walls. In this paper, a comprehensive 3D VDE benchmark is presented between these state-of-the-art codes. The simulated case is based on an experimental NSTX plasma but with a simplified rectangular wall. There are differences between the physics models and numerical methods, and the VDE evolution leads to sensitivities on the initial conditions that cannot be avoided as can be done in edge localized modes (ELM) and sawtooth simulations (due to the non-cyclical nature of VDEs). Nonetheless, the comparison serves to quantify the level of agreement in the relevant quantities used to characterize disruptions, such as the 3D wall forces and energy decay. The results bring confidence regarding the use of the mentioned codes for disruption studies, and they distinguish aspects that are specific to the models used (e.g., reduced vs full MHD models). Here, the simulations show important 3D fea- tures for a NSTX plasma, such as the self-consistent evolution of the halo current and the origin of the wall forces. In contrast to other reduced MHD models based on an ordering in the aspect ratio, the ansatz-based JOREK reduced MHD model allows capturing many aspects of the 3D dynamics even in the spherical tokamak limit considered here.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗