Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,783 records · Page 99

The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

We describe the outcome of a data challenge conducted as part of the Dark Machines (https://www.darkmachines.org) initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims to detect signals of new physics at the Large Hadron Collider (LHC) using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 billion simulated LHC events corresponding to 10\, fb^{-1} 10 f b − 1 of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

SERGHEI (SERGHEI-SWE) v1.0: a performance-portable high-performance parallel-computing shallow-water solver for hydrology and environmental hydraulics

The Simulation EnviRonment for Geomorphology, Hydrodynamics, and Ecohydrology in Integrated form (SERGHEI) is a multi-dimensional, multi-domain, and multi-physics model framework for environmental and landscape simulation, designed with an outlook towards Earth system modelling. At the core of SERGHEI's innovation is its performance-portable high-performance parallel-computing (HPC) implementation, built from scratch on the Kokkos portability layer, allowing SERGHEI to be deployed, in a performance-portable fashion, in graphics processing unit (GPU)-based heterogeneous systems. In this work, we explore combinations of MPI and Kokkos using OpenMP and CUDA backends. In this contribution, we introduce the SERGHEI model framework and present with detail its first operational module for solving shallow-water equations (SERGHEI-SWE) and its HPC implementation. This module is designed to be applicable to hydrological and environmental problems including flooding and runoff generation, with an outlook towards Earth system modelling. Its applicability is demonstrated by testing several well-known benchmarks and large-scale problems, for which SERGHEI-SWE achieves excellent results for the different types of shallow-water problems. Finally, SERGHEI-SWE scalability and performance portability is demonstrated and evaluated on several TOP500 HPC systems, with very good scaling in the range of over 20 000 CPUs and up to 256 state-of-the art GPUs.

58 GEOSCIENCES↗

A framework to evaluate machine learning crystal stability predictions

The rapid adoption of machine learning in various scientific domains calls for the development of best practices and community agreed-upon benchmarking tasks and metrics. We present Matbench Discovery as an example evaluation framework for machine learning energy models, here applied as pre-filters to first-principles computed data in a high-throughput search for stable inorganic crystals. We address the disconnect between (1) thermodynamic stability and formation energy and (2) retrospective and prospective benchmarking for materials discovery. Alongside this paper, we publish a Python package to aid with future model submissions and a growing online leaderboard with adaptive user-defined weighting of various performance metrics allowing researchers to prioritize the metrics they value most. To answer the question of which machine learning methodology performs best at materials discovery, our initial release includes random forests, graph neural networks, one-shot predictors, iterative Bayesian optimizers and universal interatomic potentials. We highlight a misalignment between commonly used regression metrics and more task-relevant classification metrics for materials discovery. Accurate regressors are susceptible to unexpectedly high false-positive rates if those accurate predictions lie close to the decision boundary at 0 eV per atom above the convex hull. The benchmark results demonstrate that universal interatomic potentials have advanced sufficiently to effectively and cheaply pre-screen thermodynamic stable hypothetical materials in future expansions of high-throughput materials databases.

Riebesell, Janosh↗

LibERI—A portable and performant multi-GPU accelerated library for electron repulsion integrals via OpenMP offloading and standard language parallelism

A portable and performant graphics processing unit (GPU)-accelerated library for electron repulsion integral (ERI) evaluation, named LibERI, has been developed and implemented via directive-based (e.g., OpenMP and OpenACC) and standard language parallelism (e.g., Fortran DO CONCURRENT). Offloaded ERIs consist of integrals over low and high contraction s, p, and d functions using the rotated-axis and Rys quadrature methods. GPU codes are factorized based on previous developments with two layers of integral screening and quartet presorting. In this work, the density screening is moved to the GPU to enhance the computational efficacy for large molecular systems. Here, the L-shells in the Pople basis set are also separated into pure S and P shells to increase the ERI homogeneity and reduce atomic operations and the memory footprint. LibERI is compatible with any quantum chemistry drivers supporting the MolSSI Driver Interface. Benchmark calculations of LibERI interfaced with the GAMESS software package were carried out on various GPU architectures and molecular systems. The results show that the LibERI performance is comparable to other state-of-the-art GPU-accelerated codes (e.g., TeraChem and GMSHPC) and, in some cases, outperforms conventionally developed ERI CUDA kernels (e.g., QUICK) while fully maintaining portability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Evaluation of the response to electrons and pions in the scintillating fiber and lead calorimeter for the future electron-ion collider

The performance of the Baby Barrel Electromagnetic Calorimeter (Baby BCAL) — a small-scale lead-scintillating-fiber (Pb/ScFi) prototype of the GlueX Barrel Electromagnetic Calorimeter (BCAL) — was tested in a dedicated beam campaign at the Fermilab Test Beam Facility (FTBF). This study provides a benchmark for the Pb/ScFi component of the future Barrel Imaging Calorimeter (BIC) in the ePIC detector at the Electron-Ion Collider (EIC). The detector response to electrons and pions was studied at beam energies between 4 and 10 GeV, extending previous GlueX tests to a higher energy regime. The calibrated detector exhibits good linearity within uncertainties, and its electron energy resolution meets EIC requirements. The data further constrain the constant term in the energy resolution to below 1.9%, improving upon previous constraints at lower energies. Simulations reproduce key features of the electron and pion data within the limitations of the collected dataset and the FTBF test environment. Electron-pion separation in the test beam setup was analyzed using multiple methods, incorporating varying degrees of beam-related effects. The inclusion of longitudinal shower profile information enhanced the separation performance, underscoring its relevance for the full-scale BIC in ePIC. These results provide essential benchmarks for the Pb/ScFi section of the future BIC, validating detector simulations and guiding optimization strategies for electron-pion discrimination.

47 OTHER INSTRUMENTATION↗

Quantum anomaly detection for collider physics

We explore the use of Quantum Machine Learning (QML) for anomaly detection at the Large Hadron Collider (LHC). In particular, we explore a semi-supervised approach in the four-lepton final state where simulations are reliable enough for a direct background prediction. This is a representative task where classification needs to be performed using small training datasets - a regime that has been suggested for a quantum advantage. We find that Classical Machine Learning (CML) benchmarks outperform standard QML algorithms and are able to automatically identify the presence of anomalous events injected into otherwise background-only datasets.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Peeking into the next decade in large-scale structure cosmology with its Effective Field Theory

After the successful full-shape analyses of BOSS data using the Effective Field Theory of Large-Scale Structure, we investigate what upcoming galaxy surveys might achieve. We introduce a “perturbativity prior” that ensures that loop terms are as large as theoretically expected, which is effective in the case of a large number of EFT parameters. After validating our technique by comparison with already-performed analyses of BOSS data, we provide Fisher forecasts using the one-loop prediction for power spectrum and bispectrum for two benchmark surveys: DESI and MegaMapper. We find overall great improvements on the cosmological parameters. In particular, we find that MegaMapper (DESI) should obtain at least a 12σ (2σ) evidence for non-vanishing neutrino masses, bound the curvature Ω k to 0.0012 (0.012), and primordial inflationary non-Gaussianities as follows: f NL loc. to ± 0.26 (3.3), f NL eq. to ±16 (92), f NL orth. to ± 4.2 (27). Such measurements would provide much insight on the theory of Inflation. We investigate the limiting factor of shot noise and ignorance of the EFT parameters.

cosmological parameters from LSS↗

Beam-dump ceiling and its experimental implications: The case of a portable experiment

We generalize the nature of the so-called beam-dump “ceiling” beyond which the improvement on the sensitivity reach in the search for fast-decaying mediators dramatically slows down, and we point out its experimental implications that motivate tabletop-sized beam-dump experiments for the search. Light (bosonic) mediators are well-motivated new-physics particles, as they can appear in dark-sector portal scenarios and models to explain various laboratory-based anomalies. Due to their low mass and feebly interacting nature, beam-dump-type experiments, utilizing high-intensity particle beams, can play a crucial role in probing the parameter space of such visibly decaying mediators—in particular, the “prompt decay” region, where the mediators feature relatively large coupling and mass. We present a general and semianalytic proof that the ceiling effectively arises in the prompt-decay region of an experiment and show its insensitivity to data statistics, background estimates, and systematic uncertainties, considering a concrete example, the search for axion-like particles interacting with ordinary photons at three benchmark beam facilities: PIP-II at FNAL, and SPS and LHC-dump at CERN. We then identify optimal criteria to perform a cost-effective and short-term experiment to reach the ceiling, demonstrating that very short-baseline compact experiments enable access to the parameter space unreachable thus far.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

FFTX-IRIS: Towards Performance Portability and Heterogeneity for SPIRAL Generated Code

FFTX-IRIS is a dynamic system to efficiently utilize novel heterogeneous platforms. This system links two next-generation frameworks, FFTX and IRIS, to navigate the complexity of different hardware architectures. FFTX provides a runtime code generation framework for high-performance Fast Fourier Transform kernels. IRIS runtime provides portability and multi-device heterogeneity, allowing computation on any available compute resource. Together, FFTX-IRIS enables code generation, seamless portability, and performance without user involvement. We show the design of the FFTX-IRIS system along with an evaluation of various small FFT benchmarks. We also demonstrate multi-device heterogeneity of FFTX-IRIS with a larger stencil application.

Rao, Sanil↗

Rolling Horizon Based Temporal Decomposition for the Offline Pickup and Delivery Problem with Time Windows

The offline pickup and delivery problem with time windows (PDPTW) is a classical combinatorial optimization problem in the transportation community, which has proven to be very challenging computationally. Due to the complexity of the problem, practical problem instances can be solved only via heuristics, which trade-off solution quality for computational tractability. Among the various heuristics, a common strategy is problem decomposition, that is, the reduction of a large-scale problem into a collection of smaller sub-problems, with spatial and temporal decompositions being two natural approaches. While spatial decomposition has been successful in certain settings, effective temporal decomposition has been challenging due to the difficulty of stitching together the sub-problem solutions across the decomposition boundaries. In this work, we introduce a novel temporal decomposition scheme for solving a class of PDPTWs that have narrow time windows, for which it is able to provide both fast and high-quality solutions. We utilize techniques that have been popularized recently in the context of online dial-a-ride problems along with the general idea of rolling horizon optimization. To the best of our knowledge, this is the first attempt to solve offline PDPTWs using such an approach. To show the performance and scalability of our framework, we use the optimization of paratransit services as a motivating example. Due to the lack of benchmark solvers similar to ours (i.e., temporal decomposition with an online solver), we compare our results with an offline heuristic algorithm using Google OR-Tools. In smaller problem instances (with an average of 129 requests per instance), the baseline approach is as competitive as our framework. However, in larger problem instances (approximately 2,500 requests per instance), our framework is more scalable and can provide good solutions to problem instances of varying degrees of difficulty, while the baseline algorithm often fails to find a feasible solution within comparable compute times.

Kim, Youngseo↗

Round 3 Update of Stress Testing Results for Organic Light-Emitting Diode Panels and Luminaires

Organic light-emitting diode (OLED) sources are a potential solid-state lighting technology for use in indoor lighting applications. Some of the advantages offered by OLEDs include thin profiles, low glare, diffuse lighting, and unique form factors. The U.S. Department of Energy (DOE) has released six technical reports focused on OLED technologies to provide information and analysis to the lighting industry. These reports include the evaluation of two different field deployment sites, a market analysis, a tear-down analysis, and two independent assessments of the performance of select commercially available OLED products. This report builds on earlier DOE efforts with OLED technology by updating information about previously benchmarked commercial OLED products. Specifically, this report provides updated findings about the robustness and reliability of commercial OLED products.

42 ENGINEERING↗

U.S. Solar Photovoltaic System and Energy Storage Cost Benchmark: Q1 2021 [Slides]

Our benchmarking method includes bottom-up accounting for all necessary system and projectdevelopment costs incurred when installing residential, commercial, and utility-scale systems, and it models the Q1 2021 costs for such systems, excluding any previous supply agreements or contracts. In general, we attempt to model the typical installation techniques and business operations from an installed-cost perspective, and our benchmarks are national averages. The residential PV-only benchmark and the commercial rooftop PV-only benchmark average costs by inverter type (string inverters, string inverters with direct current [DC] optimizers, and microinverters), weighted by inverter market share. The residential PV-only benchmark is further averaged across small installer and national integrator business models, weighted by market share. All benchmarks include variations—accounting for the differences in size, equipment, and operational use (particularly for storage)—that are currently available in the marketplace. All benchmarks assume nonunionized construction labor; residential and commercial PV systems predominantly use nonunionized labor, and the type of labor required for utility-scale PV systems depends heavily on the development process. All benchmarks assume the use of monofacial monocrystalline silicon PV modules. Benchmarks using cadmium telluride or bifacial modules could result in significantly different results. The data in this annual benchmark report inform the formulation of and track progress toward the U.S. Department of Energy (DOE) Solar Energy Technologies Office’s Government Performance and Reporting Act cost targets.

14 SOLAR ENERGY↗

The Cost of Saving Electricity: A Multi-Program Cost Curve for Programs Funded by U.S. Utility Customers

This study analyzed the cost performance of electricity efficiency programs implemented by 116 investor-owned utilities between 2009 and 2015 in 41 states, representing about three-quarters of the total spending on U.S. efficiency programs. We applied our typology to characterize efficiency programs along several dimensions (market sector, technology, delivery approach, and intervention strategy) and report the costs incurred by utilities and other program administrators to achieve electricity savings as a result of the programs. Such cost performance data can be used to compare relative costs of different types of efficiency programs, evaluate efficiency options alongside other electricity resources, benchmark local efficiency programs against regional and national cost estimates, and assess the costs of meeting state efficiency policies. The savings-weighted average cost of saved electricity for the period was $0.025/kilowatt-hour (kWh). The cost of saved electricity for programs that targeted residential customers was $0.021/kWh, compared to $0.025/kWh for programs for commercial and industrial customers. Ultimately, we developed an aggregate program savings “cost curve” for the actual electricity efficiency resource during the period that provides insights into the relative costs of various types of efficiency programs and the savings contribution of each program type to the efficiency resource at a national level.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Integrated Steady-State System Package for Nuclear Thermal Propulsion Analysis Using Multi-Dimensional Thermal Hydraulics and Dimensionless Turbopump Treatment

Nuclear thermal propulsion is an evolving technology that can be utilized for long-distance space travel. This technology yields the advantage of a high thrust and specific impulse, but requires an examination of the potential design adjustments necessary to enhance its feasibility. The development of nuclear thermal propulsion requires a comprehensive understanding of the system-level behavior during transient and steady-state operation. This paper extends our previous research by including the proper handling of turbomachinery with multi-channel thermal hydraulic simulations only for steady-state solutions. The system-level approach presented here enables the treatment of the turbopump components through non-dimensional analysis that eliminates the assumption of constant efficiencies. All the other components within the system (e.g., reflector and core) can be discretized to multiple channels and layers, in which the full thermal hydraulic solution is established. The approach chosen here enables the realistic modeling of the propellant flow within the expander cycle by capturing the pressure losses, mass flow rate splits, and enthalpy gain for various operational conditions. The verification of the package is completed through point comparisons of previous investigations into similar system designs. Furthermore, sensitivity studies are used to benchmark the capabilities of the package and investigate solution variations due to the perturbation of operational conditions and regimes. The sensitivity studies performed here are important to capture variation in flow characteristics (e.g., temperature, pressure, mass flow rates) for different design objectives such as the thrust and specific impulse. This work demonstrates that system-level simulations lacking multi-channel capability and proper turbomachinery treatment may yield higher uncertainties in understanding the engine’s response and characteristics to changing various requirements. This is extremely important when screening the design space of such propulsion systems and when transient simulations are required.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A Prolog Emulator

This note describes an efficient software emulator for the Warren Abstract Machine (WAM) Prolog architecture. The version of the WAM implemented is called Lcode. The Lcode emulator, written in C, executes the 'naive reverse' benchmark at 3900 LIPS. The emulator is one of a set of tools used to measure the memory-referencing characteristics and performance of Prolog programs. These tools include a compiler, assembler, and memory simulators. An overview of the Lcode architecture is given here, followed by a description and listing of the emulator code implementing each Lcode instruction. This note will be of special interest to those studying the WAM and its performance characteristics. In general, this note will be of interest to those creating efficient software emulators for abstract machine architectures.

Tick, Evan↗

Dynamic object management for distributed data structures

In distributed-memory multiprocessors, remote memory accesses incur larger delays than local accesses. Hence, insightful allocation and access of distributed data can yield substantial performance gains. The authors argue for the use of dynamic data management policies encapsulated within individual distributed data structures. Distributed data structures offer performance, flexibility, abstraction, and system independence. This approach is supported by data from a trace-driven simulation study of parallel scientific benchmarks. Experimental data on memory locality, message count, message volume, and communication delay suggest that data-structure-specific data management is superior to a single, system-imposed policy.

Totty, Brian K.↗

Cryogenic Propellant Long-Term Storage With Zero Boil-Off

Significant boil-off losses from cryogenic propellant storage systems in long-duration space mission applications result in additional propellant and larger tanks. The potential propellant mass loss reductions with the Zero Boil-off (ZBO) concept are substantial; therefore, further exploration through technology programs has been initiated within NASA. A large-scale demonstration of the ZBO concept has been devised utilizing the Marshall Space Flight Center (MSFC) Multipurpose Hydrogen Test Bed (MHTB) along with a cryo-cooler unit. The ZBO concept consists of an active cryo-cooling system integrated with traditional passive thermal insulation. The cryo-cooler is interfaced with the MHTB and spraybar recirculation/mixer system in a manner that enables thermal energy removal at a rate that equals the total tank heat leak. The liquid hydrogen (LH2) is withdrawn from the tank, passed through a heat exchanger, and then the chilled liquid is sprayed back into the tank through a spraybar. The test series will be performed over a 20-30 day period. Tests will be conducted at multiple fill levels to demonstrate concept viability and to provide benchmark data to be used in analytical model development. In this paper the test set-up and test procedures are presented.

Hedayat, Ali↗

Automated Generation of Message-Passing Programs: An Evaluation Using CAPTools

Scientists at NASA Ames Research Center have been developing computational aeroscience applications on highly parallel architectures over the past ten years. During that same time period, a steady transition of hardware and system software also occurred, forcing us to expend great efforts into migrating and re-coding our applications. As applications and machine architectures become increasingly complex, the cost and time required for this process will become prohibitive. In this paper, we present the first set of results in our evaluation of interactive parallelization tools. In particular, we evaluate CAPTool's ability to parallelize computational aeroscience applications. CAPTools was tested on serial versions of the NAS Parallel Benchmarks and ARC3D, a computational fluid dynamics application, on two platforms: the SGI Origin 2000 and the Cray T3E. This evaluation includes performance, amount of user interaction required, limitations and portability. Based on these results, a discussion on the feasibility of computer aided parallelization of aerospace applications is presented along with suggestions for future work.

Hribar, Michelle R.↗