Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97

Benchmarking treewidth as a practical component of tensor network simulations

Tensor networks are powerful factorization techniques which reduce resource requirements for numerically simulating principal quantum many-body systems and algorithms. The computational complexity of a tensor network simulation depends on the tensor ranks and the order in which they are contracted. Unfortunately, computing optimal contraction sequences (orderings) in general is known to be a computationally difficult (NP-complete) task. In 2005, Markov and Shi showed that optimal contraction sequences correspond to optimal (minimum width) tree decompositions of a tensor network’s line graph, relating the contraction sequence problem to a rich literature in structural graph theory. While treewidth-based methods have largely been ignored in favor of dataset-specific algorithms in the prior tensor networks literature, we demonstrate their practical relevance for problems arising from two distinct methods used in quantum simulation: multi-scale entanglement renormalization ansatz (MERA) datasets and quantum circuits generated by the quantum approximate optimization algorithm (QAOA). We exhibit multiple regimes where treewidth-based algorithms outperform domain-specific algorithms, while demonstrating that the optimal choice of algorithm has a complex dependence on the network density, expected contraction complexity, and user run time requirements. We further provide an open source software framework designed with an emphasis on accessibility and extendability, enabling replicable experimental evaluations and future exploration of competing methods by practitioners.

97 MATHEMATICS AND COMPUTING↗

Benchmarking Parametric and Machine Learning Models for Genomic Prediction of Complex Traits

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e., ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e., feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

59 BASIC BIOLOGICAL SCIENCES↗

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages

The rapid evolution of large language models (LLMs) has opened new possibilities for automating various tasks in software development. This paper evaluates the capabilities of the LLaMA 2-70B model in automating these tasks for scientific applications written in commonly used programming languages. Using representative test problems, we assess the model's capacity to generate code, documentation, and unit tests, as well as its ability to translate existing code between commonly used programming languages. Our comprehensive analysis evaluates the compilation, runtime behavior, and correctness of the generated and translated code. Additionally, we assess the quality of automatically generated code, documentation, and unit tests. Here, our results indicate that while LLaMA 2-70B frequently generates syntactically correct and functional code for simpler numerical tasks, it encounters substantial difficulties with more complex, parallelized, or distributed computations, requiring considerable manual corrections. We identify key limitations and suggest areas for future improvements to better leverage AI-driven automation in scientific computing workflows.

97 MATHEMATICS AND COMPUTING↗

A Big Problem for Small Earthquakes: Benchmarking Routine Magnitudes and Conversion Relationships with Coda Envelope-Derived Mw in Southern Kansas and Northern Oklahoma

Earthquake magnitudes are widely relied upon measures of earthquake size. Although moment magnitude (M w ) has become the established standard for moderate and large earthquakes, difficulty in reliably measuring seismic moments for small (generally M w <4) earthquakes has meant that magnitudes for these events remain plagued by a patchwork of inconsistent measurement scales. Because of this, magnitudes of small earthquakes and statistics derived from them can be biased. Furthermore, because small earthquakes are much more numerous than large ones, many applications, such as seismic hazard modeling, depend critically on analysis of events characterized by magnitudes other than M w . Therefore, to assess this problem, we apply coda envelope analysis to reliably determine moment magnitudes for a case study of small earthquakes from northern Oklahoma and southern Kansas. Not surprisingly, we find significant differences among M L , m bLg , and M w for M ~2–4 earthquakes examined here. More troublingly, we find that relations designed to convert other magnitudes to M w , which are relied upon for important applications such as seismic hazard analysis, often increase rather than decrease this bias for our dataset. In our case study, we find that converted magnitudes can result in a systematic bias sometimes exceeding 0.5 magnitude units, a difference that typically corresponds to a factor of ~3 in seismicity rate. Moreover, we find a correspondingly large bias in Gutenberg–Richter b-values, controlled primarily by inaccurate magnitude scaling in the conversion relationships. Although this study focuses on a relatively small geographic area, we can expect that similar issues exist with varying severity in other regions. Therefore, magnitudes of small earthquakes and their associated statistics, including seismicity rates and b-values, should be treated with caution.

58 GEOSCIENCES↗

Benchmarking of Touschek Beam Lifetime Calculations for the Advanced Photon Source

Particle loss from Touschek scattering is one of the most significant issues faced by present and future synchrotron light source storage rings. For example, the predicted, Touschek-dominated beam lifetime for the Advanced Photon Source (APS) Upgrade lattice in 48-bunch, 200-mA timing mode is only ~ 2 h. In order to understand the reliability of the predicted lifetime, a series of measurements with various beam parameters was performed on the present APS storage ring. This paper first describes the entire process of beam lifetime measurement, then compares measured lifetime with the calculated one by applying the measured beam parameters. The results show very good agreement.

43 PARTICLE ACCELERATORS↗

Hardware-In-the-Loop Benchmarking Setup for Phasor Based Control Validation

Phasor Based Control is a novel approach to controlling Distributed Energy Resources that aims at relieving various constraints that arise in the distribution grid. It is a two-layer control system with a supervisory control that coordinates distributed controllers to reach voltage phasor targets. The distributed controllers use local synchrophasor measurements and operate as feedback controllers. This control method is currently under development with several algorithms being under consideration for both the central and ditributed components. In this paper, we present the experimental setup that was prepared to prototype a hardware implementation and validate the control method in Hardware-In-the-Loop.

Baudette, Maxime↗

JetNet: A Python package for accessing open datasets and benchmarking machine learning methods in high energy physics

JetNet is a Python package that aims to increase accessibility and reproducibility for machinelearning (ML) research in high energy physics (HEP), primarily related to particle jets. Basedon the popular PyTorch ML framework, it provides easy-to-access and standardized interfacesfor multiple heterogeneous HEP datasets and implementations of evaluation metrics, lossfunctions, and more general utilities relevant to HEP.

97 MATHEMATICS AND COMPUTING↗

Benchmarking Exercises To Validate The Updated ELLWF GoldSim Slit Trench Model

The Savannah River National Laboratory (SRNL) results of the 2008 Performance Assessment (PA) (WSRC, 2008) sensitivity/uncertainty analyses conducted for the trenches located in the EArea LowLevel Waste Facility (ELLWF) were subject to review by the United States Department of Energy (U.S. DOE) Low-Level Waste Disposal Facility Federal Review Group (LFRG) (LFRG, 2008). LFRG comments were generally approving of the use of probabilistic modeling in GoldSim to support the quantitative sensitivity analysis. A recommendation was made, however, that the probabilistic models be revised and updated to bolster their defensibility. SRS committed to addressing those comments and, in response, contracted with Neptune and Company to rewrite the three GoldSim models.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Y12 7s/Clean Sweep Benchmarking: LANL Overview [Slides]

Overview of LANL Site: 1. Approach to Environmental Stewardship. 2. Programs Supporting Environmental Goals: UMCs, Moratorium Metals, Site Cleanup, Workspace Stewardship, P2, Site Sustainability Plan. 3. Questions?

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Agricultural Industry Advanced Vehicle Technology: Benchmark Study for Reduction in Petroleum Use

Diesel use on farms in the United States has remained relatively constant since 1985, decreasing slightly in 2009, which may be attributed to price increases and the economic recession. During this time, the United States’ harvested area also has remained relatively constant at roughly 300 million acres. In 2010, farm diesel use was 5.4% of the total United States diesel use. Crops accounting for an estimated 65% of United States farm diesel use include corn, soybean, wheat, hay, and alfalfa, respectively, based on harvested crop area and a recent analysis of estimated fuel use by crop. Diesel use in these cropping systems primarily is from tillage, harvest, and various other operations (e.g., planting and spraying) (Figure 3). Diesel efficiency is markedly variable due to machinery types, conditions of operation (e.g., soil type and moisture), and operator variability. Farm diesel use per acre has slightly decreased in the last two decades and diesel is now estimated to be less than 5% of farm costs per acre. This report will explore current trends in increasing diesel efficiency in the farm sector. The report combines a survey of industry representatives, a review of literature, and data analysis to identify nascent technologies for increasing diesel efficiency.

33 ADVANCED PROPULSION SYSTEMS↗

Differential Die-Away Instrument: Report on Benchmark Measurements and Comparison with Simulation for the Effects of Neutron Poisons

In this report, new experimental data and MCNPX simulation results of the differential die-away (DDA) instrument response to the presence of neutron absorbers are evaluated. In our previous fresh nuclear fuel experiments and simulations, no neutron absorbers or poisons were included in the fuel definition. These new results showcase the capability of the DDA instrument to acquire data from a system that better mimics spent nuclear fuel.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Embedded Volttron Specification - Benchmarking Small Footprint Compute Devices for Volttron

An embedded system is a small footprint computing unit that typically serves a specific purpose closely associated with measurements and control of hardware devices. These units are designed for reasonable durability and operations in a wide range of operating conditions. Some embedded systems support real-time operations and can demonstrate high levels of reliability. Many have failsafe mechanisms built to handle graceful shutdown of the device in exception conditions. The available memory, processing power, and network connectivity of these devices are limited due to the nature of their specific-purpose design and intended application. Industry practice is to carefully design the software for the available hardware capability to suit desired deployment needs. Volttron is an open source agent development and deployment platform designed to enable researchers to interact with devices and appliances without having to write drivers themselves. Hosting Volttron on small footprint embeddable devices enables its demonstration for embedded use. This report details the steps required and the experience in setting up and running Volttron applications on three small footprint devices: the Intel Next Unit of Computing (NUC), the Raspberry Pi 2, and the BeagleBone Black. In addition, the report also details preliminary investigation of the execution performance of Volttron on these devices.

97 MATHEMATICS AND COMPUTING↗

Consumer Views on Plug-in Electric Vehicles -- National Benchmark Report

Vehicle manufacturers, U.S. Department of Energy laboratories, universities, private researchers, and organizations from around the globe are pursuing advanced vehicle technologies that aim to reduce the consumption of petroleum in the form of gasoline and diesel. In order to make these technologies most appealing to the marketplace, they must take consumer sentiment into account. This report details study findings of broad American public sentiments toward issues that surround the advanced vehicle technologies of plug-in electric vehicles and is supported by the U.S. Department of Energy's Vehicle Technology Office in alignment with its mission to develop and deploy these technologies to improve energy security, provide mobility flexibility, reduce transportation costs, and increase environmental sustainability.

33 ADVANCED PROPULSION SYSTEMS↗