Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Molten Salt Reactor Experiment Benchmark Evaluation

As one of the six advanced reactor concepts selected by the 2002 Generation IV roadmap as a technology meriting future research, Molten Salt Reactors (MSRs) attracted broad attention and multiple private and public entities are working towards its commercialization. Facing stringent regulatory requirements, validations of computational codes used to calculate and prove the safety characteristics of MSRs play a key role. This project developed the world-first, MSR-related reactor physics benchmark based on a series of zero-power experiments conducted at the Molten Salt Reactor Experiment (MSRE). The benchmark was successfully reviewed by the International Reactor Physics Experiment Evaluation Project (IRPhEP) committee and is available in the IRPhEP handbook starting form the 2019 edition. To benchmark experimental data from the MSRE, a three-dimensional high-fidelity model was developed using the Monte Carlo neutron transport code Serpent 2 and ad-hoc methods have been implemented to account for the unique feature of fuel salt motion. The calculated effective multiplication factor, k eff , for the first criticality experiment, when 235 U was progressively added to the fuel salt in order to achieve criticality with stationary salt and isothermal conditions, was 1.02132 (±3 pcm). The total uncertainty for experimental k eff was estimated to be 420 pcm. The calculated k eff is 2.154% larger than the experimental and benchmark model value, which is approximately five times the benchmark model uncertainty. It is to be noted that, for systems containing large volume of graphite (or other carbonaceous materials), Monte Carlo codes tend to overestimate the keff of the benchmark model by 1% to 2%. The bias is possibly due to uncertainties in the impurity content of the graphite blocks, uncertainty in the neutron capture cross section of carbon or uncertainty in the properties of nuclear-grade graphite. The calculated reactivity coefficient of 235 U concentration (0.2228 ± 0.0014, represented as the change of reactivity over the relative change of 235 U mass in loop) matches well with the experimental value (0.223 ± 0.006), strengthening the confidence on the accurate representation of the fuel salt composition in the MSRE benchmark. Additional reactivity effect calculations, including the control rod bank worth, reactivity effects of fuel circulation and isothermal and fuel temperature coefficients show good agreement with the experimental values (within 1σ) as well.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Q1 2023 U.S. Solar Photovoltaic System and Energy Storage Cost Benchmarks With Minimum Sustainable Price Analysis Data File

The U.S. Department of Energy's (DOE's) Solar Energy Technologies Office (SETO) aims to accelerate the advancement and deployment of solar technology in support of an equitable transition to a decarbonized economy no later than 2050, starting with a decarbonized power sector by 2035. Its approach to achieving this goal includes driving innovations in technology, hardware, and soft cost reductions to make solar affordable and accessible for all. As part of this effort, SETO must track solar cost trends so it can focus its research and development (R&D) on the highest-impact activities. The benchmarks in this report are bottom-up cost estimates of all major inputs to PV and energy storage system installations. Bottom-up costs are based on national averages and do not necessarily represent typical costs in all local markets. Like last year's report, this year's report includes two distinct sets of benchmarks: minimum sustainable price (MSP) benchmarks and modeled market price (MMP) benchmarks. MSP benchmarks can be interpreted as the minimum price a company needs to charge to remain financially solvent in the long term based on the minimum sustainable prices of all inputs including minimum sustainable profit margins. MMP benchmarks can be interpreted as the actual cash sales price a company charges in the given benchmark period. These simplified estimates are useful for tracking technological progress, but they do not reflect all experiences. In fact, no individual estimate under any approach can reflect the diversity of the PV and storage manufacturing and installation industries.

14 SOLAR ENERGY↗
DOE OSTI2022

Beyond Energy Efficiency: A clustering approach to embed demand flexibility into building energy benchmarking

The intermittency of carbon-free renewables and the demand changes associated with the widespread push for electrifying the transportation and building sectors provides an opportunity for buildings to go beyond energy efficiency and push towards providing demand flexibility to the electricity grid. The duality of energy efficiency and demand flexibility is necessary for success in a sustainable and reliable energy transition. Current building energy benchmarking models are limited in their ability to integrate concepts of demand flexibility and/or utilize granular smart meter data. Thus, current benchmarking methods are focused annual energy usage and fail to incorporate how the time of use of energy consumption impacts emissions in a quickly changing energy grid. Without a more comprehensive view of energy usage and associated real-time emissions, current benchmarking methods are unlikely to realize the full decarbonization potential of buildings. New emerging data streams provide an opportunity to develop a new generation of benchmarking energy models that embed dimensions of energy efficiency, grid interactivity, and demand flexibility into their analysis. In this paper, we propose a four-step method for embedding grid interactivity and demand flexibility into building benchmarking models that utilizes emerging building and time-series electricity data streams. We first engineer features to produce a mix-type dataset that encompasses many attributes of grid-interactive and efficient buildings, and then we apply K-medoids using Gower's Distance to produce peer-group clusters. We apply the method to a case study of 306 primary and secondary schools in southern California, USA. The results show that the method effectively clusters buildings by attributes of demand flexibility and energy efficiency. The clustering results reveal patterns in inefficient building operations and demand inflexibility at the building peer group level. In conclusion, the interpretation of clusters can serve as an integrated energy efficiency and demand flexibility benchmarking model and inform performance-specific policy targeting for buildings that go beyond traditional efficiency measures.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Discrete fracture network model benchmarks developed and applied in a DECOVALEX-2023 repository performance assessment study

This study presents newly developed benchmarks for modeling flow and transport within discrete fracture networks (DFNs) and useful methods for analyzing the results. The new benchmarks are designed to test modeling approaches for use in probabilistic performance assessment models of deep geologic repositories in fractured rock. The benchmarks simulate flow and transport through a 1 km 3 block of fractured rock. The first simulates migration of a short pulse of tracer through a simple network of four intersecting fractures. The second adds 1089 stochastically generated fractures. The third changes the pulse to a continuous point source. Evaluation of model performance relies on moment analysis and comparison of the results of different models. The expected nondimensional first moment of the conservative tracer for each benchmark is 1. The benchmarks were simulated by teams from Canada, Czechia, Germany, Korea, Sweden, Taiwan, and the United States as part of a DECOVALEX-2023 study (decovalex.org). The teams used various approaches, including explicit DFN modeling, DFN upscaling to an equivalent continuous porous medium (ECPM), and a combination of both methods. Transport mechanisms are modeled using either the advection-dispersion equation or particle tracking. Results demonstrate strong agreement among the models in breakthrough behavior up to the 75th percentile. Significant deviations in first moments and well-clustered outputs led to the identification of inaccuracies in several models. Such findings exemplify the benefit of exercising these benchmarks and using the presented methods to test DFN flow and transport models.

Benchmark↗
DOE OSTI2020

Benchmark probabilistic solar forecasts: Characteristics and recommendations

We illustrate and compare commonly used benchmark, or reference, methods for probabilistic solar forecasting that researchers use to measure the performance of their proposed techniques. A thorough review of the literature indicates wide variation in the benchmarks implemented in probabilistic solar forecast studies. To promote consistent and sensible methodological comparisons, we implement and compare ten variants from six common benchmark classes at two temporal scales: intra-hourly forecasts and hourly resolution forecasts. Using open-source Surface Radiation Budget Network (SURFRAD) data from 2018, these benchmark methods are compared using proper probabilistic metrics and common diagnostic tools. Practical implementation issues, such as the impact of missing data and applicability for operational forecasting, are also discussed. Furthermore, we make recommendations for practitioners on the appropriate selection of benchmark methods to properly showcase state-of-the-art improvements in forecast reliability and sharpness. All code and open-source data are available on Github for reproducibility and for other researchers to apply the same benchmark methods to their own data.

14 SOLAR ENERGY↗
DOE OSTI2025

Depletion Benchmark of the AFIP-7 Experiment in the Advanced Test Reactor

Reactor physics depletion benchmarks for low-enriched uranium fuel are limited in number. In particular, there is very limited data for LEU benchmarks for U-10Mo (Uranium-10% Molybdenum) plate fuel developed for use in U.S. high-performance research reactors (USHPRR). USHPRR includes the Advanced Test Reactor (ATR), Advanced Test Reactor Critical Facility (ATR-C), High Flux Isotope Reactor (HFIR), University of Missouri Research Reactor (MURR), Massachusetts Institute of Technology Reactor (MITR), and National Bureau of Standards Reactor (NBSR) at the National Institute of Science and Technology. These reactors are fueled with high-enriched uranium dispersed fuel in a silicon/aluminum matrix. In support of conversion to a HALEU fuel, qualification of U-10Mo formed into a monolithic foil is being performed. Fuel qualification involves irradiated fueled specimens in the ATR. The irradiation tests provide an opportunity to benchmark depletion capabilities of reactor physics codes in support of the ATR operation, as well as develop benchmarks that can be used by other institutions to benchmark other reactor physics codes. This report documents the development of a benchmark model of the irradiation of the ATR Full -size plate In center flux trap Position 7 (AFIP-7) experiment.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Verifying MCNP Models of the TEX High 240 Plutonium Benchmark

Computational modeling programs are invaluable tools that allow us to understand systems, safely develop new processes, and make reliable predictions about future designs. However, the effectiveness of these codes is limited by the degree to which their parameters match the real world. In the field of nuclear engineering, cross section data is one of these vital parameters. Accurate cross section data on important fissile and fissionable isotopes promotes the design of safer and more efficient fabrication, transportation, storage, and stockpiling of nuclear fuel. Unfortunately, there are knowledge gaps in data on key isotopes. In 2011, a multinational meeting hosted by the US Department of Energy Nuclear Criticality Safety Program ranked the priority of certain cross section data needs. In response, Lawrence Livermore National Lab (LLNL) designed the Thermal and Epithermal eXperiment (TEX) series of benchmark experiments. Benchmark experiments are used to validate current cross section data. They validate data by comparing the results of an actual experiment to the predicted results from a computational model. The data a benchmark applies to depends on the isotope and energy range the experiment’s neutron multiplication factor ( k eff ) is most sensitive to. The development and testing of the TEX High 240 Plutonium Benchmark will help validate 240 Pu cross section data. The configuration and materials of this benchmark are designed to be most sensitive to 240 Pu's intermediate energy range (from 0.625 ev to 100 keV ). MCNP® models of the assembly have been developed by LLNL and the results have been written in the final design report. In order for the discrepancies between benchmark models and experiments to be attributed to cross section inaccuracies, the accuracy of the models needs to be verified. The goal of this project is to verify of the results of LLNL's modeling by creating a new set of MCNP models and comparing the results.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Depletion benchmark for a high-assay low-enriched uranium fuel experiment in the advanced test reactor

Reactor physics depletion benchmarks for high-assay low-enriched uranium (HALEU) fuel are limited in number. In particular, there is limited data for HALEU benchmarks for U-10Mo (uranium-10% molybdenum) plate fuel that is being developed for use in the United States’ high performance research reactors including the Advanced Test Reactor (ATR), Advanced Test Reactor Critical Facility (ATR-C), High Flux Isotope Reactor (HFIR), Massachusetts Institute of Technology Reactor (MITR), University of Missouri Research Reactor (MURR), National Bureau of Standards Reactor (NBSR). These six reactors currently operate with highly enriched uranium dispersed fuel in an aluminum matrix. In support of conversion to a HALEU fuel, qualification of U-10Mo formed into a monolithic foil is being performed. Fuel qualification involves irradiating fuel specimens in the ATR. The irradiation tests provide an opportunity to benchmark depletion capabilities of reactor physics codes in support of the ATR operation, as well as develop benchmarks that can be used by other institutions to benchmark other reactor physics codes. This paper documents the development of a benchmark model of the irradiation of the ATR Full-size plate In center flux trap Position 7 (AFIP-7) experiment using the depletion codes MC21 and Advanced Dimensional Depletion for Engineering of Reactors (ADDER).

Nielsen, Joseph W. [Idaho National Laboratory (INL↗
DOE OSTI2025

BuildingQA: A Benchmark for Natural Language Question Answering over Building Knowledge Graphs

Graph-based representations of building metadata using ontologies like Brick are vital for smart building applications, but querying them remains a challenge for practitioners. Knowledge Graph Question Answering (KGQA) systems, meant to retrieve answers from natural language questions, traditionally require large-scale training data, making them ill-suited for the specialized and data-scarce building domain. The advent of Large Language Models (LLMs) offers a paradigm shift, enabling zero-shot natural language querying without building/domain-specific training. Yet, there is no standardized benchmark for building-specific KGQA which can guide and validate research in this area. To address this gap, our work makes three primary contributions. First, we introduce the BuildingQA Benchmark Dataset, constructed through a multi-stage process of collecting practitioner data, augmenting it with LLMs for linguistic diversity, and curating a final set of 188 questions across 4 buildings. Second, we characterize the benchmark's complexity and ambiguity, introducing a novel method to quantify its "lexical gap" and providing a four-stage diagnostic framework for analyzing how systems fail. Third, we benchmark zero-shot LLM-powered KGQA systems to establish baseline performance and analyze their failure modes. Our evaluation reveals that top-performing systems achieve a maximum F1 score of only 0.38. This result does not indicate a failure of these powerful systems, but rather underscores the unique challenges posed by our benchmark. It demonstrates a critical performance gap, showing that current methods successful on general KGs struggle with the specific lexical and structural nuances of the building domain. BuildingQA1 thus provides the benchmark dataset and foundational analysis needed to drive the development of novel, domain-aware methods required to unlock the use of semantic data in buildings.

Mulayim, Ozan Baris↗
DOE OSTI2020

Validation of MCNP Critical Benchmark Models of PU-MET-FAST-016

A new centralized repository of high-quality MCNP models of critical benchmark experiments is currently under development at Los Alamos National Laboratory (LANL). The repository is the combined effort of the Nuclear Criticality Safety, Nuclear Data, and Monte Carlo code development/application organizations at LANL. The initial set of benchmark models in the repository are derived from the Whisper Suite provided with MCNP6.2 and are evaluated against the benchmark experiments described in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The goal is to build a single LANL benchmark collection that is up to date with the latest ICSBEP revision, has a formal review and revision process, is contained in an open-source repository, and utilizes new Python tools for improved input and output file review. This paper describes the validation of the models associated with PU-MET-FAST-016, “Flooded 3X3X3 Arrays of 3-kg Plutonium (Pu) Metal Cylinders – Phase I”. In this critical experiment, twenty-seven 3-kg Pu metal cylinders with a height of 46.33 ± 0.15 mm and diameter of 65.25 ± 0.05 mm were arranged in a cubic array. The Pu cylinders were sealed in aluminum cans, and three cans were placed in each of nine perforated aluminum sleeves, as shown in Fig. 1. The nine sleeves were supported by an aluminum frame that allowed for control of the lateral spacing of the cylinders, both in the X and Y directions. The benchmark experiment measured thirteen configurations. Of these configurations, six were modeled using MCNP.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

LANL Critical Benchmark Comparison Study and Subsequent Revision

As part of an international collaboration within the DOE Nuclear Criticality Safety Program (NCSP), LANL is involved in a comparison study to quantify differences in k-effective results from neutron transport simulations of critical benchmark experiments. The DOE NCSP Mission and Vision details the activity in which the French Institut De Radioprotection et De Sûreté Nucléaire (IRSN) leads the study with LANL and in conjunction with ORNL and LLNL to compare results of various neutron transport codes and nuclear data libraries to compute k-effective for ICSBEP benchmarks held in common by the entities. This report documents results obtained through partial completion of the overall effort with a focus on the changes made to LANL benchmarks modeled with MCNP6 using ENDF/B-VII.1 nuclear data that appeared to have discrepant results when compared with results of other codes. The feedback received through participation in the comparison collaboration has prompted an effort to review particular input files for benchmarks and revise when necessary. This report documents the results of review and revision of specific benchmarks highlighted as possibly discrepant in the comparison study. In addition, this effort prompted a new collaboration between LANL XCP and NCS Divisions in the development of a shared review/revision procedure and use of a new benchmark repository.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

IER-517: Molybdenum Optimized Benchmark System Demonstrating Integral Correlations (MOBY DICK)

Nuclear criticality experiments are essential to the validation of nuclear data used in simulation software. The quality of nuclear data becomes paramount as simulation software becomes more relied upon for criticality safety studies and designs of nuclear systems. To improve the quality of nuclear data, experimenters can design critical experiments that are sensitive to isotope reaction pairs in materials of interest. The efforts conducted by the Organisation for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA) Working Party on Nuclear Criticality Safety (WPNCS) Subgroup 8: Preservation of Expert Knowledge and Judgement Applied to Criticality Benchmarks (SG8) to categorize benchmarks according to their usefulness for nuclear data validation have been of great importance. Based on the OECD studies benchmark experiments included in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook are concisely used by nuclear data evaluators, criticality safety engineers and others to validate nuclear data and simulation results. A lack of benchmarks sensitive to molybdenum in the (ICSBEP), particularly in the intermediate range, was noted by Los Alamos National Laboratory (LANL), the French Institut de Radioprotection et de Sûreté Nucléaire (IRSN), and Y-12 National Security Site prompting them to submit a joint integral experiment request to the Nuclear Criticality Safety Program (NCSP) in 2019. The request included both HEU and Plutonium systems in order to validate differential nuclear data focusing on the intermediate energy range but also includes thermal and fast configurations. This document represents the preliminary design work for a series of molybdenum integral experiments known as Molybdenum Optimized Benchmark System Demonstrating Integral Correlations (MOBY DICK).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Machine Learning meets Algebraic Combinatorics: A Suite of Benchmark Datasets to Accelerate AI for Mathematics Research

The use of benchmark datasets has become an important engine of progress in machine learning (ML) over the past 15 years. Recently there has been growing interest in utilizing machine learning to drive advances in research-level mathematics. However, off-the-shelf solutions often fail to deliver the types of insights required by mathematicians. This suggests the need for new ML methods specifically designed with mathematics in mind. The question then is: what benchmarks should the community use to evaluate these? On the one hand, toy problems such as learning the multiplicative structure of small finite groups have become popular in the mechanistic interpretability community whose perspective on explainability aligns well with the needs of mathematicians. While toy datasets are a useful benchmark for initial work, they lack the scale, complexity, and sophistication of many of the principal objects of study in modern mathematics. To address this, we introduce a new collection of benchmark datasets, Algebraic Combinatorics Benchmarks (ACBench), representing either classic or open problems in algebraic combinatorics, a subfield of mathematics that studies discrete structures arising from abstract algebra. After describing the datasets, we discuss the challenges involved in constructing “good” mathematics benchmarks, describe baseline model performance, and discuss some of the insights these datasets can provide that may be of interest even to those who are not interested in mathematics research itself.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Ranking and Classifying AI Benchmarks

We created a set of standards to efficiently evaluate AI benchmarks through objective means. Although prevalent, especially in recent times, AI benchmarks have no single way to measure their effectiveness. The MLCommons team provided a set of criteria for evaluating benchmarks, although the criteria lacks a clearly defined set of evaluation rules. We created a rubric with preset factors to efficiently and objectively evaluate a benchmark s quality. We created a software framework for processing lists of benchmarks for visualization. The framework and rating system allows researchers to quickly check if their benchmarks are effective.

Shiraishi, Reece C. [Cornell U.]↗
DOE OSTI2025

A High-Fidelity Model of the Peach Bottom 2 Turbine-Trip Benchmark Using VERA

This work presents a high-fidelity simulation of the Peach Bottom turbine trip (PBTT) benchmark using the Virtual Environment for Reactor Applications (VERA), a multiphysics reactor modeling tool developed by the U.S. Department of Energy’s Consortium for Advanced Simulation of Light Water Reactors energy innovation hub. The PBTT benchmark, based on a 1977 transient event at the end of cycle 2 in a General Electric Type-4 boiling water reactor (BWR), is a critical test case for validating core physics models with thermal feedback during rapid reactivity events. VERA was employed to perform end-to-end, pin-resolved simulations from conditions at the beginning of cycle 1 through the turbine-trip transient, incorporating detailed neutron transport, fuel depletion, and subchannel thermal hydraulics. The simulation reproduced key benchmark observables with high accuracy: the peak power excursion occurred at 0.75 s, matching the scram time and closely aligning with the benchmark average of 0.742 s; the simulated maximum power spike was approximately 7600 MW, which is within 3% of the benchmark average of 7400 MW; and void-collapse dynamics were consistent with benchmark expectations. Reactivity predictions during cycles 1 and 2 remained within 1500 pcm and 400 pcm of criticality, respectively. These results confirm VERA’s ability to model complex coupled neutronic and thermal hydraulic behavior in a BWR turbine-trip transient, which will support its use in future studies of modeling dryout, fuel performance, and uncertainty quantification for transients of this type.

BWR↗
DOE OSTI2022

Analytical benchmark solution for 1-D neutron transport coupled with thermal conduction and material expansion

In this work we present fully analytical solutions for a class of finite, homogeneous, 1-D slab benchmark problems with nonlinear temperature feedback effects. The proposed class of benchmarks include multiplicative 1-D neutron transport (limited to quasistatic S{sub 2} with μ = ±1) coupled with thermal conduction, convection, Doppler broadening, and expansion effects along the length of the slab. This class of benchmark models, along with the corresponding analytical solutions, are valuable for validating multiphysics analysis frameworks that support coupled neutronics/thermal/structural calculations. Analytical solutions for the benchmarks are obtained by introducing an ansatz that the equilibrium flux and temperature distributions in the slab have the same shape. Specific values for the total microscopic cross section, σ{sub t,0}, and conductive heat transfer coefficient, h, that satisfy the assumed ansatz are then determined. A discussion of the procedure for generating benchmark models and analytical solutions is provided, along with numerical results for an example set of model parameters and thoughts on practical applications of the benchmarks for multiphysics code validation. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Status of the International Criticality Safety Benchmark Evaluation Project

The International Criticality Safety Benchmark Evaluation Project (ICSBEP) has continued its work generating evaluations of new and historical benchmark experiments since the last update to the nuclear criticality safety (NCS) community at the 12th International Conference on Nuclear Criticality Conference held in 2023. One additional version of the ICSBEP Handbook has been published since that update, and the Technical Review Group (TRG) held two in-person meetings to review and approve additional benchmarks. The 2022 and 2023 editions of the handbook were combined into one release (published in November 2024) and contained 13 new evaluations with 46 different configurations and two major revisions to existing evaluations. The 2024 version of the handbook, currently under publication review, will contain two new evaluations with 15 new configurations and one major revision to HEU-MET-FAST-028, the evaluation of Flattop with a uranium core. The ICSBEP TRG met again in person in April 2025 to review benchmarks for the 2025 ICSBEP Handbook and final comment resolution is currently ongoing. Many of the new benchmarks represent contemporaneous experiments that have been specifically optimized to provide validation cases relevant to the NCS community. One major area of focus for new critical experiments is to target the sparsely populated intermediate energy (or resonance) region. Another focus of many of the new benchmarks is to provide experiments sensitive to different materials, such as chlorine, hafnium, tantalum, titanium, molybdenum, chromium, and polymethyl methacrylate (PMMA, or Lucite). The ICSBEP continues to deliver high-quality, peer reviewed evaluations of integral experiments relevant to the nuclear data community.

HEU-MET-FAST-028↗
DOE OSTI2026

Intern-Artificial Intelligence Benchmarking

Benchmarks provide a standardized method for evaluating different AI models, enabling reproducibility and comparison between models, and facilitating scientific progress. As AI models continue to develop rapidly, incorporating new datasets, capabilities, and architectures becomes more complicated. Therefore, the current static benchmarks become increasingly irrelevant. The MLCommons team argues that to make AI benchmarks more relevant, it involves making the benchmarks themselves more dynamic, as well as technical innovations that make it easier for scientists and researchers at all levels to use and contribute to the benchmarks. The current progress in technical innovation is a software that allows for a detailed view of a collection of AI benchmarks to be output in various formats that are easily readable and accessible.

Krishnan, Anjay [Fermilab]↗