Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “performance model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Battery Performance and Cost Model (BatPaC) Version 6.0

SF-26-016 The Battery Performance and Cost model (BatPaC) is a calculation method based on Microsoft Excel spreadsheets that has been developed at Argonne for estimating the performance and manufacturing cost of lithium-ion batteries for electric-drive vehicles including hybrid-electrics (HEV), plug-in hybrids (PHEVs) and pure electrics. BatPaC was first developed in 2007, was subsequently peer reviewed, and it has served Argonne researchers and the greater battery community in studying the impact of material properties on performance at the pack level. BatPaC has been updated and re-released multiple times since its original public release in 2011. This current version is BatPaC 6.0, which contains additional functionality needed to handle advances in automotive batteries, like the use of lithium metal and silicon anodes and the need to accommodate cell expansion and apply high levels of pressure.

KNEHR, KEVIN [Argonne National Laboratory (ANL), A↗
DOE OSTI2026

Outdoor Deployment Data for a Four-Terminal GaAs//Si Tandem Solar Mini-Module

This dataset contains the complete outdoor measurement and analysis data for a mechanically stacked, four-terminal (4T) gallium arsenide (GaAs)//silicon (Si) tandem solar mini-module deployed from October 2019 to January 2021 at the Solar Radiation Research Laboratory (SRRL) in Golden, Colorado, USA. The data support a performance modeling and degradation analysis framework for tandem photovoltaic devices, as described in the accompanying publication. The dataset includes: (1) current–voltage (J–V) characteristics of each sub-cell measured approximately every five minutes, with extracted performance parameters; (2) spectral irradiance from an EKO MS-710 WISER spectroradiometer, along with derived spectral mismatch ratios (SMR) and average photon energy (APE); (3) one-minute resolution meteorological data from the co-located SRRL weather station and GPS-derived precipitable water vapor (PWV); (4) pre-deployment laboratory characterization (external quantum efficiency, J–V curves, standard test conditions parameters); (5) outdoor-extracted temperature and PWV correction coefficients; and (6) PVcircuit equivalent-circuit simulation outputs used for model validation. Degradation rates of −4.1 ± 0.2 %/year (GaAs) and −2.5 ± 0.9 %/year (Si) were determined using a filtering and normalization methodology adapted for fixed-tilt tandem modules. All data are provided in open, portable formats (Apache Parquet, CSV, JSON) to enable full reproducibility of the published analysis.

14 SOLAR ENERGY↗
DOE OSTI2022

Battery Performance and Cost Modeling for Electric-Drive Vehicles (A Manual for BatPaC v5.0)

This manual details the fifth version of the Battery Performance and Cost (BatPaC v5.0) model developed at Argonne National Laboratory for lithium-ion battery packs used in transportation (file “BatPaC 5.0 2022-07-22.xlsm”). BatPaC is a publicly available model that performs a bottom-up lithium-ion battery design and cost calculation. The model designs the battery for a specified power, energy, and vehicle type (i.e., hybrid, plug-in hybrid, or full-electric). The cost of the designed battery is calculated by accounting for every step in the lithium-ion battery manufacturing process. The original model and manual were publicly peer-reviewed by battery experts assembled by the U.S. Environmental Protection Agency. This revised model and manual include changes made in response to comments received from users and the observed trajectory of the industry.

25 ENERGY STORAGE↗
DOE OSTI2026

VRN3P: Variational Recurrent Neural Network Based Net-Load Prediction under High Solar Penetration

This is the final technical report for the SETO-funded VRN3P project (PNNL# 76914). The goal of this project, led by Pacific Northwest National Laboratory (PNNL), in collaboration with Lawrence Livermore National Laboratory (LLNL) and Portland General Electric (PGE), was to develop and validate a deep variational recurrent neural network-based net-load prediction (VRN3P) framework for probabilistic time-series forecasting of day-ahead net-load under high solar penetration scenarios. The project team reports successful design of a novel probabilistic net-load forecasting architecture, comprising of a variational autoencoder and a recurrent neural network, which demonstrates 30% improvement in forecast performance, 60% improvement in training time, and consumes 44% less memory, when compared with conventional baseline models. The team tested the VRN3P model performance on GridLAB-D test-cases representing varying BTM solar penetration levels of 20%, 30%, and 50%, with integrated time-series net-load profiles provided by the utility partner (PGE). The VRN3P model demonstrate <2% hourly MAPE (averaged over the year) for day- ahead net-load forecast on the test scenario with 20% BTM solar. Transfer learning extension of the VRN3P model has demonstrated 8.33× speed-up in training, while still achieving acceptable forecast performance of 2.24% hourly MAPE on the 30% BTM solar penetration test-scenario. A preliminary version of the VRN3P GridAPPS-D™has been developed, along with a web-based interactive user-interface (named ‘Forte’) which has made available on GitHub for public use.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Data and scripts associated with the manuscript "Encoding Diel Hysteresis and the Birch Effect in Dryland Soil Respiration Models through Knowledge-Guided Deep Learning"

This package contains the data and scripts used in "Encoding Diel Hysteresis and the Birch Effect in Dryland Soil Respiration Models through Knowledge-Guided Deep Learning" (Jiang et al., 2022). The data.zip file contains the flux tower and automated chamber observations used for developing the deep learning model for modeling soil respiration. The scripts.zip file contains the Jupyter notebooks and python scripts for preprocessing the data, training the deep learning models, and postprocessing the results. The src.zip contains the source code for training the deep learning model, performing mutual information analysis, and plotting functions. The trained_models.zip contains multiple folders used for hosting the trained deep-learning models and the associated soil respiration predictions. The whole process is performed using python. We include the REAMD.md to document the python package requirements.Soil respiration in dryland ecosystems is challenging to model due to its complex interactions with environmental drivers. Knowledge-guided deep learning provides a much more effective means of accurately representing these complex interactions than traditional Q10-based models. Mutual information analysis revealed that future soil temperature shares more information with soil respiration than past soil temperature, consistent with their clockwise diel hysteresis. We explicitly encoded diel hysteresis, soil drying, and soil rewetting effects on soil respiration dynamics in a newly designed Long Short Term Memory (LSTM) model. The model takes both past and future environmental drivers as inputs to predict soil respiration. The new LSTM model substantially outperformed three Q10-based models and the Community Land Model when reproducing the observed soil respiration dynamics in a semi-arid ecosystem. The new LSTM model clearly demonstrated its superiority for temporally extrapolating soil respiration dynamics, such that the resulting correlation with observational data is up to 0.7 while the correlations of both Q10-based models and the Community Land Model (CLM) are less than 0.4. Our results underscore the high potential for knowledge-guided deep learning to replace Q10-based soil respiration modules in Earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Improving the accessibility and transferability of machine learning algorithms for identification of animals in camera trap images: MLWIC2

Motion-activated wildlife cameras (or “camera traps”) are frequently used to remotely and noninvasively observe animals. The vast number of images collected from camera trap projects has prompted some biologists to employ machine learning algorithms to automatically recognize species in these images, or at least filter-out images that do not contain animals. These approaches are often limited by model transferability, as a model trained to recognize species from one location might not work as well for the same species in different locations. Furthermore, these methods often require advanced computational skills, making them inaccessible to many biologists. We used 3 million camera trap images from 18 studies in 10 states across the United States of America to train two deep neural networks, one that recognizes 58 species, the “species model,” and one that determines if an image is empty or if it contains an animal, the “empty-animal model.” Our species model and empty-animal model had accuracies of 96.8% and 97.3%, respectively. Furthermore, the models performed well on some out-of-sample datasets, as the species model had 91% accuracy on species from Canada (accuracy range 36%–91% across all out-of-sample datasets) and the empty-animal model achieved an accuracy of 91%–94% on out-of-sample datasets from different continents. Our software addresses some of the limitations of using machine learning to classify images from camera traps. By including many species from several locations, our species model is potentially applicable to many camera trap studies in North America. We also found that our empty-animal model can facilitate removal of images without animals globally. We provide the trained models in an R package (MLWIC2: Machine Learning for Wildlife Image Classification in R), which contains Shiny Applications that allow scientists with minimal programming experience to use trained models and train new models in six neural network architectures with varying depths.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Improving E3SM Land Model Photosynthesis Parameterization via Satellite SIF, Machine Learning, and Surrogate Modeling

The parameterization of key photosynthesis parameters is one of the key uncertain sources in modeling ecosystem gross primary productivity (GPP). Solar-induced chlorophyll fluorescence (SIF) offers a good proxy for GPP since it marks the actual process of photosynthesis; while machine learning (ML) provides a robust approach to model the GPP-SIF relationship. Here, we trained the boosted regressing tree (BRT) and the Random Forest ML models with Greenhouse Gases Observing Satellite SIF data and in situ GPP observations from 49 eddy covariance towers. These trained ML GPP-SIF models were fed into the Energy Exascale Earth System Model (E3SM) Land Model (ELM) to generate ELM-simulated global SIF estimates, which were then benchmarked against satellite SIF observations with a surrogate modeling approach. Our results indicated good modeling performance of the ML-based GPP-SIF relationship. The ELM model when fed with the ML GPP-SIF models also can well predict the spatial-temporal variations in SIF. We also found high model accuracy for the surrogate modeling. Model parameter sensitivity analysis suggested that the fraction of leaf nitrogen in RuBisCO (flnr) is the most sensitive parameter to the SIF; other sensitive parameters include the Ball-Berry stomatal conductance slope (mbbopt) and the vcmax entropy (vcmaxse). The posterior uncertainty in simulated GPP was greatly reduced after benchmarking, and the model produced improved spatial patterns of mean GPP relative to FLUXCOM GPP. Our integrated approach provides a new avenue for improving land models and using remote-sensing SIF, which can be further improved in the future with more ground- and satellite-based observations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

CovTransformer: A transformer model for SARS-CoV-2 lineage frequency forecasting

With hundreds of SARS-CoV-2 lineages circulating in the global population, there is an ongoing need for predicting and forecasting lineage frequencies and thus identifying rapidly expanding lineages. Accurate prediction would allow for more focused experimental efforts to understand pathogenicity of future dominating lineages and characterize the extent of their immune escape. Here, we first show that the inherent noise and biases in lineage frequency data make a commonly-used regression-based approach unreliable. To address this weakness, we constructed a machine learning model for SARS-CoV-2 lineage frequency forecasting, called CovTransformer, based on the transformer architecture. We designed our model to navigate challenges such as a limited amount of data with high levels of noise and bias. We first trained and tested the model using data from the UK and the USA, and then tested the generalization ability of the model to many other countries and US states. Remarkably, the trained model makes accurate predictions two months into the future with high levels of accuracy both globally (in 31 countries with high levels of sequencing effort) and at the US-state level. Our model performed substantially better than a widely used forecasting tool, the multinomial regression model implemented in Nextstrain, demonstrating its utility in SARS-CoV-2 monitoring. Assuming a newly emerged lineage is identified and assigned, our test using retrospective data shows that our model is able to identify the dominating lineages 7 weeks in advance on average before they became dominant. Overall, our work demonstrates that transformer models represent a promising approach for SARS-CoV-2 forecasting and pandemic monitoring.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Importance of different parameterization changes for the updated dust cycle modeling in the Community Atmosphere Model (version 6.1)

Abstract. The Community Atmosphere Model (CAM6.1), the atmospheric component of the Community Earth System Model (CESM; version 2.1), simulates the life cycle (emission, transport, and deposition) of mineral dust and its interactions with physio-chemical components to quantify the impacts of dust on climate and the Earth system. The accuracy of such quantifications relies on how well dust-related processes are represented in the model. Here we update the parameterizations for the dust module, including those on the dust emission scheme, the aerosol dry deposition scheme, the size distribution of transported dust, and the treatment of dust particle shape. Multiple simulations were undertaken to evaluate the model performance against diverse observations, and to understand how each update alters the modeled dust cycle and the simulated dust direct radiative effect. The model–observation comparisons suggest that substantially improved model representations of the dust cycle are achieved primarily through the new more physically-based dust emission scheme. In comparison, the other modifications induced small changes to the modeled dust cycle and model–observation comparisons, except the size distribution of dust in the coarse mode, which can be even more influential than that of replacing the dust emission scheme. We highlight which changes introduced here are important for which regions, shedding light on further dust model developments required for more accurately estimating interactions between dust and climate.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Measurement of Wind Loading on Heliostats at the Crescent Dunes Power Plant: An Overview

The cost of solar collectors constitutes almost one third of the total cost of a CSP plant. One of the ongoing challenges in the design of these collectors is wind loading on mirrors, support structures, and drives. A particular challenge is dynamic wind loading, caused by the turbulent wind flow. To date, the design of solar collector structures has relied on wind tunnel experiments and numerical simulations that do not entirely capture the dynamic effects observed at scale. The CSP industry has shown increased interest in validating the idealized assumptions with measurements obtained in operational settings to improve wind load assumptions and increase reliability and cost-efficiency of the collector design. Further, performance models need realistic assumptions about wind loading and its impact on optical performance. In a parabolic trough field campaign, NREL successfully collected a wealth of long-term, high-resolution wind and loads data [2], that are publicly available and that can be used for the above-mentioned purposes. Heliostats are impacted differently by wind than parabolic troughs, due to their different shape, size, and field layout. To study the impact of wind and turbulence on heliostats, we initiated another field campaign in an operational power- tower plant, Crescent Dunes, in Nevada, USA. In this work, we present an overview of the measurements, first results, and potential implications of wind driven loads on Heliostats.

concentrated solar power↗
DOE OSTI2020

Evaluating three evapotranspiration estimates from model of different complexity over China using the ILAMB benchmarking system

The land surface models range in complexity of terrestrial evapotranspiration, yet it is unknown how model complexity translates to accuracy of modeled evapotranspiration estimates. Here, we use the International Land Model Benchmarking system to assess ET estimates from three models of varying complexity driven by the same forcing datasets: an earth system model, a terrestrial biosphere model, and a stand-alone ET model. The performance assessment includes both temporal and spatial evaluation, and different plant functional types across China. Our results indicate that the most complex model, an earth system model, performed best against the benchmarking datasets and metrics. Terrestrial biosphere model performed best in simulating inter-annual variability of ET, while earth system model performed best in simulating the seasonal cycle. The more complex models (earth system model and terrestrial biosphere model) perform better in forest, shrub and crop ecosystems, while the simpler model (stand-alone ET model) perform better in grass ecosystems. Our study demonstrates the impact of model complexity on ET estimates and highlights directions for future ET model improvements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Dry in-pile fracture test (DRIFT) for separate-effects validation of ceramic fuel fracture models

Fracture is an important component of nuclear fuel behavior, and significant efforts have been invested into developing fuel performance models that are capable of accurately representing fracture. Usable data on the process of fracture propagation in nuclear fuel under realistic service conditions are very limited. To address this need, a series of separate-effects experiments were developed and performed at Idaho National Laboratory's Transient Reactor Test (TREAT) facility. These experiments employ a heat sink to radially remove heat from the fuel in a manner that approximates the effect of coolant in an operating light-water reactor (LWR). The test holder for these experiments is known as the Dry In-pile Fracture Test (DRIFT). A series of experiments employing DRIFT and TREAT were performed to provide data on the extent and nature of fracture in fresh fuel at various points during a ramp to full power. Novel aspects of these experiments include the way they employ a heat sink to replicate steady-state LWR conditions, as well as the use of fiber optic sensors for in-reactor thermal instrumentation. Details on the development of this experiment, experimental conditions, and resulting data (including in situ thermal measurements and post-irradiation imaging of fracture) are provided in this work. LWR-equivalent powers ranging from 10 to 25 kW/m were tested using this apparatus. Cracking was visible at all power levels, with increasing cracking extent as the power level increased, although there was little difference in the cracking between the two highest-power tests, which had LWR-equivalent powers of 20 and 25 kW/m.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Deep-learning-based domain adaptation for cavity fault prediction at Jefferson Laboratory

Superconducting radio-frequency (SRF) cavities are the core components of the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab, providing high-power electron beams for nuclear physics experiments. The facility comprises 418 SRF cavities, and any fault in these cavities can lead to interruptions in the electron beam supply. Cavity faults are the leading cause of beam trips in CEBAF. Predicting and mitigating those faults before onset can help maintain normal operation. Existing models face challenges in distinguishing between normal and fault signals when changes occur in the underlying time-series data, from changes in control software, operational parameters, or the environment. This work proposes a deep learning domain adaptation model that leverages transfer learning to address fault prediction challenges by improving accuracy. The model is trained and fine-tuned using a dataset collected for faulty and normal operation using a data acquisition system in CEBAF. Our deep learning-based domain adaptation model achieves a prediction accuracy of 89.61% of the fault and normal signals. The developed model effectively predicts normal running signals compared to the baseline approach without domain adaptation. This capacity is essential for the fault prediction task in the CEBAF because of heavily imbalanced data containing vast amounts of normal signals. The model performs well for predicting faults several hundred milliseconds before the fault onset compared to other models where no adaptation is applied. Incorporating deep learning-based domain adaptation techniques will significantly improve the fault prediction performance.

Rahman, Md Monibor [Old Dominion Univ., Norfolk, V↗
DOE OSTI2021

PreMevE Update: Forecasting Ultra-Relativistic Electrons Inside Earth's Outer Radiation Belt

Energetic electrons inside Earth's Van Allen belts pose a major radiation threat to spaceborne electronics that often play vital roles in modern society. Ultra-relativistic electrons with energies greater than or equal to two megaelectron-volt (MeV) are of particular interest, and thus forecasting these ≥2 MeV electrons has a significant meaning to all space sectors. Here, we update the latest development of the predictive model for MeV electrons in the outer radiation belt. The new version, called PREdictive MEV Electron (PreMevE)-2E, forecasts ultra-relativistic electron flux distributions across the outer belt, with no need for in situ measurements of the trapped MeV electron population except at the geosynchronous orbit (GEO). Model inputs include precipitating electrons observed in low-Earthorbits by NOAA satellites, upstream solar wind speeds and densities from solar wind monitors, as well as ultra-relativistic electrons measured by one Los Alamos GEO satellite. We evaluated 32 supervised machine learning models that fall into four different classes of linear and neural network architectures, and successfully tested ensemble forecasting by using groups of top-performing models. All models are individually trained, validated, and tested by in situ electron data from NASA's Van Allen Probes mission. It is shown that the final ensemble model outperforms individual models at most L-shells, and this PreMevE-2E model can provide 25-h (~1-day) and 50-h (~2-day) forecasts with high mean performance efficiency and correlation values. Our results also suggest that this new model is dominated by nonlinear components at L-shells <~4 for ultra-relativistic electrons, different from the dominance of linear components for 1 MeV electrons as previously discovered.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

A High-Pressure Shock Tube Study of Hydrogen and Ammonia Addition to Natural Gas for Reduced Carbon Emissions in Power Generation Gas Turbines

Ignition delay times from undiluted mixtures of natural gas (NG)/H2/Air and NG/NH3/Air were measured using a high-pressure shock tube at the University of Central Florida. The combustion temperatures were experimentally tested between 1000-1500 K near a constant pressure of 25 bar. Mixtures were kept undiluted to replicate the same chemistry pathways seen in gas turbine combustion chambers. Recorded combustion pressures exceeded 200 bar due to the large energy release, hence why these were performed at the high-pressure shock tube facility. The data is compared to the predictions of the NUIGMech 1.1 mechanism for chemical kinetic model validation and refinement. An exceptional agreement was shown for stoichiometric conditions in all cases but strayed at lean and rich equivalence ratios, especially in the lower temperature regime of H2 addition and all temperature ranges of the baseline NG mixture. Hydrogen addition also decreased ignition delay times by nearly 90%, while NH3 fuel addition made no noticeable difference in ignition delay time. NG/NH3 exhibited similar chemistry to pure NG under the same conditions, which is shown in a sensitivity analysis, demonstrating hydrogen chemistry to be dominant in NG/H2 mixtures and hydrocarbon chemistry to be dominant in NG/NH3 mixtures. The reaction CH3 + O2 = CH3O + O is identified and suggested as a possible modification target to improve model performance. Increasing the robustness of chemical kinetic models via experimental validation will directly aid in designing next-generation combustion chambers for use in gas turbines, which in turn will greatly lower global emissions and reduce greenhouse effects.

Pierro, Michael↗
DOE OSTI2023

Uncertain role of clouds in shaping summertime atmosphere-sea ice connections in reanalyses and CMIP6 models

Downwelling longwave radiation (DLR) driven by the atmospheric and cloud conditions in the troposphere is suggested to be a dominant factor to determine the summertime net surface energy budget over the Arctic Ocean and thus plays a key role to shape the September sea ice. We use reanalyses and the self-organizing map (SOM) method to distinguish CMIP6 model performance in replicating the observed strong atmosphere-DLR connection. We find all models can reasonably simulate the linkage between key atmosphere variables and the clear sky DLR but behave differently in replicating the atmosphere-DLR connection due to cloud forcing. In ERA5 and strongly coupled models, tropospheric high pressure is associated with decreased clouds in the mid- and high-levels and increased clouds near the surface. This out-of-phase structure indicates that DLR cloud forcing is nearly neutral, making the clear sky DLR more important to bridge JJA circulation to late-summer sea ice. In MERRA-2 and weakly coupled models, tropospheric clouds display a vertically homogeneous reduction; the cloud DLR is thus strongly reduced due to the cooling effect, which partially cancels out the clear sky DLR and makes the total DLR less efficient to translate circulation forcing to sea ice. The differences of cloud vertical distribution in CMIP6 appear to be differentiated by circulation related relative humidity. Therefore, a better understanding of the discrepancy of different reanalyses and remote sensing products is critical to comprehensively evaluate simulated interactions among circulation, clouds, sea ice and energy budget at the surface in summer.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Dissipation Scaled Internal Wave Drag in a Global Heterogeneously Coupled Internal/External Mode Total Water Level Model

This study showcases a global, heterogeneously coupled total water level system wherein salinity and temperature outputs from a coarser-resolution (~12 km) ocean general circulation model are used to calculate density-driven terms within a global, higher-resolution (~2.5 km) depth-averaged total water level model. We demonstrate that the inclusion of baroclinic forcing in the barotropic model requires modification of the internal wave drag term to prevent excess degradation of tidal results compared to the barotropic model. By scaling the internal tide dissipation by an easy to calculate dissipation ratio, the resulting heterogeneously coupled model has complex root mean square errors (RMSE) of 2.27 cm in the deep ocean and 12.16 cm in shallow waters for the M 2 tidal constituent. While this represents a 10%–20% deterioration as compared to the barotropic model, the improvements in total water level prediction more than offset this degradation. Global median RMSE compared to observations of total water levels, 30-day sea levels, and non-tidal residuals improve by 1.86 (18.5%), 2.55 (42.5%), and 0.36 (5.3%) cm respectively. The drastic improvement in model performance highlights the importance of including density-driven effects within global hydrodynamic models and will help to improve the results of both hindcasts and forecasts in modeling extreme and nuisance flooding. With only an 11% increase in model run time compared to the fully barotropic total water level model, this approach paves the way for high resolution coastal water level and flood models to be used alongside climate models, improving operational forecasting of total water levels.

Blakely, Coleman Peter [University of Notre Dame, ↗
DOE OSTI2026

Systematic Benchmarking of Climate Models: Methodologies, Applications, and New Directions

As climate models become increasingly complex, there is a growing need to comprehensively and systematically assess model performance with respect to observations. Given the increasing number and diversity of climate model simulations in use, the community has moved beyond simple model intercomparison and toward developing methods capable of benchmarking a large number of simulations against a suite of climate metrics. Here, we present a detailed review of evaluation and benchmarking methods and approaches developed in the last decade, focusing primarily on scientific implications for Coupled Model Intercomparison Project (CMIP) simulations and CMIP6 results that contributed to the Intergovernmental Panel on Climate Change (IPCC) Sixth Assessment Report (AR6). Based on this review, we explain the resulting contemporary philosophy of model benchmarking, and provide clear distinctions and definitions of the terms model verification, process validation, evaluation, and benchmarking. While significant progress has been made in model development based on systematic evaluation and benchmarking efforts, some climate system biases still remain. The development of open‐source community software packages has played a fundamental role in identifying areas of significant model improvement and bias reduction. We review the key features of several software packages that have been commonly used over the past decade to evaluate and benchmark global and regional climate models. Additionally, we discuss best practices for the selection of evaluation and benchmarking metrics and for interpreting the obtained results, the importance of selecting suitable sources of reference data and accurate uncertainty quantification.

Environmental sciences↗